Lecture 5 Detailed Explanation

This lecture replaces complexity on a fixed sample by complexity on a tree where history changes the future. This report covers all six source pages, expands the two main proofs, the tree quantifiers, and binary relabeling. Minimax and general contraction remain explicitly external theorems. Source: lecture5.pdf (local source, not published online). Companion: full handout. Previous: Lecture 4; next: Lecture 6.

1 Reading Map and Notation

Lectures 2–4 built the chain from generalization error to Rademacher complexity, covers, and combinatorial dimensions under i.i.d. sampling. This lecture retains that strategy while rebuilding the parts affected by time dependence. The result bounds the performance of an existing low-regret strategy; it does not yet give an executable original-game strategy. Lecture 7 begins algorithm design.

Topic	Source pages	What to understand
Game and duality	1–2	Who chooses distributions first and what is observed
Theorem 1	2–3	One comparison produces conditional centering
Trees and Theorem 2	4–5	Why ghost samples cannot be exchanged immediately
Lemmas 1–2	5–6	Tree relabeling versus general contraction

Notation: $D$ is the decision space, $F \subseteq D$ the comparator class, $z_{t}$ the outcome, $y_{t}$ the learner decision, and $q_{t}, p_{t}$ their distributions. In supervised learning $z_{t} = (x_{t}, y_{t})$ ; a learner decision may be a prediction function, so do not confuse it with the true label $y_{t}$ .

Assume the required variables, suprema, and conditional kernels are measurable, expectations exist, and the invoked minimax conditions hold. Use arbitrarily accurate approximate extrema when extrema are not attained. Finite examples avoid these technical issues; linearity alone does not justify minimax for arbitrary infinite classes.

2 What Regret Compares

Reg (F, n) = t = 1 \sum n ℓ (y_{t}, z_{t}) - f \in F in f t = 1 \sum n ℓ (f, z_{t}), V^{seq} = π in f environment sup E [Reg / n] .

The comparator is one fixed function chosen afterward on the same realized outcome sequence. It is neither a different optimum each round nor a counterfactual sequence generated by inserting that comparator into the environment. Adaptivity makes both the realized outcomes and hindsight comparator random, so the expectation surrounds the entire difference, including its infimum. Total regret $o (n)$ means vanishing average regret.

The current $z_{t}$ may use past history but cannot be chosen after observing the current private draw $y_{t}$ . If the environment could negate a realized random prediction, any randomized algorithm would err every round. The handout uses an environment that does not anticipate the current private randomness.

3 Why Repeated ERM Can Fail

Here is a supplementary example used again in Lecture 7. There are two constant experts; FTL chooses +1 on cumulative-loss ties. Present labels −1,+1,−1,+1,…:

Round	Prediction	Label	Loss
1	+1	−1	1
2	−1	+1	1
3	+1	−1	1
4	−1	+1	1

After each even round the two experts tie again, so induction shows FTL errs every round. For even $n$ , each comparator errs $n /2$ times and regret is $n /2$ . The sequence is even fixed in advance. The leader reacts too sharply to a one-unit loss difference; optimal historical loss need not imply stable future behavior. This does not say FTL fails on every structured problem.

4 What Minimax Exchanges

First consider the one-round zero-one game with two labels. If the learner predicts +1 with probability $q$ , expected loss is $1 - q$ against +1 and $q$ against −1. Minimizing the maximum gives $q = 1/2$ and value $1/2$ . In the dual, the environment chooses probability $p$ of +1; knowing $p$ , the learner picks the more likely label and incurs $min (p, 1 - p)$ . Maximizing this again gives $1/2$ .

q \in [0, 1] in f max {q, 1 - q} = p \in [0, 1] sup min {p, 1 - p} = \frac{1}{2} .

Allowing environmental randomization does not change the primal value: once the learner’s mixed strategy is fixed, expectation is linear in the outcome distribution and cannot exceed the worst pure outcome. Minimax exchanges the current pair of mixed-strategy optimizations. In the dual the objective is linear in $q_{t}$ , so a pure decision suffices. Multiple rounds require backward induction through continuation values, not moving every infimum to the front.

A dual strategy receives $p_{t}$ ; a primal strategy does not. “Predict the most common label under the announced distribution” is therefore not a primal algorithm. Later algorithm design addresses this gap between a learnability bound and an implementable strategy.

5 Theorem 1: From the Dual to Martingale Differences

Let $G_{t - 1} = σ (z_{1}, \dots, z_{t - 1})$ . For a fixed joint law, define conditional risk and optimal current risk by

A_{t} (f) = E_{z_{t}^{'} \sim P (\cdot ∣ z_{1 : t - 1})} ℓ (f, z_{t}^{'}), a_{t} = a \in D in f A_{t} (a) .

Generalizing the handout’s two-round equality chain gives the exact dual expression. Each learner decision affects only its current loss, while the comparator depends only on outcomes. Eliminate learner decisions backward from the final round; environmental choices become kernels from histories to distributions. Such kernels encode a joint law. Thus

V^{seq} (F, n) = \frac{1}{n} P sup E [t = 1 \sum n a_{t} - f \in F in f t = 1 \sum n ℓ (f, z_{t})] .

Replace the negative infimum by a supremum. Every fixed $f$ is an admissible decision, so $a_{t} \leq A_{t} (f)$ in each round. This is the only comparison step:

t \sum a_{t} - f in f t \sum ℓ (f, z_{t}) = f sup t \sum {a_{t} - ℓ (f, z_{t})} \leq f sup t \sum {A_{t} (f) - ℓ (f, z_{t})} .

Take expectations, the worst joint distribution, and divide by $n$ to obtain Theorem 1. The same $f$ is used throughout the sum. We never replace the supremum of the sum by a sum of per-round suprema; that further relaxation often produces a linear bound.

6 Why Zero Mean Is Not Enough

D_{t} (f) = A_{t} (f) - ℓ (f, z_{t}), E [D_{t} (f) ∣ G_{t - 1}] = 0.

For fixed $f$ , the cumulative difference is a martingale; independence is unnecessary because conditional centering suffices. But a hindsight-selected function uses the entire sample path. One cannot first set every mean to zero and then maximize. For a fair sign $ϵ$ , both $ϵ$ and $- ϵ$ have mean zero, yet their maximum is always 1.

f sup E X_{f} \leq E f sup X_{f}, max {E ϵ, E (- ϵ)} = 0 < 1 = E max {ϵ, - ϵ} .

As a supplementary check, suppose there are $K$ functions with losses in $[0, 1]$ . Conditional Hoeffding gives $E [e^{λ D_{t} (f)} ∣ G_{t - 1}] \leq e^{λ^{2} /8}$ . Iteration bounds the cumulative moment-generating function by $e^{n λ^{2} /8}$ . A log-sum-exp bound over the functions yields

E f max \frac{1}{n} t \sum D_{t} (f) \leq λ > 0 in f (\frac{ln K}{nλ} + \frac{λ}{8}) = \frac{ln K}{2 n} .

This anticipates Lecture 6: the key is concentration of each predictable process plus control of distinguishable behaviors in the class, not independence of all observations. Infinite classes will need tree covers.

7 A Tree Encodes All Possible Histories

Text

                     z1
               −1 /      \ +1
             z2(−1)      z2(+1)
           −1 / \ +1    −1 / \ +1
        z3(−,−) z3(−,+) z3(+,−) z3(+,+)

This explanatory sketch is supplementary; the source contains no figure. The entire tree can be written before random signs are drawn. Random left/right moves determine the visited nodes. A level- $t$ node can read only the first $t - 1$ signs: this is predictability. A depth- $n$ tree has $2^{n} - 1$ nodes, but a realization visits only $n$ . The final sign does not change visited nodes but still multiplies the last function value, explaining the source footnote.

R_{n}^{seq} (H) = z sup E_{ϵ} h \in H sup \frac{1}{n} t = 1 \sum n ϵ_{t} h (z_{t} (ϵ_{1 : t - 1})) .

The order is: choose one complete tree, draw the random path, then choose one function to fit the entire path. The root cannot see $ϵ_{1}$ , and $h$ cannot change each round. Statistical Rademacher complexity is the special case where all nodes at a level are identical. A fixed tree is not a fixed path, and a worst-case tree cannot foresee future signs.

8 The Symmetrization Trap

With independent samples, each original/ghost pair has the same role, so multiplying a difference by a random sign is equivalent to swapping the pair. Dependence breaks this. As a minimal example, let $z_{1}$ be a fair binary variable and $z_{2} = z_{1}$ . An independent $z_{1}^{'}$ is fair, but both second-round draws use the actual history, so $z_{2}^{'} = z_{2} = z_{1}$ .

Pr (z_{2} = z_{1}) = 1, Pr (z_{2} = z_{1}^{'}) = \frac{1}{2} .

Swapping the first-round variables without changing the second-round conditional law changes the joint distribution. This alone invalidates a general exchangeability argument, even if a particular loss happens to give equal values. Second-round draws are exchangeable given history, so work backward from the final round.

9 Theorem 2: Relax the Future Before Swapping the Past

Take two rounds and write the loss as $h_{f}$ . Pulling out ghost expectations uses convexity of the supremum: maximizing after averaging is no larger than averaging the maximum. Write $Δ_{t} (f) = h_{f} (z_{t}^{'}) - h_{f} (z_{t})$ , giving

E f sup {E^{'} h_{f} (z_{1}^{'}) - h_{f} (z_{1}) + E^{'} h_{f} (z_{2}^{'}) - h_{f} (z_{2})} \leq E f sup {Δ_{1} (f) + Δ_{2} (f)} .

Symmetrize the second round conditional on real history, then replace the conditional expectation over that pair with a supremum. Future choices no longer privilege the real first-round draw. Only now symmetrize the first round:

E_{z_{1}, z_{1}^{'}} E_{z_{2}, z_{2}^{'} ∣ z_{1}} E_{ϵ_{2}} f sup {Δ_{1} + ϵ_{2} Δ_{2}} \leq E_{z_{1}, z_{1}^{'}} z_{2}, z_{2}^{'} sup E_{ϵ_{2}} f sup {Δ_{1} + ϵ_{2} Δ_{2}} = E_{z_{1}, z_{1}^{'}} E_{ϵ_{1}} z_{2}, z_{2}^{'} sup E_{ϵ_{2}} f sup {ϵ_{1} Δ_{1} + ϵ_{2} Δ_{2}} \leq z_{1}, z_{1}^{'} sup E_{ϵ_{1}} z_{2}, z_{2}^{'} sup E_{ϵ_{2}} f sup {ϵ_{1} Δ_{1} + ϵ_{2} Δ_{2}} .

The second-round supremum is inside the expectation over $ϵ_{1}$ , so choices can depend on the first sign. This defines a depth-two tree. Split positive and negative parts using

f sup (A_{f} + B_{f}) \leq f sup A_{f} + f sup B_{f}

Recoding signs together with branches makes the two worst-tree values equal, giving the factor 2.

\frac{1}{n} z, z^{'} sup E_{ϵ} f sup t \sum ϵ_{t} {h_{f} (z_{t}^{'} (ϵ)) - h_{f} (z_{t} (ϵ))} \leq 2 R_{n}^{seq} (ℓ (F)) .

For general $n$ , repeat backward: introduce the final sign, remove the final pair’s preferential dependence on real history, then continue toward the root. Operators alternate between choosing nodes and drawing signs. Every observed prefix permits two continuations, producing a pair of binary trees. Split the positive and negative sums only once at the end: the factor is 2, not $2^{n}$ . This repeats the same operators without an extra independence assumption.

Together with Theorem 1 this gives $V^{seq} \leq 2 R^{seq} (ℓ (F))$ . Worst trees replace the original distribution because expectations were relaxed.

10 Removing Binary Loss Requires Relabeling the Tree

1 {f (x) \neq = y} = \frac{1 - y f ( x )}{2}, f (x), y \in {- 1, + 1} .

After substitution the constant term $\sum_{t} ϵ_{t} / (2 n)$ has mean zero. The remaining sign is $s_{t} = - ϵ_{t} y_{t} (ϵ_{< t})$ . Given the past, the label is fixed, so this only flips the current fair coin. A recursive inverse proves that the whole $s$ vector is jointly uniform and independent:

ϵ_{1} = - s_{1} y_{1}, ϵ_{2} = - s_{2} y_{2} (ϵ_{1}), ϵ_{t} = - s_{t} y_{t} (ϵ_{< t}) .

Inputs were indexed by $ϵ$ . After changing variables define $x_{t}^{'} (s_{< t}) = x_{t} (ϵ_{< t} (s_{< t}))$ . Otherwise one changes signs but forgets the associated data path. Consequently,

R^{seq} (ℓ (F); (x, y)) = \frac{1}{2} R^{seq} (F; x^{'}) .

Example: root input a with children b,c; root label +1 and second-level labels −1. The first sign flips and the second does not, so the new tree has root a and children c,b. If every label is +1, all levels are mirrored. After taking suprema, relabeled trees remain valid; the factor one-half cancels symmetrization’s factor 2.

11 Regression Contraction: Scope and Proof Boundary

The handout states general contraction as $G O (ln^{3/2} n)$ times function-class complexity and omits its proof. Nonlinear transformations change relative function behavior on adaptive trees, so the fixed-sample coordinatewise contraction argument cannot simply be reused.

Primary-source check: Rakhlin, Sridharan, and Tewari (2015), Corollary 5, gives bounded-class contraction with complexity at least $1/ n$ . Theorem 8 directly removes convex Lipschitz losses from the supervised game without that logarithm. These are external theorems, not proved here.

The aside that logarithms are unavoidable should not be overgeneralized. Block, Dagan, and Rakhlin (2021), Corollary 19, gives logarithm-free worst-tree contraction for their $(c, p)$ -bounded classes. This is not a universal fixed-tree statement for every label-dependent transformation, but the handout does not establish universal optimality of exponent $3/2$ .

Use $ln (e n)$ instead of bare $ln n$ to avoid a zero one-round upper bound. Outside a theorem’s scope, cite an applicable version rather than deleting assumptions. Squared loss also needs range restrictions: predictions and labels in $[- 1, 1]$ give derivative magnitude at most 4; unbounded labels do not permit this uniform constant.

12 Worked Check, Pitfalls, and Self-Test

Worked check. With two constant functions $F = {- 1, + 1}$ over two rounds, inputs do not matter. The sign sum is −2,0,2 with probabilities $1/4, 1/2, 1/4$ , so complexity is $E ∣ ϵ_{1} + ϵ_{2} ∣/2 = 1/2$ ; the zero-one loss-class complexity is $1/4$ . This checks the factor and shows that one function may be chosen after the whole path but cannot change each round.

Pitfalls. Dual is not automatically primal; martingale differences need not be independent; suprema do not freely commute with expectations; nodes cannot see current signs; a bound for every tree is stronger than one for a chosen tree; Lipschitz constants require an argument and range.

Self-test with answers.

Why does the root read no signs? There is no past in round one, and current randomness cannot be revealed.
Why retain the last sign? It multiplies the last term even though it selects no new node.
Where does Theorem 1 use inclusion? In $a_{t} \leq A_{t} (f)$ .
Why does the factor not multiply every round? Split the two tree sums only once at the end.
Why can the maximum have positive mean? Hindsight selection depends on data; use $ϵ, - ϵ$ .
What is missing for Lecture 6? Computable dimensions or covering bounds for tree complexity.

13 Suggested Reading Route

Spend 15 minutes connecting Sections 2, 4, and 7 to understand protocol and trees. Spend 25 minutes rewriting Sections 5 and 9 with dependencies marked. Spend another 15 minutes drawing the relabeling example and doing the self-test. Then read Lecture 6, distinguishing a fixed tree from representatives selected for each path.