Lecture 7 Detailed Explanation

This lecture turns learnability into algorithms: Halving converts mistakes into elimination, Hedge balances past performance through exponential weights, partial covers turn finite Littlestone dimension into an agnostic strategy, and Perceptron uses geometric margins for an efficient guarantee. Source: lecture7.pdf (local source, not published online); companion: full handout; previous: Lecture 6. Expected regret, deterministic mistake bounds, realizability, and running time are kept distinct.

1 What the Four Methods Solve

Method	Main condition	Guarantee	Implementation
Halving	Finite class, perfect classifier	Logarithmically many mistakes	Enumerate survivors
Hedge	Finite experts, bounded full-information losses	Expected square-root regret	Evaluate all expert losses
Generalized Halving and partial covers	Finite Littlestone dimension	At most d realizable mistakes; aggregate for agnostic learning	Dimension computation may be hard; large expert pool
Perceptron	Euclidean normalization and positive margin	Inverse-square-margin mistake bound	Inner product each round; vector update on mistakes

Source pages 1–3 cover finite classes, Lemma 1, and Theorem 1; pages 4–6 cover Algorithms 1–3, Lemma 2, and Theorems 2–3; pages 7–8 cover Assumption 1, Algorithm 4, and Theorem 4.

The environment cannot choose current losses after observing the current private random draw. Observing $z_{t}$ permits evaluation of all $ℓ (f, z_{t})$ : this is full information. If only the chosen expert’s loss is observed, the problem is bandit and needs different estimators; this proof does not directly apply.

2 Halving: A Mistake Eliminates at Least Half

Realizability means one fixed $f^{⋆}$ is correct in all rounds. The version space $V_{t} = {f : f (x_{s}) = y_{s}, s < t}$ never becomes empty. Arbitrarily choosing a survivor may eliminate only one per mistake, giving $K - 1$ . Majority vote ensures that a mistake eliminates at least half.

1 \leq ∣ V_{n + 1} ∣ \leq K 2^{- M} ⟹ M \leq lo g_{2} K .

Starting with 16 candidates, mistakes leave at most 8,4,2,1, so at most four mistakes occur. Since the comparator has zero loss, mistake count equals regret. A fixed tie label still eliminates half on errors. The majority-vote function may lie outside the original class, making this improper, but it remains a legitimate input-to-label rule.

3 Why the Agnostic Case Needs Randomness

Two constant experts suffice. A deterministic learner’s current label is determined by history, so the environment can anticipate and negate it, forcing $n$ errors. The two constant experts’ errors sum to $n$ , so one errs at most $n /2$ . Regret is at least $n /2$ . This concerns discrete-label decisions; convexified predictions with convex loss may permit deterministic averaging, so it is not a claim about every online problem.

FTL follows the current leader too sharply. Hedge changes two things: an error reduces weight rather than eliminating the expert, and the algorithm samples according to weights rather than always selecting the leader. The low-temperature limit $η \to \infty$ approaches FTL, though tied leaders may still be randomized.

4 Hedge: Executable Rule and a Worked Update

L_{0} (i) = 0, w_{t} (i) = e^{- η L_{t - 1} (i)}, p_{t} (i) = \frac{w _{t} ( i )}{\sum _{j} w _{t} ( j )}, L_{t} (i) = L_{t - 1} (i) + ℓ_{t} (i) .

Compute the distribution, sample and act, then observe all current losses and update. With two experts and $η = ln 2$ , initial weights are (1,1) and probabilities (1/2,1/2). After losses (1,0), weights become (1/2,1) and probabilities (1/3,2/3). After another (1,0), weights are (1/4,1) and probabilities (1/5,4/5). Experts are not permanently eliminated and can recover if conditions change.

This pseudocode uses a shifted softmax to avoid exponential underflow. It changes numerical implementation, not the algorithm.

Python

loss_total = zeros(K)
for t in range(n):
    logits = -eta * loss_total
    weights = exp(logits - max(logits))
    probabilities = weights / sum(weights)
    chosen = sample(probabilities)
    act(chosen)
    losses = observe_all_expert_losses()
    loss_total += losses

5 Lemma 1: The Normalizer as a Potential

Set $W_{t} = \sum_{i} e^{- η L_{t} (i)}$ and $Φ_{t} = ln W_{t} / η$ . Their ratio is the exponential-loss average under the current distribution, linking the weight update to current expected loss:

\frac{W _{t}}{W _{t - 1}} = i \sum p_{t} (i) e^{- η ℓ_{t} (i)} .

The required bound is $e^{- x} \leq 1 - x + x^{2}$ for $x \geq - 1$ . For $x \geq 0$ , Taylor’s integral remainder is at most $x^{2} /2$ . For $- 1 \leq x \leq 0$ , set $u = - x \in [0, 1]$ ; the quadratic-and-higher series is at most $(e - 2) u^{2} < u^{2}$ . This explains $η ℓ_{t} (i) \geq - 1$ and permits some negative losses.

Φ_{t} - Φ_{t - 1} = \frac{1}{η} ln i \sum p_{t} (i) e^{- η ℓ_{t} (i)} \leq \frac{1}{η} ln (1 - η ⟨ p_{t}, ℓ_{t} ⟩ + η^{2} i \sum p_{t} (i) ℓ_{t}^{2} (i)) \leq - ⟨ p_{t}, ℓ_{t} ⟩ + η i \sum p_{t} (i) ℓ_{t}^{2} (i) .

The last step uses $ln (1 + u) \leq u$ . Sum over rounds to telescope:

t \sum ⟨ p_{t}, ℓ_{t} ⟩ \leq Φ_{0} - Φ_{n} + η t, i \sum p_{t} (i) ℓ_{t}^{2} (i) .

Initially $W_{0} = K$ , hence $Φ_{0} = ln K / η$ . For any comparator $i^{⋆}$ , the sum contains its term, so $W_{n} \geq e^{- η L_{n} (i^{⋆})}$ and $Φ_{n} \geq - L_{n} (i^{⋆})$ . Substitution gives

t \sum ⟨ p_{t}, ℓ_{t} ⟩ - L_{n} (i^{⋆}) \leq \frac{ln K}{η} + η t, i \sum p_{t} (i) ℓ_{t}^{2} (i) .

This is Lemma 1, simultaneously for all experts on every realized loss sequence. The squared-loss term contains more information than “at most one per round”; Theorem 1 further relaxes it to $n$ for a simple bound.

6 The Hindsight Expert Stays Inside the Expectation

Conditioned on history and the environment’s current choice, expected sampled loss is $⟨ p_{t}, ℓ_{t} ⟩$ . Iterated expectation converts learner loss to expected weighted loss. The best expert depends on the random history, however, so $E min_{i} L_{n} (i)$ must not be replaced by $min_{i} E L_{n} (i)$ .

The lemma holds pathwise for every expert, so selecting the best expert on that path before taking expectations is legitimate. For losses in $[0, 1]$ ,

E Reg \leq \frac{ln K}{η} + η n, η = \frac{ln K}{n} ⟹ E Reg \leq 2 n ln K .

The learning rate balances initial uncertainty $ln K / η$ and update cost $η n$ . With $K = 1$ , follow the single expert rather than use a zero-rate formula. A doubling-horizon restart schedule gives the same order without knowing $n$ ; that is a supplementary strategy, not an algorithm specified in the handout.

7 Generalized Halving as a Mistake Budget

An infinite class cannot vote by counting functions, but Lecture 6 shows at least one label subclass has smaller Littlestone dimension. Algorithm 1 predicts the larger-dimensional label and retains the true-label subclass. A correct prediction cannot increase dimension; an error retains the smaller side and lowers dimension by at least one.

Ldim (V_{t + 1}) \leq Ldim (V_{t}) - 1 {y_{t} \neq = y_{t}} .

Realizability keeps the version space nonempty and its dimension nonnegative. Starting at d permits at most d drops, proving Theorem 2. Dimension acts as a budget of forced mistakes, not parameter count. For singleton indicators, predict negative until the first positive example, after which the target is uniquely determined, giving at most one mistake. Existence does not imply efficient dimension computation.

8 From Realizable to Agnostic Learning: Enumerate Mistake Times

If the input tree were known, Hedge could run over its zero-cover. Future inputs are unknown, so a whole tree cannot be constructed. The key idea is that a realizable run disagrees with Generalized Halving on at most d rounds. Build an expert for each set of at most d rounds and flip its default prediction on those rounds. This covers every target function on every actual path.

T \subseteq [n], ∣ T ∣ \leq d, K = m = 0 \sum m i n (d, n) (m n) .

Each expert has its own version space. After receiving an input, it computes the default Generalized Halving label, flips according to its fixed set T, and updates using its own output. True labels do not update this internal version space; they only determine the expert’s loss for outer Hedge.

Text

                   current input x_t
                     /    |    \
                  expert expert expert
                  T = ∅ T = {1} T = {2}
                     \    |    /
                     Hedge sampling
                           |
                       prediction
                           |
                     true label y_t
                           |
                 update outer loss weights

This is a supplementary sketch. Experts update themselves internally; the final arrow only updates outer weights.

9 Lemma 2: The Cover Proof by Induction

Fix a target $f$ and any actual input path. Imagine another Generalized Halving run labeled by $f$ . It is realizable, so its mistake set $T_{f}$ has size at most d and is in the expert pool. Prove that expert $T_{f}$ matches the target:

Both runs start with the same class. Suppose their version spaces agree before round t, so their default labels agree. If the default equals $f (x_{t})$ , then $t \in / T_{f}$ and the expert does not flip. Otherwise $t \in T_{f}$ and it flips, again outputting $f (x_{t})$ . Both retain the subclass predicting $f (x_{t})$ , so the next spaces agree. Induction completes the proof.

Experts do not know which $T_{f}$ is correct or which target is best. Hedge maintains all candidates. The proof establishes a matching expert without requiring it to be identified beforehand. Other experts may predict impossible labels and enter an empty class; define all their later outputs as −1. The matching expert never becomes empty. Duplicate behaviors among indices only reduce the effective pool size.

10 Theorem 3: Comparator Direction and Computation

T min t \sum 1 {y_{t}^{T} \neq = y_{t}} \leq f \in F in f t \sum 1 {f (x_{t}) \neq = y_{t}} .

Every original function has a matching expert on the realized path, so the expert pool may perform better. Subtract both sides from the same learner loss:

Reg_{F} \leq Reg_{experts}, E Reg_{F} \leq 2 n ln K \leq 2 n d ln (e n / d) .

This corrects the source’s statement that the regrets are the same: inequality suffices. For $1 \leq d \leq n$ , it implies the handout’s $O (d n ln n)$ , using $ln (e n)$ at small horizons. For $d = 0$ , predict the single function. History-dependent experts do not invalidate Hedge, since their current predictions are fixed before the label and current sampling are revealed.

For $d = 2, n = 5$ , there are $1 + 5 + 10 = 16$ indices. Instead of expanding 31-node trees, maintain 16 internal states. Generally $K$ is roughly $n^{d}$ , and computing dimensions adds cost. This is a constructive learnability proof, not a universally efficient algorithm.

The handout cites Hanneke, Livni, and Moran (2021) for near-optimal agnostic regret with randomized proper algorithms. That result requires additional theory; the improper construction above is not presented as a proper algorithm.

11 How Margin Changes the Impossibility Result

Lecture 6 showed online zero-one impossibility for real thresholds because the adversary can probe arbitrarily close to the boundary. A positive margin under normalization excludes that endlessly fine subdivision. Assume dual norms with

∥ x_{t} ∥_{q} \leq 1, ∥ θ^{⋆} ∥_{p} \leq 1, \frac{1}{p} + \frac{1}{q} = 1, y_{t} ⟨ θ^{⋆}, x_{t} ⟩ \geq γ > 0.

Hölder implies $0 < γ \leq 1$ . Mere linear separability without a uniform lower bound permits inputs arbitrarily close to the hyperplane and does not give a fixed mistake bound. In Euclidean geometry the distance is $∣ ⟨ θ^{⋆}, x ⟩ ∣/∥ θ^{⋆} ∥_{2}$ , at least $γ$ under the unit-ball restriction.

The handout first identifies zero hinge loss with unit score margin, then normalizes inputs and introduces arbitrary $γ$ . Normalization does not preserve an unconditional unit-margin requirement: scaling changes scores.

12 Learning Margin Classification with a Finite Cover

Take a $γ /2$ -net of the parameter ball of size at most $(1 + 4/ γ)^{d}$ . Its representative $θ^{'}$ is within $γ /2$ of the separator. Hölder gives

y_{t} ⟨ θ^{'}, x_{t} ⟩ = y_{t} ⟨ θ^{⋆}, x_{t} ⟩ + y_{t} ⟨ θ^{'} - θ^{⋆}, x_{t} ⟩ \geq γ - ∥ θ^{'} - θ^{⋆} ∥_{p} ∥ x_{t} ∥_{q} \geq γ /2 > 0.

A classifier in the net is therefore perfect, and Halving makes at most $d lo g_{2} (1 + 4/ γ)$ mistakes. This is horizon-independent and logarithmic in inverse margin, but the net is exponentially large in d. The next method trades worse margin dependence for an efficient dimension-free guarantee.

13 Perceptron: A Vector Correction on Each Mistake

θ_{1} = 0, s_{t} = sign (⟨ θ_{t}, x_{t} ⟩), θ_{t + 1} = θ_{t} + 1 {s_{t} \neq = y_{t}} y_{t} x_{t} .

Fix $sign (0) = + 1$ . A mistake implies nonpositive signed score; the update increases that score by $∥ x_{t} ∥_{2}^{2}$ , without guaranteeing it crosses zero immediately. Not updating on correct predictions lets norm growth be charged only to mistakes. Storage is one d-dimensional vector; each inner product costs $O (d)$ , or proportional to nonzero inputs in the sparse case.

14 Theorem 4: Two Bounds on the Same Vector

Let $I_{t} = 1 {s_{t} \neq = y_{t}}$ and $M = \sum_{t} I_{t}$ . First, every mistaken update makes progress of at least $γ$ toward the true separator.

⟨ θ_{t + 1}, θ^{⋆} ⟩ = ⟨ θ_{t}, θ^{⋆} ⟩ + I_{t} y_{t} ⟨ x_{t}, θ^{⋆} ⟩ \geq ⟨ θ_{t}, θ^{⋆} ⟩ + I_{t} γ .

Summing from zero makes alignment grow linearly in the number of mistakes. Second, each mistake increases squared norm by at most 1.

∥ θ_{t + 1} ∥_{2}^{2} = ∥ θ_{t} ∥_{2}^{2} + 2 I_{t} y_{t} ⟨ θ_{t}, x_{t} ⟩ + I_{t} ∥ x_{t} ∥_{2}^{2} \leq ∥ θ_{t} ∥_{2}^{2} + I_{t} .

The cross term is nonpositive because an update occurs only on mistakes. The final term is at most 1 because inputs are normalized. Norm therefore grows at most as the square root of mistakes. Cauchy–Schwarz combines the two facts:

M γ \leq ⟨ θ_{n + 1}, θ^{⋆} ⟩ \leq ∥ θ_{n + 1} ∥_{2} ∥ θ^{⋆} ∥_{2} \leq M .

If $M = 0$ , the result is immediate. Otherwise divide by $M$ and square to obtain $M \leq 1/ γ^{2}$ , proving Theorem 4. It holds pathwise for any adaptive sequence satisfying the margin assumption, not merely in expectation.

For general scaling, if $∥ x_{t} ∥_{2} \leq R$ , $∥ θ^{⋆} ∥_{2} \leq B$ , and $y_{t} ⟨ θ^{⋆}, x_{t} ⟩ \geq ρ$ , the same argument gives

M ρ \leq B R M, M \leq (B R / ρ)^{2} .

15 A Worked Perceptron Example

Take separator $θ^{⋆} = (1, 0)$ and examples $((- 1, 0), - 1)$ , $((0.6, 0.8), + 1)$ , $((0.6, - 0.8), + 1)$ . All lie in the unit ball and have margin at least 0.6. With +1 predicted at zero:

The first prediction is +1 and wrong, updating to $(1, 0)$ . The next two scores are both 0.6, so no update occurs. There is only one actual mistake; the bound $1/0. 6^{2} \approx 2.78$ means at most two integer mistakes. The bound need not be attained. On a sequence causing a second error, the proof still counts only mistaken updates, regardless of the number of correct rounds.

16 Sequential Fat and the Dimension Qualification

The handout ends with the claim that linear sequential fat is “of order $1/ γ^{2}$ .” First prove the dimension-free upper bound. Use the Euclidean complexity calculation from Lecture 6, avoiding confusion between a general witness tree and zero-threshold margin. If a depth-m tree is shattered at scale $α$ , choose its matching function on each path:

f sup t = 1 \sum m ϵ_{t} f (x_{t} (ϵ)) \geq t = 1 \sum m ϵ_{t} y_{t} (ϵ) + m α /2.

The witness is predictable, so every $ϵ_{t} y_{t}$ has expectation zero. Divide by m and take expectations: complexity is at least $α /2$ , while Euclidean linear complexity is at most $1/ m$ . Hence

sfat (F, α) \leq 4/ α^{2} .

A matching lower bound needs enough dimensions. Let $k = min (d, ⌊ 4/ α^{2} ⌋)$ , use basis vector $e_{i}$ at level i, and use witness zero. For each sign vector choose $θ = (α /2) \sum_{i = 1}^{k} ϵ_{i} e_{i}$ . Then

∥ θ ∥_{2}^{2} = k α^{2} /4 \leq 1, ϵ_{i} ⟨ θ, e_{i} ⟩ = α /2.

Thus sfat is at least k. Only with sufficiently large d does this match $Θ (1/ α^{2})$ . Why not claim this unconditionally in fixed low dimension? On the one-dimensional parameter interval $[- 1, 1]$ , witnesses for two distinct paths differ by at least $α$ at their first shared branching input, hence their parameters differ by at least $α$ . A depth-m tree has $2^{m}$ paths, so

2^{m} \leq 1 + 2/ α, m \leq lo g_{2} (1 + 2/ α) (d = 1) .

This differs sharply from an unconditional inverse-square lower bound. The derivation also explains the factor 4 caused by the $α /2$ convention versus a direct margin $γ$ .

17 Pitfalls, Self-Test, and Reading Route

Why does deterministic Halving not contradict the two-expert lower bound? It assumes a perfect classifier.
Can the best Hedge expert move outside the expectation? No; the pathwise lemma lets us choose it before taking expectation.
Do T experts internally update using true labels? No; they use their own predictions. True labels update outer weights.
Which regret is larger? The expert benchmark can be better, so expert regret is at least original-class regret.
Where does Perceptron’s nonpositive cross term come from? Updates occur only on mistakes, whose signed score is nonpositive.
What fails without bounded input norm? The bound of one on each squared-norm increase.
Is one update guaranteed to correct its example? No; only score improvement is proved.
Does a dimension-free mistake upper bound imply a matching sfat lower bound in every dimension? No; see the one-dimensional packing argument.

Spend 20 minutes deriving Halving and the two-expert example, then 30 minutes recalculating the Hedge potential. Simulate two T experts on a small class to distinguish internal and outer updates. Finally derive the two Perceptron inequalities independently, check normalization, and read the supplementary sequential-fat argument.