Lecture 6 Full Handout

Fall 2026. Haipeng Luo.

Reading edition of all eight pages, retaining four sections, subsection 3.1, Lemma 1, Theorems 1–6, Corollary 1, Proposition 1, Equations (1)–(3), both regression examples, and the footnote. Source: lecture6.pdf (local source, not published online). Companion: detailed explanation. Editorial notes are explicit. The original has no figures or bibliography.

1 Online Learning with a Finite Class

The previous lecture introduced online learning and derived the following sequence of upper bounds on the game value:

V^{seq} (F, n) \leq P \in Δ (Z^{n}) sup E_{z_{1 : n} \sim P} f \in F sup \frac{1}{n} t = 1 \sum n (E_{z_{t}^{'} \sim P (\cdot ∣ z_{1 : t - 1})} ℓ (f, z_{t}^{'}) - ℓ (f, z_{t})) \leq 2 R^{seq} (ℓ (F)) ≲ R^{seq} (F) .

The first two steps are general; the last is specific to supervised learning. Its shorthand suppresses the Lipschitz constant and possible logarithmic factor from contraction. This lecture bounds $R^{seq} (F)$ , starting with finite classes and the maximal inequality.

Lemma 1 (Maximal Inequality). If the finite collection ${U_{f}}_{f \in F}$ consists of variables with the zero-centered $σ$ -sub-Gaussian moment-generating-function bound, then

E f \in F max U_{f} \leq σ 2 ln ∣ F ∣ .

Theorem 1. Let $F \subseteq Y^{X}$ be finite. For every input tree,

R^{seq} (F; x) \leq \frac{1}{n} 2 (f \in F max ϵ max t = 1 \sum n f^{2} (x_{t} (ϵ))) ln ∣ F ∣ .

Consequently, if $Y \subseteq [- C, C]$ , then

R^{seq} (F) \leq C \frac{2 ln ∣ F ∣}{n} .

Proof. Write

U_{f, τ} = t = 1 \sum τ ϵ_{t} f (x_{t} (ϵ)), σ^{2} = f max ϵ max t = 1 \sum n f^{2} (x_{t} (ϵ)) .

It suffices to show that $U_{f, n}$ is $σ$ -sub-Gaussian, then apply Lemma 1 and divide by $n$ . For positive $λ$ , first peel off the last round:

E e^{λ U_{f, n}} = E [e^{λ U_{f, n - 1}} E [e^{λ ϵ_{n} f (x_{n} (ϵ))} ∣ ϵ_{1 : n - 1}]] \leq E [e^{λ U_{f, n - 1}} e^{λ^{2} f^{2} (x_{n} (ϵ)) /2}] .

Conditionally on the past, $ϵ_{n} f (x_{n} (ϵ))$ is sub-Gaussian with parameter $∣ f (x_{n} (ϵ)) ∣$ . Peeling one more round gives

E [e^{λ U_{f, n - 2}} E [e^{λ ϵ_{n - 1} f (x_{n - 1} (ϵ))} e^{λ^{2} f^{2} (x_{n} (ϵ)) /2} ∣ ϵ_{1 : n - 2}]] .

The final squared term still depends on $ϵ_{n - 1}$ , so it cannot simply be treated as constant. First take a worst case:

\leq E [e^{λ U_{f, n - 2}} E [e^{λ ϵ_{n - 1} f (x_{n - 1} (ϵ))} ∣ ϵ_{1 : n - 2}] ϵ_{n - 1} max e^{λ^{2} f^{2} (x_{n} (ϵ)) /2}] \leq E [e^{λ U_{f, n - 2}} ϵ_{n - 1} max e^{\frac{λ ^{2}}{2} (f^{2} (x_{n - 1} (ϵ)) + f^{2} (x_{n} (ϵ)))}] \leq E [e^{λ U_{f, n - 2}} ϵ_{n - 2}, ϵ_{n - 1} max e^{\frac{λ ^{2}}{2} (f^{2} (x_{n - 1} (ϵ)) + f^{2} (x_{n} (ϵ)))}] .

Continuing in the same fashion yields

E e^{λ U_{f, n}} \leq ϵ max e^{\frac{λ ^{2}}{2} \sum_{t = 1}^{n} f^{2} (x_{t} (ϵ))} \leq e^{λ^{2} σ^{2} /2} .

This proves that finite bounded classes are online learnable and will be a key ingredient for infinite classes.

2 Infinite Class: Online Binary Classification and Zero-Cover

Start with binary classification and zero-one loss. In statistical learning, even an infinite class has a finite projection on a fixed sample. The online analogue is

R^{seq} (F; x) = \frac{1}{n} E_{ϵ} f \in F sup t \sum ϵ_{t} f (x_{t} (ϵ)) = \frac{1}{n} E_{ϵ} v \in V sup t \sum ϵ_{t} v_{t} (ϵ) \leq \frac{2 ln ∣ V ∣}{n}, (1)

V = F ∣_{x} = {(f \circ x_{1}, \dots, f \circ x_{n}) : f \in F} .

The projection is a finite set of binary-valued trees. A depth- $n$ tree has $2^{n} - 1$ nodes, so its projection can have size $2^{2^{n} - 1}$ , whereas a statistical sample has at most $2^{n}$ labelings. Both crude bounds are vacuous for learnability. Perhaps a simple class has a small projection? Unfortunately, consider the singleton-indicator class on the real line:

F = {f_{θ} (x) = {+ 1, - 1, x = θ, x \neq = θ, θ \in R} . (2)

Each classifier predicts +1 at exactly one input. It cannot shatter two points, so its VC dimension is exactly 1 and it is statistically learnable. But if all nodes of the input tree are distinct, the projection contains the all-negative tree and $2^{n} - 1$ trees with a single positive node: $2^{n}$ trees in total. Bound (1) is again vacuous. Is projection the wrong measure, or is even this simple class not online learnable? Fortunately, the former is true.

A projection requires

\forall f \in F, \exists v \in V, \forall ϵ \in {- 1, + 1}^{n}, f (x_{t} (ϵ)) = v_{t} (ϵ) \forall t .

Swapping two quantifiers gives a weaker property that already suffices for (1), with its second equality changed to an inequality:

\forall f \in F, \forall ϵ \in {- 1, + 1}^{n}, \exists v \in V, f (x_{t} (ϵ)) = v_{t} (ϵ) \forall t .

Such a set $V$ is a zero-cover of $F ∣_{x}$ , and its minimum size is $N_{0} (F ∣_{x})$ . Hence

R^{seq} (F; x) \leq \frac{2 ln N _{0} ( F ∣ _{x} )}{n} .

A zero-cover is no larger than the projection, which covers itself. It is also always at most $2^{n}$ : take all trees constant within each level, one for each binary sequence of length $n$ . They match every possible path labeling. This crude bound is still vacuous but matches the statistical setting’s crude bound.

The class in (2) has a zero-cover of size at most $n + 1$ , implying online learnability. First consider the special case with no repeated input on any path. Use the all-negative tree and, for each level $t$ , a tree positive throughout level $t$ and negative elsewhere. Repeated values make direct construction less obvious. Ideally a combinatorial parameter analogous to VC dimension would control covers through a Sauer-type lemma. The next section provides one.

Editorial note: the source says “zero-covering number $n + 1$ .” For an arbitrary fixed tree this is an upper bound, not always equality. If every input is identical, two trees suffice; a worst-case tree can attain $n + 1$ .

3 Littlestone Dimension

A class shatters a tree $x$ if every path/label sequence can be matched by one function along the entire path:

\forall ϵ \in {- 1, + 1}^{n}, \exists f \in F, f (x_{t} (ϵ)) = ϵ_{t} (t = 1, \dots, n) .

The Littlestone dimension $Ldim (F)$ is the maximum depth of a shattered tree: zero when no positive-depth tree is shattered, and infinity when every finite depth is possible. (Original note 1) VC dimension never exceeds Littlestone dimension: fill level $t$ with the $t$ -th point of a shattered fixed sample. For a nonempty class, Littlestone dimension zero implies VC dimension zero and thus only one distinct function.

To prove dimension $d$ , construct a shattered depth- $d$ tree and rule out all depth- $d + 1$ trees. The singleton class in (2) has dimension exactly 1: any root can be shattered, but after a positive first label only one function remains, so both $(+ 1, + 1)$ and $(+ 1, - 1)$ cannot be realized. HW2 studies a higher-dimensional extension. The following Sauer analogue also provides a recursive cover construction.

Theorem 2. For any depth- $n$ input tree and binary class of dimension $d < n$ ,

N_{0} (F ∣_{x}) \leq i = 0 \sum d (i n) \leq (\frac{e n}{d})^{d} .

Proof. Let $g (d, n) = \sum_{i = 0}^{d} (i n)$ . The second inequality was established in Sauer’s lemma. Induct on $d + n$ . The base $d = 0, n = 1$ is immediate, and any case $d = 0$ needs one tree. Now take $d > 0$ and partition by the root input $x_{1}$ :

F_{-} = {f \in F : f (x_{1}) = - 1}, F_{+} = {f \in F : f (x_{1}) = + 1} .

Let $x^{ℓ}, x^{r}$ be the root’s subtrees. Take minimal zero-covers $V_{+}^{ℓ}, V_{+}^{r}$ for $F_{+}$ on them. Construct $V_{+}$ with positive roots and pair elements of the two subcovers as left and right subtrees, ensuring every element appears. Repeat elements of the shorter list, so only the maximum of the two sizes is needed. A path enters only one subtree, making this a valid cover of $F_{+} ∣_{x}$ . Construct $V_{-}$ similarly and take their union.

Both subclasses cannot have dimension $d$ . Otherwise attach their respective shattered depth- $d$ trees under $x_{1}$ , producing a shattered depth- $d + 1$ tree, a contradiction. Without loss of generality the negative subclass has dimension at most $d - 1$ . Induction gives

∣ V_{+} ∣ ∣ V_{-} ∣ N_{0} (F ∣_{x}) = max {∣ V_{+}^{ℓ} ∣, ∣ V_{+}^{r} ∣} \leq g (d, n - 1), = max {∣ V_{-}^{ℓ} ∣, ∣ V_{-}^{r} ∣} \leq g (d - 1, n - 1), \leq ∣ V_{-} ∣ + ∣ V_{+} ∣ \leq g (d - 1, n - 1) + g (d, n - 1) = g (d, n) .

The final equality is Pascal’s identity. Zero-covering allows pairing by the maximum; covering entire projected trees could require a product of subtree cover sizes. For the singleton class, this gives $g (1, n) = n + 1$ , even with repeated inputs. Try the recursion on a small depth-three tree.

Editorial note: the original induction is stated for $d < n$ , but recursion can reach the boundary. Add the trivial case $g (d, m) = 2^{m}$ for $d \geq m$ to close the induction. An empty subclass needs no cover; interpret the $d = 0$ power bound as 1.

Corollary 1. The value for a binary class satisfies

V^{seq} (F, n) \leq R^{seq} (F) \leq \frac{2 d ln ( e n / d )}{n}, d = Ldim (F) .

Finite Littlestone dimension is sufficient for online learnability and is also necessary. Use the display for $1 \leq d \leq n$ ; the value is zero at $d = 0$ , and use a trivial bound for $d > n$ .

Theorem 3. If Littlestone dimension is infinite, then the game value is at least $1/2$ for every horizon.

Proof. Take a shattered depth- $n$ tree and independent random signs. At time $t$ , the environment provides $(x_{t} (ϵ), ϵ_{t})$ . For every proper or improper binary algorithm, the current label is a fresh fair sign, so expected total loss is exactly $n /2$ . Shattering provides a zero-error comparator on every realized path, so expected regret is $n /2$ . HW2 establishes the stronger lower bound $d / (8 n)$ when $d \leq n$ .

Summary. Combining the classification bounds gives

\frac{d}{8 n} \leq V^{seq} (F, n) \leq P sup E f sup \frac{1}{n} t \sum (E_{t - 1} ℓ (f, z_{t}) - ℓ (f, z_{t})) \leq 2 R^{seq} (ℓ (F)) \leq R^{seq} (F) \leq x sup \frac{2 ln N _{0} ( F ∣ _{x} )}{n} \leq \frac{2 d ln ( e n / d )}{n} .

Here $d \leq n$ , and the conditional expectation denotes the loss of an independent conditional copy. The bounds still differ by a logarithmic factor; “tight” here does not mean equality up to an absolute constant.

3.1 Online Learning Is Strictly Harder

The online-to-batch conversion from Lecture 1 showed that online learning is at least as hard. The singleton class, whose two dimensions both equal 1, does not show strict separation. Instead consider thresholds on the real line:

F = {f_{θ} (x) = {+ 1, - 1, x \leq θ, x > θ, θ \in R} . (3)

Lecture 2 established VC dimension exactly 1, but its Littlestone dimension is infinite. Proposition 1. The threshold class in (3) has infinite Littlestone dimension.

Proof. Construct an infinite input tree with root $1/2$ . A node of value $a$ at level $t$ has left and right children $a - 2^{- (t + 1)}$ and $a + 2^{- (t + 1)}$ . Draw the tree to see the construction. For any finite depth and path, set

L = t : ϵ_{t} = + 1 max x_{t} (ϵ), U = t : ϵ_{t} = - 1 min x_{t} (ϵ),

Use defaults 0 and 1 when the corresponding sets are empty. We claim $L < U$ ; then every $θ \in [L, U)$ matches all path labels. After a positive step, every later node on the path is larger; after a negative step, every later node is smaller. Thus every positive node is strictly smaller than every negative node, regardless of their temporal order. Hence $L < U$ , completing the proof.

Thus this simple class is statistically learnable but not online learnable. More generally, affine linear classifiers

F = {x \mapsto sign (⟨ x, θ ⟩ + b) : θ \in R^{d}, b \in R}

contain thresholds and therefore have infinite Littlestone dimension despite finite VC dimension $d + 1$ . Online learning is strictly harder. Is this too restrictive if even linear classification is impossible? For online zero-one loss, in a sense yes. Yet in statistical learning, finite VC dimension only gives learnability via ERM; computing ERM can still be difficult for linear classifiers. Practice therefore uses nicer surrogate losses, moving toward regression, where more possibilities open up online as well.

4 Online Regression

Now let $F \subseteq [- 1, + 1]^{X}$ . A set $V$ of trees valued in this interval is an $ℓ_{p}$ $α$ -cover of $F ∣_{x}$ if

\forall f \in F, \forall ϵ \in {- 1, + 1}^{n}, \exists v \in V, (\frac{1}{n} t = 1 \sum n ∣ f (x_{t} (ϵ)) - v_{t} (ϵ) ∣^{p})^{1/ p} \leq α .

The quantifier order agrees with zero-covering and parallels approximate covering in the statistical setting. The smallest size is $N_{p} (F ∣_{x}, α)$ . It is nonincreasing in radius $α$ , and, under this normalization, nondecreasing in $p$ . For $p = \infty$ , use maximum error along a path. Covering reduces infinite classes to finite ones.

Theorem 4. For every input tree,

R^{seq} (F; x) \leq α \geq 0 in f {α + \frac{2 ln N _{1} ( F ∣ _{x} , α )}{n}} .

Proof. Fix $α$ , choose a minimal cover $V$ , and let $v^{f, ϵ}$ be a representative for that function and path. Add and subtract it, split the supremum, and apply the approximation property and Theorem 1:

R^{seq} (F; x) = \frac{1}{n} E f sup t \sum ϵ_{t} {f (x_{t} (ϵ)) - v_{t}^{f, ϵ} (ϵ) + v_{t}^{f, ϵ} (ϵ)} \leq \frac{1}{n} E f sup t \sum ϵ_{t} {f (x_{t} (ϵ)) - v_{t}^{f, ϵ} (ϵ)} + \frac{1}{n} E f sup t \sum ϵ_{t} v_{t}^{f, ϵ} (ϵ) \leq α + \frac{1}{n} E v \in V sup t \sum ϵ_{t} v_{t} (ϵ) \leq α + \frac{2 ln N _{1} ( F ∣ _{x} , α )}{n} .

Take the infimum over radii. Similar chaining techniques give the tighter Dudley entropy integral; the handout omits the proof.

Theorem 5.

R^{seq} (F; x) \leq 0 \leq α \leq 1 in f {4 α + \frac{12}{n} \int_{α}^{1} ln N_{2} (F ∣_{x}, δ) d δ} .

Again a combinatorial parameter controls covering numbers. A tree is $α$ -shattered if there exists a witness tree $y$ , valued in $[- 1, + 1]$ , such that

\forall ϵ \in {- 1, + 1}^{n}, \exists f \in F, ϵ_{t} (f (x_{t} (ϵ)) - y_{t} (ϵ)) \geq α /2 (t = 1, \dots, n),

and the largest such depth is the sequential fat-shattering dimension, denoted $sfat (F, α)$ . The following connection is also stated without proof.

Theorem 6. For positive $α$ , depth $n$ , and the preceding class, the handout states

ln N_{\infty} (F ∣_{x}, α) \leq sfat (F, α) ln (\frac{2 e n}{α}) .

Editorial note: use this at the effective scales $0 < α \leq 1$ ; for $α \geq 1$ , the zero tree already covers and the covering number is 1. Avoid formal products of infinite dimensions and negative logarithms. Match the handout’s $α /2$ margin convention when comparing sources.

Example 1: Linear Functions

Recall $F = {f_{θ} (x) = ⟨ θ, x ⟩ : θ \in B_{p}^{d}}$ on $X = B_{q}^{d}$ , with $p, q \geq 1$ and $1/ p + 1/ q = 1$ . Proposition 2 of Lecture 3 gave a pointwise cover $H$ of size at most $(2/ α + 1)^{d}$ . Projecting it onto any tree gives a sequential cover, hence

N_{\infty} (F ∣_{x}, α) \leq (2/ α + 1)^{d}, R^{seq} (F) = O (d / n) .

The latter follows from the previously computed Dudley integral. Linear real-valued classes are therefore online learnable, with the same complexity rate as statistically. Passing to a game value through general Lipschitz contraction still carries its additional factor; the complexity rate is not unconditionally the regret rate for every loss.

Example 2: Nondecreasing Functions

Statistically, all nondecreasing functions from the real line to $[- 1, + 1]$ have a covering estimate written in the handout as $(n + 1)^{1/ α}$ and fat dimension $O (1/ α)$ . Online, no corresponding learnability bound is possible because the problem is not online learnable. Use the bisection tree from Proposition 1 and let the environment choose

(x_{t}, y_{t}) = (x_{t} (- ϵ), ϵ_{t}), ℓ (f, (x, y)) = (f (x) - y)^{2} .

Notice the reversed path signs. The current label is equally likely to be positive or negative, so every prediction has conditional expected squared loss at least

\frac{1}{2} y min {(y - 1)^{2} + (y + 1)^{2}} = 1.

On the other hand, every negatively labeled input lies to the left of every positively labeled input. A nondecreasing function can interpolate all observations with zero squared loss, forcing linear regret. The handout additionally remarks, without details, that the problem becomes learnable in the transductive setting where the input set is known in advance, though neither its order nor labels are known.

Editorial note: 1 is a lower bound on conditional expected loss, not on every realized loss. The statistical cover expression retains the source’s scale shorthand; interval length, rounding, and radius conventions affect constants in its exponent, not the online impossibility conclusion.

Bridging Classification and Regression

Binary classifiers often have the form $H = {sign (f (\cdot)) : f \in F}$ : a score’s sign predicts the label and its magnitude expresses confidence. Choosing scores instead of their classifiers does not change the zero-one problem. With the handout’s conservative convention for zero scores, loss is $1 {y f (x) \leq 0}$ . Infinite $Ldim (H)$ does not become learnable merely by rewriting it.

But one may optimize an online-learnable surrogate $ℓ^{'}$ that upper-bounds zero-one loss. Then

t = 1 \sum n 1 {y_{t} f_{t} (x_{t}) \leq 0} \leq t = 1 \sum n ℓ^{'} (f_{t}, (x_{t}, y_{t})) = f \in F in f t = 1 \sum n ℓ^{'} (f, (x_{t}, y_{t})) + Reg_{ℓ^{'}} (F, n) .

The source mixes $T$ and $n$ here; this edition consistently uses $n$ . If expected average regret tends to zero, the learner’s average mistakes are controlled by the best function’s average surrogate loss plus a vanishing term. This avoids the zero-one impossibility using a different comparator loss. Statistical learning often does the same for computational reasons. Typical surrogates are

ℓ_{hinge} (f, (x, y)) = max {1 - y f (x), 0}, ℓ_{logistic} (f, (x, y)) = lo g (1 + e^{- y f (x)}) .

The handout says both losses are 1-Lipschitz in the score. Sufficient control of sequential fat dimension at every scale then yields online learnability and meaningful mistake bounds.

Editorial note: with natural logarithms, logistic loss equals $ln 2 < 1$ at zero score and does not directly upper-bound this zero-one loss. The correct inequality is $1 {y f (x) \leq 0} \leq ℓ_{logistic} / ln 2$ ; the normalized loss has Lipschitz constant $1/ ln 2$ . Both claims hold for hinge loss. A fixed binary sign convention at zero may also differ from the conservative zero-margin loss.

Summary

For $G$ -Lipschitz regression loss, the handout summarizes

V^{seq} (F, n) \leq P sup E f sup \frac{1}{n} t \sum {E_{t - 1} ℓ (f, z_{t}) - ℓ (f, z_{t})} \leq 2 R^{seq} (ℓ (F)) \leq O (G ln^{3/2} n) R^{seq} (F) \leq O (G ln^{3/2} n) x sup 0 \leq α \leq 1 in f {4 α + \frac{12}{n} \int_{α}^{1} ln N_{2} (F ∣_{x}, δ) d δ} \leq O (G ln^{3/2} n) 0 \leq α \leq 1 in f {4 α + \frac{12}{n} \int_{α}^{1} sfat (F, δ) ln (2 e n / δ) d δ} .

The handout states that these bounds are tight in an appropriate sense and sequential fat dimension is the right complexity measure. This finishes its discussion of statistical and online learnability for classification and regression. The next lecture begins algorithm design. The full equivalence theorem and its conditions are not proved here and do not follow automatically from this single logarithmic upper bound.

Original Footnote

Original note 1: In sequential Rademacher complexity and shattering, $ϵ$ represents both a path and a labeling. In zero-covering, it represents only a path.