CSCI 678 is USC's theoretical machine learning course (Fall 2026, Haipeng Luo), built around the core tools of statistical learning theory: Rademacher complexity, covering numbers, VC dimension and pseudo-dimension, fat-shattering, chaining, and neural-network spectral complexity and margins, continuing into online learning, Littlestone dimension, Hedge, and Perceptron. These notes support lecture preparation and later review.
1. How the notes are organized
Each lecture has a complete handout edition and detailed explanation: original sections, results, and numbering are retained, while motivations, proofs, examples, assumptions, pitfalls, and self-tests are developed. Omitted deep theorems remain explicit, and supplementary derivations and corrections are identified. KaTeX renders the mathematics, with matching mathematical content in English and Chinese.
2. Currently published
Each lecture has two companion documents: a detailed explanation report (section-by-section, filling in proof details) and a full translation of the slides (preserving the original numbering and structure).
- Lecture 1: Detailed Explanation | Full Translation — formalizing the learning problem, the statistical learning framework, the no-free-lunch theorem, online and partial-information learning.
- Lecture 2: Detailed Explanation | Full Translation — uniform convergence, symmetrization and Rademacher complexity, growth function and VC dimension.
- Lecture 3: Preview Report | Full Translation — covering numbers, Dudley's entropy integral, and pseudo-dimension.
- Lecture 4: Detailed Explanation | Full Handout — fat-shattering dimension, neural-network covers, spectral complexity, and margin bounds, including the original experiment figures and step-by-step derivations.
- Lecture 5: Detailed Explanation | Full Handout — Derives tree symmetrization from minimax duality and martingale differences, explaining dependent data, binary relabeling, and contraction conditions.
- Lecture 6: Detailed Explanation | Full Handout — Develops sequential Sauer, threshold and monotone counterexamples, regression entropy integrals, and surrogate normalization.
- Lecture 7: Detailed Explanation | Full Handout — Derives the exponential-weights potential, mistake-indexed experts, and the Perceptron bound, with margin and dimension qualifications.
Notes for later lectures will be added over time. Note updates live under Updates.