CS 57800 Statistical Machine Learning, Fall 2026

Course Details

Description: This graduate course develops the mathematical foundations of statistical machine learning, with statistical reasoning and probabilistic inference as recurring themes. Modern AI systems can often produce predictions or implement standard learning algorithms automatically. The harder questions are what those predictions mean, whether they are supported by the available data, which assumptions and approximations they rely on, and when they may fail. Accordingly, this offering is theory-focused: rather than presenting a catalog of methods, it develops a mathematical account of four questions that recur throughout modern machine learning:

These questions organize the course into six connected units: statistical learning theory; unsupervised and latent-variable learning; probabilistic inference and statistical physics; diffusion-model theory; modern neural-network theory; and trustworthy machine learning. Students will also read selected research papers to see how these ideas are applied, tested, and extended in current research.

Time/Location: MWF 1:30 PM - 2:20 PM at LWSN B155

Instructor: Ruizhe Zhang Office hours: M 10:30 AM - 11:30 AM at DSAI 1119B

Teaching Assistants: Shreya Nasa (Office hours: Th 9:00 AM - 10:00 AM at DSAI B061), Yucheng Zhang (Office hours: Th 3:00 PM - 3:30 PM at DSAI B061)

Course policy: See syllabus for detailed overview.

Suggested projects list: See [link 1] and [link 2].

Problem Sets

Problem sets will be posted below and in Brightspace when they become available.

Lectures

The schedule will be updated as we progress through the course.

Date Topic Materials Resources
Aug 24 Introduction [slides]
Aug 26 Statistical Learning Theory: Model [slides] [Notations]
Aug 28 Statistical Learning Theory: PAC learning [slides]
Aug 31 Statistical Learning Theory: The VC dimension [slides]
  • Understanding Machine Learning [Shalev-Shwartz-Ben-David] (Sec. 6)
    • Sep 2 Statistical Learning Theory: The VC dimension [slides]
  • Understanding Machine Learning [Shalev-Shwartz-Ben-David] (Sec. 6)
    • Sep 4 Statistical Learning Theory: SVM and Rademacher complexity [slides]
  • Understanding Machine Learning [Shalev-Shwartz-Ben-David] (Sec. 15 & Sec. 26)
    • Sep 9 Statistical Learning Theory: SVM generalization bound [slides]
  • Understanding Machine Learning [Shalev-Shwartz-Ben-David] (Sec. 26)
    • Sep 11 Unsupervised Learning: Clustering [slides]
  • Understanding Machine Learning [Shalev-Shwartz-Ben-David] (Sec. 22)
    • Sep 14 Unsupervised Learning: Clustering [slides]
  • Understanding Machine Learning [Shalev-Shwartz-Ben-David] (Sec. 22)
    • Sep 16 Unsupervised Learning: k-means [slides]
  • Understanding Machine Learning [Shalev-Shwartz-Ben-David] (Sec. 22)
  • [Dasgupta notes]
    • Sep 18 Unsupervised Learning: k-means++ [slides]
  • [Dasgupta notes]
    • Sep 21 Unsupervised Learning: Spectral Clustering [slides]
  • [Shewchuk notes]
  • Spectral and Algebraic Graph Theory [Spielman] (Sec. 20 & Sec. 21)
    • Sep 23 Unsupervised Learning: Spectral Partitioning [slides]
  • Cheeger's Inequality: [Shewchuk notes] (Appendix B), [Trevisan notes]
  • Spectral and Algebraic Graph Theory [Spielman] (Sec. 20 & Sec. 21)
    • Sep 25 Latent Variables Learning: Matrix Completion [slides]
  • Algorithmic Aspects of Machine Learning: [Moitra] (Sec. 7)
    • Sep 28 Latent Variables Learning: Matrix Completion [slides]
  • Algorithmic Aspects of Machine Learning: [Moitra] (Sec. 7)
    • Sep 30 Latent Variables Learning: Matrix Completion [slides]
  • Algorithmic Aspects of Machine Learning: [Moitra] (Sec. 7)
    • Oct 2 Latent Variables Learning: Matrix Completion [slides]
  • Algorithmic Aspects of Machine Learning: [Moitra] (Sec. 7)
  • Matrix Completion: [Recht 2011]
    • Oct 5 Latent Variables Learning: Method of Moments [slides]
  • Algorithmic Aspects of Machine Learning: [Moitra] (Sec. 3)
    • Oct 7 Midterm 1
    Oct 9 Latent Variables Learning: Method of Moments [slides]
  • Algorithmic Aspects of Machine Learning: [Moitra] (Sec. 3)
    • Oct 14 Latent Variables Learning: Method of Moments [slides]
  • Algorithmic Aspects of Machine Learning: [Moitra] (Sec. 3)
  • Tensor decomposition: [Vijayaraghavan'20]
    • Oct 16 Latent Variables Learning: Method of Moments [slides]
  • Algorithmic Aspects of Machine Learning: [Moitra] (Sec. 4)
    • Oct 19 Probabilistic Inference and Statistical Physics: Graphical Models [slides]
  • Information, Physics, and Computation: [Mézard-Montanari]
  • Statistical Physics Methods in Optimization and Machine Learning: [Krzakala-Zdeborová]
  • Lecture Notes for Graphical Models: [Montanari]
  • Graphical Models, Exponential Families, and Variational Inference: [Wainwright-Jordan]
    • Oct 21 Probabilistic Inference and Statistical Physics: Variational Inference [slides]
    Oct 23 Probabilistic Inference and Statistical Physics: Belief Propagation [slides]
    Oct 26 Probabilistic Inference and Statistical Physics: Belief Propagation [slides]
    Oct 28 Probabilistic Inference and Statistical Physics: BP and Bethe Free Energy [slides]
    Oct 30 Probabilistic Inference and Statistical Physics: Z2 Synchronization [slides]
    Nov 2 Probabilistic Inference and Statistical Physics: AMP for Z2 Synchronization [slides]
  • Survey on approximate message passing [Feng et al. '21]
    • Nov 20 Midterm 2

    Resources