Advanced Mathematics for Machine Learning

Research-level mathematical topics and learning theory for understanding modern ML research.

Prerequisites: Complete foundational/ and mml-book/ sections first.

This folder is for deepening theory after the core curriculum starts making concrete sense. It is not a first-pass requirement, and it should be approached selectively based on your actual interests.

Notebooks

Part I: Learning Theory

#	Notebook	Topics
01	Introduction to Learning Theory	Generalization, bias-variance tradeoff
02	Concentration Inequalities	Hoeffding, Bernstein, McDiarmid’s inequality
03	Rademacher Complexity	Uniform convergence, capacity measures
04	PAC-Bayes Theory	PAC learning framework, Bayesian perspective
05	Neural Tangent Kernel	Infinite-width neural networks, kernel methods

Part II: Advanced Optimization & Inference

#	Notebook	Topics
06	Variational Inference	Mean-field approximation, ELBO
07	Bayesian Nonparametrics	Dirichlet Process, Chinese Restaurant Process
08	Expectation Maximization	EM algorithm, convergence proofs, GMM
09	Gradient Descent Convergence	Implicit bias, convergence analysis

Part III: Advanced Models & Theory

#	Notebook	Topics
10	State Space Models	Kalman Filters, Hidden Markov Models
11	Copula Theory	Dependency modeling, multivariate distributions
12	Determinantal Point Processes	Diversity modeling, sampling
13	Johnson-Lindenstrauss	Random projections, dimensionality reduction
14	Duality Theory	Lagrangian duality, KKT conditions
15	Conjugate Gradients	Efficient second-order optimization
16	Matrix Concentration	Matrix-valued concentration inequalities

Learning Paths

Theoretical ML Researcher: 01 -> 02 -> 03 -> 04 -> 05

Probabilistic ML: 06 -> 07 -> 08 -> 10

Optimization: 09 -> 14 -> 15

How To Use This Folder Well

Pick one path based on your interest instead of trying to complete all sixteen notebooks at once.
Use this folder to support research reading or a specific technical curiosity.
Return to practical phases after each deep dive so the theory stays grounded.

Prerequisites

Solid understanding of linear algebra, multivariable calculus, probability, and basic ML
Python 3.8+, NumPy, SciPy, Matplotlib

foundational/ - Core prerequisites
mml-book/ - Intermediate theory
mlpp-book/ - Probabilistic ML depth

What Comes Next

Continue to ../../24-advanced-deep-learning/README.md if you want theory connected to modern architectures.
Continue to ../../28-practical-data-science/README.md if you want to reconnect theory to applied work.
Return to ../README.md to choose another math path only if you have a clear reason.