ECE 6254 - Statistical Machine Learning

Lecture Notes and Handouts

 

PDF versions of the lecture notes

 

Introductory supervised learning

• Course Introduction (January 10 & 22)
  • 4 slides per page for printing
• A first model of learning: Concentration inequalities and generalization bounds (January 22)
  • 4 slides per page for printing
  • Notes
• The Bayes classifier and nearest neighbors classifiers (January 22 & 24)
  • 4 slides per page for printing
  • notes
  • Updated Lecture (January 24)
  • Updated Lecture with 4 slides per page for printing
• Plugin Methods (January 24)
  • 4 slides per page for printing
  • Notes
• Plugin Methods II: Logistic Regression (January 31)
  • 4 slides per page for printing
  • notes on grad descent
  • notes on Newton's method
• Non-parametric Linear Classifiers (perceptron & max margin hyperplanes) (January 31-Feb 5)
  • 4 slides per page for printing
• Kernels (Feb 7)
  • 4 slides per page for printing
  • Notes
• Support Vector Machines (Feb 12)
  • 4 slides per page for printing
  • In-class notes on SVMs (updated 2/14)
  • Notes
• Theory of generalization: Dichotomies, the growth function, shattering, and break points (Feb 14)
  • 4 slides per page for printing
  • some other slides from USC that cover this material well
• Theory of generalization 2: The Vapnik-Chervonenkis generalization bound (Feb 19)
  • 4 slides per page for printing
  • Notes
• Regression, least squares, and Tikhonov regularization (Feb 21)
  • 4 slides per page for printing
• Bias and Variance of regression function sets (Feb 26)
  • 4 slides per page for printing
• The LASSO, robust regression, kernel regression, and regularization in classification (Feb 28)
  • 4 slides per page for printing
  • Notes
• Model Selection and Validation (Mar 5)
  • 4 slides per page for printing

 

Introductory unsupervised learning

• Dimensionality reduction, feature selection, and principal component analysis (Mar 12)
  • 4 slides per page for printing
  • Notes
• Multidimensional scaling (Mar 14)
  • 4 slides per page for printing
• Nonlinear dimensionality reduction (Mar 26)
  • 4 slides per page for printing
• Structured matrix factorization (Mar 28-Apr 2)
  • 4 slides per page for printing
  • Notes
  • Extra notes on k-SVD
  • Paper on k-SVD
• Kernel density estimation and k-means clustering (Apr 4)
  • 4 slides per page for printing
• Gaussian mixture models and expectation maximization
  • 4 slides per page for printing
• Spectral clustering, density-based clustering, and hierarchical clustering
  • 4 slides per page for printing

 

Advanced Topics

• Decision trees
  • 4 slides per page for printing
• Ensemble methods
  • 4 slides per page for printing
• Neural networks
  • 4 slides per page for printing
• Deep learning
  • 4 slides per page for printing