This document provides an overview of a graduate level machine learning course. It will be taught in a topic-driven format with introductory lectures, paper readings, homework, and in-class discussions. Students will present on a paper or subject at the end of the course. The course will cover foundational machine learning theories and tools to help students develop their own models and algorithms, rather than focusing on teaching specific algorithms like support vector machines. Topics will include Markov chain Monte Carlo, exponential family distributions, generalized linear models, empirical risk minimization, proximal optimization methods, graphical models, variational inference, and handling big data.

1. Lecture 0
Course Overview
Dahua Lin
The Chinese University of Hong Kong
1

2. About this Course
This is a graduate level introduction to advanced
statistical learning.
2

3. Course Format
• No Exams!
• Topic driven
• For each topic:
• Introductory lecture
• Paper reading and Homework
• In-class discussion
• You will present a paper/subject at the end of
3

4. What is Machine
Learning?
Machine learning is a scientific discipline that
explores the construction and study of algorithms
that can learn from data. Such algorithms operate
by building a model based on inputs and using
that to make predictions and decisions, rather than
following only explicitly programmed instructions.
-- Wikipedia
4

5. Elements of Machine
Learning
• Elements:
• Data
• Model
• Learning Algorithms
• Prediction
• Learn from old data, make predictions on new
5

6. Please write down five machine learning
algorithms that you know.
Don't write Deep Learning.
6

7. Basic Forms of Machine
Learning
• Supervised learning
• Unsupervised learning
• Semi-supervised learning
• Reinforcement learning
7

8. Machine Learning Tasks
• Classification
• Regression
• Clustering
• Dimension Reduction
• Density Estimation
8

9. What this Course is About
• The course is not to teach you:
• Support Vector Machine
• Linear Regression
• ...
• Deep Learning
• Instead, you are going to learn foundational
theories and tools for developing your own
models and algorithms.
9

10. Topics
• Markov Chain Monte Carlo
• Exponential family distributions and
conjugate prior
• Generalized linear model
• Empirical risk minimization and Stochastic
gradient descent
• Proximal methods for optimization
10

11. Topics (cont'd)
• Graphical models: Bayesian Networks and
Markov random fields
• Sum-product and max-product algorithms,
Belief propagation
• Variational inference methods
• Gaussian Processes and Copula Processes
• Handling Big Data
11