Lecture schedule
Lectures will be hybrid, meaning both online and offline. The class time is Fridays 12:15-14:00 in Delta room 2047 and over Zoom (link here. The password is ati).
You can find the video and materials for each class here in the schedule, after they are available. Also, please refer to links for additional sources about each class.
Please pick the slot for your presentation/test in this Google Sheet.
Tentative schedule for the lectures:
- 12.02.2021 - Lecture 1: A Toy Model: Linear Regression (Tambet) - (video) (slides) (notebook)
- 19.02.2021 - Practice 1: JAX Tutorial (Mikhail) - (video) (slides) (notebook tutorial 1) (notebook tutorial 2)
- 26.02.2021 - Lecture 2: Taylor Approximations (Oriol) - (video) (slides)
- 05.03.2021 - Practice 2: Problem Set 1: Gradient Descent with Momentum (Tarun) - (video)
- 12.03.2021 - Lecture 3: Metrics (Raul) - (video)
- 19.03.2021 - Practice 3: Problem Set 2: Computing the Grassmannian Length (Mykyta) - (video)
- 26.03.2021 - Lecture 4: Second-Order Optimization (Markus) - (video)
- 02.04.2021 - Holiday (no lecture)
- 09.04.2021 - Practice 4: Problem Set 3: Path Energy and Geodesics (Tetiana) - (video)
- 16.04.2021 - Lecture 5: Adaptive Gradient Methods, Normalization, and Weight Decay (Viacheslav) - (video)
- 23.04.2021 - Lecture 6: Infinite Limits and Overparameterization (Kallol) - (video)
- 30.04.2021 - Lecture 7: Stochastic Optimization and Scaling (Farid) - (video)
- 07.05.2021 - Lecture 8: Bayesian Inference and Implicit Regularization (Meelis) - (video) (slides)
- 14.05.2021 - Lecture 9: Dynamical Systems and Momentum (Florian) - (video)
- 21.05.2021 - No lecture
- 28.05.2021 - Lecture 10: Differential Games (Sven) - (video)
The reading materials can be found on original course homepage:
https://www.cs.toronto.edu/~rgrosse/courses/csc2541_2021/
Homework deadlines:
- 12.03.2021 - Problem Set 1: Gradient Descent with Momentum
- 26.03.2021 - Problem Set 2: Computing the Grassmannian Length
- 16.04.2021 - Problem Set 3: Path Energy and Geodesics
Homework assignments can be found here (please disregard the header of the pdf, as it is specific to the reference course that we are following). Note that homework 1 corresponds to the first set of questions ("gradient descent with momentum"), and so forth for the other two sets. There is also starter code for Problem Set 2 and Problem Set 3. Grades sheet available here.