Institute of Computer Science
  1. Courses
  2. 2020/21 spring
  3. Special Course in Machine Learning: Neural Network Training Dynamics (MTAT.03.317)
ET
Log in

Special Course in Machine Learning: Neural Network Training Dynamics 2020/21 spring

  • Main
  • Schedule
  • Submission
  • Links

Lecture schedule

Lectures will be hybrid, meaning both online and offline. The class time is Fridays 12:15-14:00 in Delta room 2047 and over Zoom (link here. The password is ati).

You can find the video and materials for each class here in the schedule, after they are available. Also, please refer to links for additional sources about each class.

Please pick the slot for your presentation/test in this Google Sheet.

Tentative schedule for the lectures:

  • 12.02.2021 - Lecture 1: A Toy Model: Linear Regression (Tambet) - (video) (slides) (notebook)
  • 19.02.2021 - Practice 1: JAX Tutorial (Mikhail) - (video) (slides) (notebook tutorial 1) (notebook tutorial 2)
  • 26.02.2021 - Lecture 2: Taylor Approximations (Oriol) - (video) (slides)
  • 05.03.2021 - Practice 2: Problem Set 1: Gradient Descent with Momentum (Tarun) - (video)
  • 12.03.2021 - Lecture 3: Metrics (Raul) - (video)
  • 19.03.2021 - Practice 3: Problem Set 2: Computing the Grassmannian Length (Mykyta) - (video)
  • 26.03.2021 - Lecture 4: Second-Order Optimization (Markus) - (video)
  • 02.04.2021 - Holiday (no lecture)
  • 09.04.2021 - Practice 4: Problem Set 3: Path Energy and Geodesics (Tetiana) - (video)
  • 16.04.2021 - Lecture 5: Adaptive Gradient Methods, Normalization, and Weight Decay (Viacheslav) - (video)
  • 23.04.2021 - Lecture 6: Infinite Limits and Overparameterization (Kallol) - (video)
  • 30.04.2021 - Lecture 7: Stochastic Optimization and Scaling (Farid) - (video)
  • 07.05.2021 - Lecture 8: Bayesian Inference and Implicit Regularization (Meelis) - (video) (slides)
  • 14.05.2021 - Lecture 9: Dynamical Systems and Momentum (Florian) - (video)
  • 21.05.2021 - No lecture
  • 28.05.2021 - Lecture 10: Differential Games (Sven) - (video)

The reading materials can be found on original course homepage:
https://www.cs.toronto.edu/~rgrosse/courses/csc2541_2021/

Homework deadlines:

  • 12.03.2021 - Problem Set 1: Gradient Descent with Momentum
  • 26.03.2021 - Problem Set 2: Computing the Grassmannian Length
  • 16.04.2021 - Problem Set 3: Path Energy and Geodesics

Homework assignments can be found here (please disregard the header of the pdf, as it is specific to the reference course that we are following). Note that homework 1 corresponds to the first set of questions ("gradient descent with momentum"), and so forth for the other two sets. There is also starter code for Problem Set 2 and Problem Set 3. Grades sheet available here.

  • Institute of Computer Science
  • Faculty of Science and Technology
  • University of Tartu
In case of technical problems or questions write to:

Contact the course organizers with the organizational and course content questions.
The proprietary copyrights of educational materials belong to the University of Tartu. The use of educational materials is permitted for the purposes and under the conditions provided for in the copyright law for the free use of a work. When using educational materials, the user is obligated to give credit to the author of the educational materials.
The use of educational materials for other purposes is allowed only with the prior written consent of the University of Tartu.
Terms of use for the Courses environment