Institute of Computer Science
Courses.cs.ut.ee Institute of Computer Science University of Tartu
  1. Courses
  2. 2025/26 spring
  3. Parallelism in Deep Learning (LTAT.06.030)
ET
Log in

Parallelism in Deep Learning 2025/26 spring

  • Pealeht
  • Loengud
  • Laborid
  • Viited

Course syllabus:

Part 1: Foundations of Parallelism & Deep Learning (Weeks 1–4)

  • Week 1: Introduction & Mathematical Refresher
  • Week 2: Hardware & Parallelism Basics
  • Week 3: Profiling & Bottlenecks
  • Week 4: Overview of Parallelism Strategies

Part 2: Core Parallel Strategies in Practice (Weeks 5–11)

  • Week 5: Data Parallelism with DataParallel (DP)
  • Week 6: DistributedDataParallel (DDP) & Collective Communication
  • Week 7: DDP Optimization and Debugging
  • Week 8: Model Parallelism (Layer-wise)
  • Week 9: Pipeline Parallelism
  • Week 10: Hybrid Parallelism
  • Week 11: Recap and Project Q&A

Part 3: Project Work and Assessment (Weeks 12–16)

  • Week 12: Project Work Session 1
  • Week 13: Project Work Session 2
  • Week 14: Project Work Session 3
  • Week 15: Final Project Presentations
  • Week 16: Assessment
  • Institute of Computer Science
  • Faculty of Science and Technology
  • University of Tartu
In case of technical problems or questions write to:

Contact the course organizers with the organizational and course content questions.
The proprietary copyrights of educational materials belong to the University of Tartu. The use of educational materials is permitted for the purposes and under the conditions provided for in the copyright law for the free use of a work. When using educational materials, the user is obligated to give credit to the author of the educational materials.
The use of educational materials for other purposes is allowed only with the prior written consent of the University of Tartu.
Terms of use for the Courses environment