Homework 1: DistributedDataParallel — Scaling, Communication & Limits. Homework 2: DDP Optimization Homework 3: Model Parallelism Homework 4: Pipeline Parallelism