Lectures
- Given by Mark, mark@tartunlp.ai
- Mondays at 12:15 in Delta 1007
- Open to anyone interested
- Recordings and slides available after the lecture
- Lecture videos end up here
- Lecture slides end up here
| 1. | Sep 11 | Motivation and overview |
| 2. | Sep 18 | Transformer basics |
| 3. | Sep 25 | Language models |
| 4. | Oct 9 | Large language models |
| 5. | Oct 23 | Machine translation |
| 6. | Oct 30 | Multilinguality |
| 7. | Nov 13 | Image processing |
| 8. | Nov 20 | Speech processing (video, slides) |
| 9. | Nov 27 | Multimodality and multitasking |
| 10. | Dec 4 | Efficiency |
| 11. | Dec 11 | Knowledge representation |
| 12. | Dec 18 | Future trends |
Notes for self:
- image processing (ViT; CLIP; ImageBind; vector embedding vs whole attention on output)
- speech recognition and synthesis (spectrograms; wav2vec2 / XLSR; CLAP)
- bio and health data
- efficiency (distillation; quantization; pruning; non-autoregressive decoding;
- multimodality () and multi-tasking (T5, BART, others)
- knowledge representation in transformers, probing, BERTology
- future trends