Topics
A list of topics for each seminar is provided below. Topics are subject to change. You can expect more detailed information about the subtopics approximately one week before each seminar via the subtopics registration sheet (log into courses to see the link) .
- September 13: Intro
- Short seminar, no paper
- September 20: Data curation and pretraining
- September 27: Instruction and Parameter-Efficient Fine-Tuning
- Papers: Instruction Tuning Survey and QLoRA
Submit your overview here!
- October 4: Reinforcement Learning from Human Feedback (RLHF)
- Papers: Direct Preference Optimization (DPO), Zephyr and RLAIF vs. RLHF
Submit your overview here!
- October 11: Prompt Engineering
- Papers: Prompt Engineering Guide, Self-Refine and Revisiting Demonstration Selection
Submit your overview here!
- October 18: Inference-time Algorithms
- Papers: From Decoding to Meta-Generation
Submit your overview here!
- October 25: Model Merging
- Papers: Task Arithmetic, TIES-Merging and DARE
Submit your overview here!
- November 1: Mixture of experts (MoE)
- Papers: Switch Transformers and OLMoE
Submit your overview here!
- November 8: Retrieval-Augmented Generation (RAG)
- Papers: RAG, Fine-Tuning or Retrieval and RAG or Long-Context LLMs
Submit your overview here!
- November 15: LLM Agents
- Papers: LLM Powered Autonomous Agents, Toolformer and WebArena
Submit your overview here!
- November 22: Evaluation and Benchmarking
- Papers: Reproducible Evaluation, Judging LLM-as-a-Judge and GSM-Symbolic
Submit your overview here!
- November 29: LLM Safety
- Papers: Compromising Safety, Persuading LLMs to Jailbreak and Grounding LLMs in Privacy Laws
Submit your overview here!
- December 6: Varia: Knowledge distillation, fusion, and editing
- Papers: MiniLLM, Knowledge Fusion and Learning to Edit
Submit your overview here!
- December 13: Large Multimodal Models
- December 20
- Paper: Selective Language Modeling
Submit your overview here!