NLP seminar 2024/25 fall

Topics

A list of topics for each seminar is provided below. Topics are subject to change. You can expect more detailed information about the subtopics approximately one week before each seminar via the subtopics registration sheet (log into courses to see the link) .

1. September 13: Intro
   • Short seminar, no paper
2. September 20: Data curation and pretraining
   • Papers: Dolma and OLMo
   • Submit your overview here!
3. September 27: Instruction and Parameter-Efficient Fine-Tuning
   • Papers: Instruction Tuning Survey and QLoRA
   • Submit your overview here!
4. October 4: Reinforcement Learning from Human Feedback (RLHF)
   • Papers: Direct Preference Optimization (DPO), Zephyr and RLAIF vs. RLHF
   • Submit your overview here!
5. October 11: Prompt Engineering
   • Papers: Prompt Engineering Guide, Self-Refine and Revisiting Demonstration Selection
   • Submit your overview here!
6. October 18: Inference-time Algorithms
   • Papers: From Decoding to Meta-Generation
   • Submit your overview here!
7. October 25: Model Merging
   • Papers: Task Arithmetic, TIES-Merging and DARE
   • Submit your overview here!
8. November 1: Mixture of experts (MoE)
   • Papers: Switch Transformers and OLMoE
   • Submit your overview here!
9. November 8: Retrieval-Augmented Generation (RAG)
   • Papers: RAG, Fine-Tuning or Retrieval and RAG or Long-Context LLMs
   • Submit your overview here!
10. November 15: LLM Agents
    • Papers: LLM Powered Autonomous Agents, Toolformer and WebArena
    • Submit your overview here!
11. November 22: Evaluation and Benchmarking
    • Papers: Reproducible Evaluation, Judging LLM-as-a-Judge and GSM-Symbolic
    • Submit your overview here!
12. November 29: LLM Safety
    • Papers: Compromising Safety, Persuading LLMs to Jailbreak and Grounding LLMs in Privacy Laws
    • Submit your overview here!
13. December 6: Varia: Knowledge distillation, fusion, and editing
    • Papers: MiniLLM, Knowledge Fusion and Learning to Edit
14. December 13: -
15. December 20: -