Keeletehnoloogia seminar 2024/25 sügis

Topics

A list of topics for each seminar is provided below. Topics are subject to change. You can expect more detailed information about the subtopics approximately one week before each seminar via the subtopics registration sheet (log into courses to see the link) .

1. September 13: Intro
   • Short seminar, no paper
2. September 20: Data curation and pretraining
   • Papers: Dolma and OLMo
   • Submit your overview here!
3. September 27: Instruction and Parameter-Efficient Fine-Tuning
   • Papers: Instruction Tuning Survey and QLoRA
   • Submit your overview here!
4. October 4: Reinforcement Learning from Human Feedback (RLHF)
   • Papers: Direct Preference Optimization (DPO), Zephyr and RLAIF vs. RLHF
   • Submit your overview here!
5. October 11: Prompt Engineering
   • Papers: Prompt Engineering Guide, Self-Refine and Revisiting Demonstration Selection
   • Submit your overview here!
6. October 18: Inference-time Algorithms
   • Papers: From Decoding to Meta-Generation
   • Submit your overview here!
7. October 25: Model Merging
   • Papers: Task Arithmetic, TIES-Merging and DARE
   • Submit your overview here!
8. November 1: Mixture of experts (MoE)
   • Papers: Switch Transformers and OLMoE
   • Submit your overview here!
9. November 8: Retrieval-Augmented Generation (RAG)
   • Papers: RAG, Fine-Tuning or Retrieval and RAG or Long-Context LLMs
   • Submit your overview here!
10. November 15: LLM Agents
    • Papers: LLM Powered Autonomous Agents, Toolformer and WebArena
    • Submit your overview here!
11. November 22: Evaluation and Benchmarking
    • Papers: Reproducible Evaluation, Judging LLM-as-a-Judge and GSM-Symbolic
    • Submit your overview here!
12. November 29: LLM Safety
13. December 6: Varia: Knowledge distillation, fusion, editing
14. December 13: -
15. December 20: -