Institute of Computer Science
  1. Courses
  2. 2025/26 fall
  3. Transformers (MTAT.06.055)
ET
Log in

Transformers 2025/26 fall

  • Main
  • Lectures
  • Labs
  • Homeworks
  • Projects
  • Grades

Transformers

Transformers are a short name for neural networks with self-attention, multiple heads and several layers, and for the last couple of years they have been the source of state-of-the-art solutions in several AI solutions. Think ChatGPT, translate.ut.ee or Google Translate, neurokone.ee -- all actually different applications of transformers. Very recently they have shown great results with image and speech data, so their future is looking bright.

At the lectures we will discuss the core transformer architecture, applications and associated considerations. The practical part assumes that you are comfortable with Python and ML, and will teach you to train models in Colab, run things on HPC GPUs, etc.

There will be 4 homeworks as well as a project, where you are also free to focus on whichever kind of data you want (text, speech, image, time series, etc.).

Instructors:

Mark Fishel (lectures)

Agnes Luhtaru, Maksym Del, Taido Purason, Martin Vainikko (labs)

  • Institute of Computer Science
  • Faculty of Science and Technology
  • University of Tartu
In case of technical problems or questions write to:

Contact the course organizers with the organizational and course content questions.
The proprietary copyrights of educational materials belong to the University of Tartu. The use of educational materials is permitted for the purposes and under the conditions provided for in the copyright law for the free use of a work. When using educational materials, the user is obligated to give credit to the author of the educational materials.
The use of educational materials for other purposes is allowed only with the prior written consent of the University of Tartu.
Terms of use for the Courses environment