CHALLENGES/PROJECT INFORMATION

If you are graduating this year and need the final grade in ois earlier, you need to submit final report by that date and notify us in advance!

Now that homeworks and tests are over, it is time to build something real and applicable. You had lectures with more complex networks, now you can learn to implement them and to use hpc cluster for their training. We propose to you the following three options to get your 40 points. Read through them, select your favorite and fill out the form at the bottom of this page with your choice:

Track 1 - Deep Reinforcement Learning Challenge [responsible T.A.: Jesus]

You can train your own agent and build a Maze. You will learn about Q-Learning, DRL, how to optimize it and build your own environments. As you had a lecture about RL already, you have an idea how exciting reinforcement learning is.

Grading and timeline for [Track 1]:

Week 1 - [ 08/05/2025 ]
Tutorial environment

Week 1 - Code
Week 1 - Presentation

In the code you will find two notebooks, your task is to reverse engineer the process described in the notebook 02_QTable.ipynb to learn how to interact with the environment (q_learner.py) and how to train an agent using tabular Q-Learning (qtable.py).
NB If you didn't delivered your diagram (draw by hand) please submit the photos of your diagram in here!!

Week 2 - [ 15/05/2025 ]
Q-Learning, personal QAs, tutorial code (10 points), Deep Learning notebook
Week 2 - Code
Week 2 - Presentation

The code contains an implementation of a linear neural network, the task for week 02 is to come up with at least two variations (two notebooks) of this architecture, the goal is to learn while keeping the network simple.

Week 3 - [ 22/05/2025 ]
Personal QAs, DL code (10 points), Optimization

So far we have learned how the code, for solving a maze works and we learned two ways to tackle the task of training an artificial agent to reach a goal from an starting point.

Now is your turn to come up with your own architecture!!!

You can try any creative architecture you want, as complex as you wish, remember to try different mazes to observe how your learning algorithm behaves under different circumstances.
Upload your code in the next section!!!

Week 4 - [ 29/05/2025 ]
Personal QAs, optimization code (10 points), testing unknown environment

It has been fun until now to experiment with different fixed environments, but it is boring to see always the same landscape.

This week we are evaluating all that we have learned during the past weeks with three different and challenging environments.

Every maze gives you points. 1. Starting point at (5, 2). 3 points 2. Starting point at (0, 0). 3 points 3. Starting point at (0, 0). 4 points
The code attached includes the matrixes and the update of the q_learner.py file to modify the ending point of the task.

Week 4 - Code

Week 5 - [ 05/06/2025 ]

Generalization test (10 points)

This is the last session and the final evaluation.

The task this time is a Q&A evaluation for you!!!

Instructions.
1. Train your best architecture with the train maze.
2. Test your trained model with the test maze, starting in different starting points.
---------- Week 5 - Code---------------

3. Bring to the session your notebook in your computer or printed, and any other material you consider necessary to explain your experience.
4. I'll ask you about your experiment in a session of 5 minutes maximium and get your last evaluation from this interview.

Note: If you are not able to attend please send me a message to evaluate another time slot.

Track 2 - Deep Noise Suppression Challenge [responsible T.A.: Artem]

you have to develop a deep speech enhancement model for full band audio. The final goal of the project is to create a full pipeline from data processing to the evaluation metrics and train a deep neural network for noise suppression. The project is based on series of Microsoft DNS challenges and adapted to our course.

Grading and timeline for [Track 2]:
This track has several building blocks and by Week 3 [29 May] you should have any of 4/6 blocks working (regardless of its performance) which you show during practice session, then you guarantee yourself 25 points. By Week 5 [12 June] you have to complete all steps (finishing remaining steps and improving performance + 5 points) and submit final report, which gives you the remaining 10 points.
1) Data Loader implementation
2) Data Preprocessing
3) Model implementation
4) Data postprocessing
5) Metrics / evaluation scripts
6) Report (literature review, experiments setup, model architecture, results, ablation study)

Code repository: https://gitlab.ut.ee/artem.domnich/nnp2025_track02

Submit your code and final report here!

Track 3 - The project of your choice [responsible T.A.s: Marharyta & Victor]

You can come up with your own project. In this case during the first week of challenges you need to pitch us your project idea, so we can validate the scope and requirements.

Grading and timeline for Personal Project:
7th May – project pitch, , practice session time (10 points)
28th May – similarly to previous track you need to demonstrate that you have MVP of your project (data preprocessing, some model is working, you have some measures of performance running) – 15 points
11th Jun – final report (10 points + 5 for model improvements)

Tracks 1 (led by Jesus) and 2 (led by Artem) are guided individual challenges, where you will receive the information gradually. During practice session, we will help you if you are stuck anywhere and we will complete the challenge in 4 weeks. Track 3, personal project, can be done individually or in pairs and you can ask your questions from Victor on Wednesday.

Google form

To estimate time and resources, let us know which option you would like to choose by filling out this form: google form.