CARLA Project
In this semester Autonomous Driving Lab (ADL) organizes a seminar to participate CARLA Autonomous Driving Challenge 2022, part of Machine Learning for Autonomous Driving Workshop at NeurIPS 2022. Your task in this challenge is to train self-driving car to drive around in a simulated city.
If your team manages to get to top-10 in CARLA Leaderboard, ADL sponsors visit of NeurIPS 2022 in New Orleans (November 28th - December 3rd) for two members of the team, including flight, accommodation and conference ticket.
Prerequisites
Knowledge of machine learning and neural networks is essential, everything else you can learn on the go. Having PC/laptop with GPU is handy for debugging. It helps to be comfortable running GPU processes (including headless OpenGL) in Rocket cluster.
Deliverables
To earn the credits for this course you need to:
- Present one paper from CARLA leaderboard in weekly seminar.
- Submit the results of your team to CARLA leaderboard by November 7th.
- Submit technical report (up to 4 pages) describing your submission by November 24th.
Classes
- Weekly seminars will be held in Delta room 1021 on Wednesdays at 16:15.
- Additionally you can use computer class in room 2006 on Tuesdays at 12-14. This computer class has PCs with Nvidia GTX 2070 GPUs, suitable for running CARLA. Instructions how to boot these computers to Ubuntu were given during the first practice session.
Schedule
| Date | Topic | Presenter |
|---|---|---|
| 2022-08-31 | Introduction to CARLA | Tambet |
| 2022-09-05 | Training materials revealed | - |
| 2022-09-06 | CARLA introduction in computer class | Tambet |
| 2022-09-07 | Introduction to end-to-end driving lecture (slides) (old video) | Tambet |
| 2022-09-13 | Computer class reserved for independent work | - |
| 2022-09-14 | End to End Learning for Self-Driving Cars presentation | Pavlo |
| 2022-09-20 | Computer class reserved for independent work | - |
| 2022-09-21 | Learning by Cheating presentation | Matevž |
| 2022-09-27 | Computer class reserved for independent work | - |
| 2022-09-28 | InterFuser: Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer presentation | Uroš |
| 2022-10-04 | Computer class reserved for independent work | - |
| 2022-10-05 | TransFuser/NEAT video, (Latent) TransFuser video, discussion | - |
| 2022-10-11 | Computer class reserved for independent work | - |
| 2022-10-12 | Philipp Krähenbühl CVPR 2022 WAD workshop keynote, discussion | - |
| 2022-10-18 | Computer class reserved for independent work | - |
| 2022-10-19 | Learning to Drive from a World on Rails video, discussion | - |
| 2022-10-25 | Computer class reserved for independent work | - |
| 2022-10-26 | NEAT: Neural Attention Fields for End-to-End Autonomous Driving video, discussion | - |
| 2022-11-01 | Computer class reserved for independent work | - |
| 2022-11-02 | Everybody is busy with final submission | - |
| 2022-11-07 | Challenge closure, last day to submit | - |
| 2022-11-10 | Results notification | - |
| 2022-11-24 | Deadline for technical report | - |
| 2022-12-03 | ML4AD workshop at NeurIPS 2022 in New Orleans | - |
Future perspective
We are working on our own Tartu simulator in CARLA. Once your solution works well enough in CARLA standard towns, we can try it out in Tartu simulation. If it works well enough there, we might give it a try with a real car. For that you probably would need to work on sim-to-real transfer, which is an excellent thesis topic.
Contact
For more details contact Tambet Matiisen, tambet.matiisen@ut.ee, Delta room 3095.