Projects with DonkeyCar
DonkeyCar S1 is a 1:10 scale toy car equipped with a camera, Raspberry Pi 4b, and MM1 remote control board. It comes assembled and functional, no hardware skills are needed to use it. It has a strong and responsive community in Discord.
The DonkeyCar:
- Has a mobile app that allows you to get driving with the car in 5 minutes. Driving means collecting human-driving data that AI can learn from or that you can use to develop your computer-vision and rule-based approach!
You can train basic neural network models within the app, without even having to install anything on your computer (but it's slow, the phone is not very powerful). You just drive around, collect data, and based on the recordings of your driving, the app can train a model to imitate you.
- The car can be controlled via a mobile app, web application, or a game controller. You can choose when to record and if to delete the last 100 frames (in case of behavior you don’t want the model to learn, e.g. you crashed the car), and so on, either via application or via buttons on the controller. You can clear the data also afterward, there are tools with sufficiently comfortable UI for this.
- For more advanced models, there is a codebase that allows you to transfer recordings from the car to your computer, train a model in your computer or in Google Colabs, transfer the code/models back to the car, and launch it, let it drive.
- The codebase supports multiple model types.
- The cars can be configured to connect to any Wi-Fi network. Being in the same network you can connect to the car via ssh if you know the device name.
- The battery lasts for a few hours of driving but is a fire hazard and needs to be kept in fireproof bags when not in use. Please adhere to the safety rules regarding the batteries.
Based on prior projects, a few more words about the capabilities and limitations of the hardware:
- 180-degree turning diameter as measured by outside wheel 140-160 cm, depending on the car
- Cannot maintain speed when hardware heats up (need to manually turn up throttle)
- All driving so far has been done with end-to-end systems. So far only steering has been controlled, but it is possible to control speed as well; it just needs more data to learn.
- Raspberry Pi 4b can compute simple CNN prediction in 40ms. 25 Hz is enough frequency for driving
- Can be taught to drive between walls, or follow a line. Can avoid obstacles. Can adhere to conditional commands "turn left, turn right" or other similar (e.g. drive slow).
- Can be taught to give way to another donkey coming from the right
- Can drive indoors and outdoors
- Quite sensitive to light conditions, as expected
Thorough teaching materials guide the participants through the common pitfalls of machine learning models, such as data amount, data quality, model inference time, generalization and overfitting. Following these materials also gives good experience in using the hardware platform.
Example of model performance in the generalization/overfitting task of the teaching materials: youtube video.
After mastering the use of the car and training the first self-driving models in the toy town, participants choose a project to complete using the car for the remainder of the semester.
NOTEBOOKS:
Week 1: https://colab.research.google.com/drive/1MDQPtZuRmSC-fWNefgqBzKTtk4r9aGfR#forceEdit=true&sandboxMode=true
DELIVERABLE: video of vehicle driving good after training the large model
Week 2: https://colab.research.google.com/drive/1H_quEPa9QZmwt-NRCMwaLegst9tuajjC?usp=sharing
DELIVERABLE: video of the model performing bad after the label corruption. Uploda here: https://drive.google.com/drive/folders/138p9R_NKBrkEMUro_Y1zzT6ZNo5EP-NK?usp=sharing
Week 3: https://colab.research.google.com/drive/1NjvMyczDuak0fbGzual4nCIojcCm13C5#forceEdit=true&sandboxMode=true DELIVERABLE: video of the model performing bad in not-trained cases. Upload here: https://drive.google.com/drive/folders/138p9R_NKBrkEMUro_Y1zzT6ZNo5EP-NK?usp=sharing
Week 4: https://colab.research.google.com/drive/1P0ttwcueRawhWCv2Mb6dZxOC_BxFhsI7#forceEdit=true&sandboxMode=true Upload here: https://drive.google.com/drive/folders/138p9R_NKBrkEMUro_Y1zzT6ZNo5EP-NK?usp=sharing
Week 5: https://colab.research.google.com/drive/1LgW54886JxuXerK5GSj9EQSOvWpudYbS#forceEdit=true&sandboxMode=true Upload here: https://drive.google.com/drive/folders/138p9R_NKBrkEMUro_Y1zzT6ZNo5EP-NK?usp=sharing
PROJECT
DELIVERABLES: Project report in the form of a blog post. Must contain videos of actual autonomous driving, experimentations.
OPENLY AVAILABLE MATERIALS FROM PREVIOUS YEARS (COOL VIDEOS) https://drive.google.com/drive/folders/1MUAlXCHcrs0sn2VCE76IqvsahIm_SpEs