Special Course in Machine Learning: Ensemble Methods
Seminars: Thursdays 14:15, at Paabel (Ülikooli 17), room 218
The first seminar is on the 14th September! No seminar on the 7th September!
Meelis Kull, firstname.lastname@example.org, Liivi 2, room 326
Organisational slides from the first seminar:
Slides in PDF
Introductory lecture about ensemble methods:
Slides in PDF
Ensemble methods have turned out to be among the most successful approaches in machine learning. The simple idea of improving predictions by combining the outputs of several models (an ensemble of models) has led to the development of many learning algorithms. Recent studies have showed that the ensemble methods ’random forest’ and ’boosting’ are among the best families of classifiers on small and medium-sized datasets. Furthermore, many machine learning competitions on big datasets have also been won by ensembles, consisting of multiple deep neural networks.
In this course we will study the foundations and algorithms of ensemble methods. The course will start with an introductory lecture and a practice session reminding of the preliminaries and giving a brief overview of ensemble methods, making the course accessible to students with less knowledge about machine learning. This will be followed by seminars where we learn about the ensemble methods in more detail. In particular, we will discuss the details of simple and weighted averaging, majority and plurality voting, weight learning algorithms, mixture of experts. We will learn why ensemble diversity is important and what the options are in achieving it. We will discuss methods of ensemble pruning and how to learn ensembles if costs or classes are imbalanced. Hopefully, by the end of the course we will understand quite well why ensemble methods such as random forests and boosting are so powerful!
After the introductory lecture and practice session we will have seminars based on the book Ensemble Methods: Foundations and Algorithms by Zhi-Hua Zhou (2012). At the start of the course we will divide the topics between participants and prepare the schedule so that everyone will get a chance to present a topic at a scheduled week. Each week before the seminar everyone will have to read the current week’s section from the book. During the seminar some participants will present the topic, this is followed by a discussion to clarify the details and to put the new knowledge into context. Most seminars will end with a test about the reading materials.
The course is mainly based on the following book:
The introductory lecture and practice session will reuse some of the following materials:
- Lecture on ensemble methods within ML course, 2016
- Practice session on ensemble methods within ML course, 2016
This is a pass/fail course. In order to pass the course each student must:
- Attend the seminars and participate in discussions
- Read the assigned materials
- Present one topic once during the course
- Prepare and grade one test to check whether other students have understood the topic
In order to succeed in this course the students should have basic knowledge about:
- Linear algebra
- Probabilities and calculus
- Machine learning