Labor: AutoML

Prof. Dr. rer. nat. Marius Lindauer

Übungsbetreuung:

Background

Machine Learning (ML) has achieved remarkable success in recent years. However, the choice of ML algorithms (SVM, random forest or deep neural network) and their hyper-parameters is yet another process of “learning”. e.g. , designing a well performed ML system requires a lot of expert knowledge and it is often the result of repeated “trial and error”. Even worse, the “no free lunch” theorem indicates that there is no single approach that works best across every task. Hence, the above tedious process will be repeated again and again facing new tasks. To alleviate the above problems, Automated Machine Learning (AutoML) is proposed to automate the design of the whole ML pipeline, including but not limited to the techniques mentioned above. In this practical lab course, you will learn to implement the main ideas of an AutoML system from scratch and how to apply AutoML to applications.

Requirements

You should know the foundations and have hands-on experience with Machine Learning and Deep Learning, i.e., successfully attending at least one course on both of these.
You should have very good Python programming skills; everything will be done in Python in this lab course.
We strongly recommend that you successfully attended the AutoML lecture in the summer semester 2020.

Topics

Let’s start: getting familiar with a package for defining a configuration space and with AutoML benchmarks (NASBench-101, 201, 301 and HPOlib2)
Efficient ways of Hyperparameters Optimization (HPO), you will learn to implement different ways of black optimization algorithms:

Bayesian Optimization, including different surrogate models (GP, RF, NN...) and acquisition functions, and extensions to multi-fidelity optimization.
Evolutionary Algorithms, incl. different mutation operators, crossover-operators, selection schemes, etc.

Speeding up training process: meta-learning and warm-starting
Final Competition: In the last phase, you will build your own AutoML framework (based on your previous implementations) and compete against other teams on a new unseen dataset. Finally, you have to hand in a report (about 4 pages) describing the ideas and results of your system.

Literature

Dynamics

This course will be in English only and fully virtual. At the beginning of the term, we will discuss and decide whether we will meet weekly for 4h or whether we will do the lab course in an one-week block at the end of the semester.