Automatic Budget Determination for HyperBand
Im Rahmen des Projekts AutoML
A well known approach for efficient automated machine learning (AutoML) is to stop the evaluation of model training when it does not look promising, e.g., by looking at learning curves of deep neural networks. One simple but robust approach is successive halving (and its extension HyperBand), which simply throws away half of the candidate models at user-defined budget points, e.g., after 4, 8, 16 and 32 epochs.
The drawback of this approach is that the users need expert knowledge to define these budget points. The task of this project is to study different techniques that learn a policy from data, automatically determining the budget points. This will further improve the usability and hopefully also the performance of well known AutoML tools, such as BOHB and SMAC.
We strongly recommend that students should have hands-on experience for the following topics:
- Python programming
- Machine Learning
- Deep Learning
- Optional: reinforcement learning
To apply for this project, please send us an email with the following information:
- Which ML-related courses have you taken?
- Can you please attach your transcript of records?
- Which projects have you done so far (in Hannover and elsewhere)?
- Which topics interest you most?
Since different projects require different skill sets, please also rate your skills in the following categories on a scale from ++ (very good) to -- (no knowledge/skill):
- Creativity / ideas for developing new algorithms
- Getting someone else's large code base to run
- Running comprehensive experimental studies / keeping track of results
- Self-motivation to push through even if things don't work for a while
- Coding skills
- Ability to read a RL paper, implement it and get it to work
- Ability to read a DL paper, implement it and get it to work
Ansprechpartner: Marius Lindauer