Hyperparameter tuning is an omnipresent problem in machine learning as it is an integral aspect of obtaining the state-of-the-art performance for any model. Most often, hyperparameters are optimized just by training a model on a grid of possible hyperparameter values and taking the one that performs best on a validation sample. In this seminar, we will look at the commonly used approaches to hyperparameter optimization, how to leverage knowledge transfer to save time in tuning an algorithm on a new data set, and structursal hyperparameter tuning!