Transfer Learning and Invariance for Bayesian Optimization

Project

Transfer Learning and Invariance for Bayesian Optimization

Principal Investigator

Boston College

Oracle Principal Investigator

Michael Wick, Principal Research Scientist

Summary

Bayesian Optimization (BO) is used to perform hyperparameter optimization for Machine Learning models. Hyperparameter optimization can result in significant improvements to model performance, but it is computationally expensive, especially for large neural models.

This research is aimed at determining whether Bayesian Optimizations can be transferred to newer models. For example, for a given task and model, can we transfer from a Bayesian Optimization on low quality data to one on high quality data? Can we transfer from a Bayesian Optimization on a small model to one with more parameters?