Fish & Richardson P.C.
Multi-task learning using knowledge distillation
Methods, systems, and apparatus, including computer programs encoded on computer storage media for performing multi-task learning. In one method a system obtains a respective set of training data for each of multiple machine learning tasks. For each of the machine learning tasks, the system configures a respective teacher machine learning model to perform the machine learning task by training the teacher machine learning model on the training data. The system trains a single student machine learning model to perform the multiple machine learning tasks using (i) the configured teacher machine learning models, and (ii) the obtained training data.
Google LLC
G G G G G
G06F G06F G06N G06N G06F
inventional inventional inventional inventional inventional
Wolfgang Melvin Jose Junyoung Michael
Macherey Johnson Premkumar Chung Schuster