AI Lunchtime Seminar- Bethe Lectures, Professor Lenka Zdeborová, École Polytechnique Fédérale de Lausanne

Title: On Generalization and Uncertainty in Learning with Neural Networks

Host:  Thorsten Joachims

Abstract:  Statistical physics has studied exactly solvable models of neural networks for more than four decades. In this talk, Lenka Zdeborová will examine this work in the perspective of recent questions stemming from deep learning. She will describe several types of phase transitions that appear in the high-dimensional limit as a function of the amount of data. Discontinuous phase transitions are linked to adjacent algorithmic hardness. This so-called hard phase influences the behavior of gradient-descent-like algorithms. She will show a case where the hardness is mitigated by overparametrization, proposing that the benefits of overparametrization may be linked to the usage of a specific type of algorithm. She’ll then discuss the overconfidence of overparametrized neural networks and evaluate methods to mitigate it and calibrate the uncertainty.