Description
【The 11th Seminar】
Date and Time: March 16th 6:00pm – 7:00pm(JST) 10:00am-11:00pm(CET)
Venue:Zoom webinar
Language: English
Speaker: Lénaïc Chizat, EPFL-CIS
Title: Mean-Field Langevin Dynamics: convergence and applications
Abstract:
The Langevin algorithm is a standard method to minimize, in the space of probability measures, the sum of a linear functional and the entropy. In this talk, motivated by the analysis of noisy gradient descent on wide two-layer neural networks, we consider the « Mean-Field Langevin Dynamics », a nonlinear generalization of the Langevin dynamics that minimizes the sum of a convex functional and the entropy. We show that, under a certain uniform log-Sobolev assumption, the dynamics converges exponentially fast to global minimizers (this result was also proven independently in [Nitanda et al. 2022]). We also present the « simulated annealing » variant of this dynamics, and show that for a suitable noise decay, it converges in value to the global minimizer of the convex functional. As a consequence of these abstract results, we obtain a convergence rate for noisy gradient descent on certain infinitely wide two-layer neural networks. Other applications will be discussed as well, such as the grid-free computation of Wasserstein barycenters.
Bio:
Lénaïc Chizat is a tenure track assistant professor at EPFL in the Institute of Mathematics, where he leads the DOLA chair (Dynamics of Learning Algorithms). Until 2021, he was a CNRS researcher at Laboratoire de mathématiques d’Orsay in France. In his research, he studies and develops optimization algorithms for machine learning and signal processing. He has in particular contributed to the following fields: optimization in the space of measures, theory of optimal transport, and the analysis of artificial neural networks.
