Description
【The 5th Seminar】
Date and Time: November 17th 6:00pm – 7:00pm(JST)
10:00am-11:00pm(CEST)
Venue:Zoom webinar
Language: English
Speaker: Martin Jaggi, EPFL CIS
Title: Learning with Strange Gradients
Abstract:
Gradient methods form the foundation of current machine learning. A vast literature covers the use of stochastic gradients being simple unbiased estimators of the full gradient of our objective. In this talk, we discuss four applications motivated from practical machine learning, where this key assumption is violated, and show new ways to cope with gradients which are only loosely related to the original objective. We demonstrate that algorithms with rigorous convergence guarantees can still be obtained in such settings, for
1) federated learning on heterogeneous data,
2) personalized collaborative learning,
3) masked training of neural networks with partial gradients,
4) learning with malicious participants, in the sense of Byzantine robust training.
Bio:
Martin Jaggi is a Tenure Track Assistant Professor at EPFL, heading the Machine Learning and Optimization Laboratory. Before that, he was a post-doctoral researcher at ETH Zurich, at the Simons Institute in Berkeley, and at École Polytechnique in Paris. He has earned his PhD in Machine Learning and Optimization from ETH Zurich in 2011, and a MSc in Mathematics also from ETH Zurich. He is a Fellow of ELLIS, and a co-founder of EPFL’s Applied Machine Learning Days.
