説明
DATE and Time: Friday May 9th, 11:00 – 13:00 JST
VENUE: RIKEN AIP Nihonbashi Office, Open Space
TITLE: Model Merging – What, Why, and How
SPEAKER: Prof. Emanuele Rodolà
ABSTRACT:
In this talk, I will introduce the emerging field of model merging — the process of combining multiple neural networks into a single model without retraining. We’ll begin with foundational concepts such as linear mode connectivity and task vectors, and explore two main settings: (1) merging models trained from scratch on the same task but with different initializations, and (2) merging models finetuned on different tasks from a shared pretrained base. I will then present a series of recent works of mine that expand the model merging toolkit. These include the use of cycle consistency in permutation-based merging, insights into how task vectors relate to gradients, SVD-based approaches for low-rank model combination, and the application of evolutionary algorithms to discover optimal merging coefficients. Throughout, we’ll see how these techniques can be applied in real-world scenarios, from model compression in Computer Vision to state-of-the-art synthesis of LLMs for low-resource languages.
BIO:
Emanuele Rodolà is a Full Professor of Computer Science at Sapienza University of Rome, where he leads the GLADIA AI group. His work in this field is supported by an ERC grant, a FIS grant, and a Google Research Award. In the past, he was a postdoctoral researcher at USI Lugano (2016–2017), an Alexander von Humboldt Fellow at TU Munich (2013–2016), and a JSPS Research Fellow at the University of Tokyo (2013), in addition to visiting periods at Tel Aviv University, Technion, École Polytechnique, and Stanford. He is a fellow of ELLIS and the only Italian AI researcher to be a fellow of the Young Academy of Europe. Professor Rodolà has received numerous awards for his research and plays an active role in the academic community, serving on program committees and as Area Chair for major conferences in AI and ML. His current research focuses primarily on neural model fusion, representation learning, ML for audio and music, and multimodal learning, with around 170 publications in these areas. His work has been featured in media outlets including Fortune, Wired, Italian national broadcast and newspapers.
