Description
The 72nd Seminar
Date and Time: August 7th 1:15 pm – 2:15 pm(JST)
Venue: Zoom webinar from Nihonbashi, Tokyo
Language: English
Speaker: Yutong He (Carnegie Mellon University)
Title: Controllable Image Generation FOR FREE with Diffusion Models
Short Abstract:
Generative modeling has gained tremendous success in recent years. From ChatGPT to Stable Diffusion, generative models have shown great potential in improving productivity in many real life applications. However, to control the generation results, it often requires the practitioners to collect task-specific training data, design and train separate model architectures, and/or spend additional inference time in order to obtain the desired outcomes. In this talk, I would like to discuss ways to control image generative models, more specifically diffusion models, with none of these obstacles: practitioners can use these methods without any training, model part design or additional inference time. The talk will contain a brief review of the general framework of diffusion models, an introduction of two papers of mine that tackle the conditional (text-to-)image generation task with no extra cost from different perspectives, and the open problems and potential applications with these methods.
Bio:
Yutong (Kelly) He is a first-year PhD student in the Machine Learning Department, School of Computer Science at Carnegie Mellon University, advised by Prof. Zico Kolter and Prof. Ruslan Salakhutdinov. Before coming to CMU, she was a master’s student at Stanford Computer Science with distinction in research. She was advised by Prof. Stefano Ermon, and closely worked with Prof. Christopher Manning, Prof. David Lobell and Prof. Marshall Burke. She completed her B.S in Mathematics and B.S. in Data Science with highest distinction at University of Rochester, where she worked with Prof. Henry Kautz and Prof. Jiebo Luo. She was selected as a Siebel Scholar and a Xerox Engineering Research Fellow. She was also awarded an Outstanding Paper Award at ICLR 2022 and Doris Ermine Smith Award for Achievement in Mathematics. Her research interests include generative models, representation learning, computational sustainability and broad deep learning topics in general.
