Center for Advanced Intelligence Project

Date, Time and Location:
July 1, 2025, 15:00 – 16:00 (JST)
This lecture will be held both in person at the AIP open space and online by Zoom.
*AIP open space is available to AIP researchers only
Title: Gaussian Process Tilted Nonparametric Density Estimation using Fisher Divergence Score Matching
Abstract: We present three Fisher divergence (FD) minimization algorithms for learning Gaussian process (GP) based score models for lower dimensional density estimation problems. By representing the GP part of the score as a linear function using the random Fourier feature (RFF) approximation, we show that all learning problems can be solved in closed form. This includes the basic and noise conditional versions of the Fisher divergence, as well as a novel alternative to noise conditional FD models based on variational inference (VI). Here, we propose using an ELBO-like optimization of the approximate posterior with which we derive a Fisher variational predictive distribution. The RFF representation of the GP, which is functionally equivalent to a single layer neural network score model with cosine activation, provides a unique linear form for which all VI expectations are in closed form. We demonstrate our three learning algorithms, as well as a MAP baseline algorithm, on several low dimensional density estimation problems.
Bio: John Paisley is an Associate Professor in the Department of Electrical Engineering at Columbia University, where he is also a member of the Data Science Institute. His research interests include Bayesian models and inference, with applications to machine learning problems. Before joining Columbia in 2013, he was a postdoctoral researcher in the computer science departments at Princeton University and UC Berkeley. He received the BSE and PhD degrees in Electrical and Computer Engineering from Duke University in 2004 and 2010, respectively.