革新知能統合研究センター

This talk will be held in a hybrid format, both in person at AIP Open Space of RIKEN AIP (Nihonbashi office) and online by Zoom. AIP Open Space: *only available to AIP researchers.

DATE, TIME & LOCATION
Friday May 18th, 11:00 – 13:00, RIKEN AIP Nihombashi Office, Open Space

TITLE
Spectral-factorized Positive-definite Curvature Learning

BIO
Wu Lin is a postdoctoral fellow at the Vector Institute. His research lies at the intersection of differential geometry and machine learning, with a particul ar emphasis on optimization and inference. Currently, his projects investigate geometric approaches with broad implications for large-scale optimization and emerging paradigms in machine learning.

ABSTRACT
Many training methods, such as Adam(W) and Shampoo, learn a positive-definite curvature matrix and a pply an inverse root before preconditioning. Recently, non-diagonal training methods have gained significant attention; however, they remain comp utationally inefficient and are often limited to specific types of curvature information due to the costly matrix root computation via matrix decomposition. To address this, we propose a Riemannian optimization approach that dynamically adapts spectral fact orized positive-defined curvature estimates, enabling the efficient application of arbitrary matrix roots and generic curvature learning. We demonstrate the efficacy and versatility of our approach in positive-definite matrix optimization and covariance adaptation for gradient-free optimization, as well as its efficiency in curvature le arning for neural net training.