革新知能統合研究センター

Date and Time: June 27, 2025, 10:30 – 11:30 (JST)
Venue: Online and Meeting Room C at the RIKEN AIP Nihonbashi office
*Meeting Room C is available to AIP researchers only

Title: Graph Transformers for Molecular Science — Overcoming Limitations in Graph Representation Learning

Speaker: Bresson Xavier (National University of Singapore)

Abstract: Graph Neural Networks (GNNs) have demonstrated remarkable potential in graph representation learning. Traditional GNNs rely on local message-passing mechanisms that propagate information through the graph by stacking multiple layers. However, this approach is limited by two key issues: over-squashing of information and difficulty in capturing long-range dependencies. In this work, we introduce a novel approach to address these challenges by leveraging the Visual Transformer architecture, originally designed for computer vision tasks. We present Graph ViT (Graph Vision Transformer), a new class of Graph Transformers that incorporates three critical advantages. First, Graph ViT effectively captures long-range dependencies, as shown on the long-range LRGB datasets, and mitigates the over-squashing issue, as demonstrated on the TreeNeighbour dataset. Second, it enhances memory and computational efficiency, outperforming existing methods. Third, Graph ViT exhibits high expressive power in graph isomorphism, being capable of distinguishing graphs that are 3-WL isomorphic. Consequently, this innovative architecture achieves significantly improved performance over traditional message-passing GNNs, especially in molecular science applications.

Bio: Xavier Bresson (PhD 2005, EPFL, Switzerland) is Associate Professor in Computer Science at NUS, Singapore. He is a leading researcher in the field of Graph Deep Learning, a new framework that combines graph theory and deep learning techniques to tackle complex data domains in natural language processing, computer vision, combinatorial optimization, quantum chemistry, physics, neuroscience, genetics and social networks.