Center for Advanced Intelligence Project

Four papers from the AIP Center have been accepted for presentation at the 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL 2025), one of the leading international conferences in the field of computational linguistics and natural language processing.

NAACL 2025 will be held from April 29 to May 4, 2025, in Albuquerque, New Mexico, USA.

[NAACL 2025] https://2025.naacl.org/

• Accepted papers

[Main Conference]

• Repetition Neurons: How Do Language Models Produce Repetitions?
  Tatsuya Hiraoka（MBZUAI / RIKEN AIP)
  Kentaro Inui （MBZUAI / RIKEN AIP)
• STEP: Staged Parameter-Efficient Pre-training for Large Language Models
  Kazuki Yano (Tohoku University)
  Takumi Ito (Tohoku University / Langsmith Inc.)
  Jun Suzuki (Tohoku University / RIKEN AIP / NII LLMC)
• The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces
  Ahmed Oumar El-Shangiti (MBZUAI / RIKEN AIP)
  Tatsuya Hiraoka （MBZUAI / RIKEN AIP)
  Hilal AlQuabeh (MBZUAI)
  Benjamin Heinzerling (RIKEN AIP / Tohoku University)
  Kentaro Inui （MBZUAI / Tohoku University / RIKEN AIP)

[Findings]

• Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inference
  Go Kamoda (Tohoku University)
  Benjamin Heinzerling (RIKEN AIP / Tohoku University)
  Tatsuro Inaba (Kyoto University)
  Keito Kudo (Tohoku University)
  Keisuke Sakaguchi (Tohoku University / RIKEN AIP)
  Kentaro Inui (Tohoku University / RIKEN AIP / MBZUAI)

Related Laboratories