Center for Advanced Intelligence Project

**The recent advancements in large language models (LLMs) have significantly enhanced their capabilities, leading to a paradigm shift in how AI can be integrated into our society.** This is particularly evident in the case of ChatGPT, including the latest iteration, GPT-4. It is intriguing how the straightforward approach of Transformer models trained on massive datasets has achieved such remarkable progress, which should pique our interest on both conceptual and technical levels.

This mini-workshop aims to provide tutorial-style talks introducing the technology behind GPT-4 and discussing its implications for society. The technical talks are tailored for researchers in the field of AI.

**Program:**

**Day 1 – June 30th**

– 14:30 – 14:50 *Matthias Weissenbacher* Opening Remarks: Transformers and the Power of GPT4
– 14:50 – 15:15 (5 min Q&A) *Noriki Nishida* Introductory Talk: Standard supervision vs. context learning in NLP
– 14:15 – 15:40 (5 min Q&A) *Benjamin Heinzerling* Why can LMs learn in-context? An overview of current theories
– 15:40-16:10 (10 min Q&A) *Koiti Hasida* Future R&D directions of generative AI & social impacts

**Day 2 – July 3rd**

– 13:30 – 13:35 Opening Remarks – Day 2
– 13:35 – 14:00 (5 min Q&A) *Hiroshi Nakagawa* How to live together with ChatGPT?
– 14:00 – 14:40 (10 min Q&A) *Hiroki Teranishi* How to train large language models on the example of GPT3 (LLaMA)
– 14:40 – 15:20 (10 min Q&A) *Yuji Matsumoto* Instruct GTP and the role of experts feedback (via RL) – example ChatGPT/GPT4
– 15:20 – 15:30 *Koiti Hasida* (Moderator) Open Discussion on the Future of AI (AGI) and Humanity