Our weekly SRI Seminar Series welcomes Adam Tauman Kalai, a technical staff member and research scientist at OpenAI working on AI safety and ethics. In this talk, Kalai will explore the phenomenon of “hallucinations” in large language models, demonstrating how these occurrences are naturally encouraged through the pre-training of language models and suggesting methods for mitigation. This session will be moderated by Nisarg Shah.
Talk title:
“When calibration goes awry: Hallucination in language models”
Abstract:
“Hallucinations” are a major problem for language models. We shed light on this phenomenon by showing that calibration, which is naturally encouraged during the pre-training of language models, leads to hallucinations. Moreover, the rate of hallucinations depends on the domain via the classic Good-Turing estimator. Interestingly, this estimate is small for facts like paper titles, which have been a notorious source of hallucinations. The analysis also suggests methods for mitigating hallucinations. This is joint work with Santosh Vempala and was done while the speaker was at Microsoft Research New England.
Suggested reading:
Kalai, Adam Tauman and Vempala, Santosh S. “Calibrated Language Models Must Hallucinate” in Proceedings of the 56th Annual ACM Symposium on Theory of Computing (STOC), 2024.
About Adam Tauman Kalai
Adam Tauman Kalai is a member of technical staff and research scientist at OpenAI working on AI safety and ethics. He has worked in multiple fields, including algorithms, fairness, machine learning theory, game theory, and crowdsourcing. He received his BA from Harvard and PhD from Carnegie Mellon University. Kalai has served as an assistant professor at Georgia Tech and the Toyota Technological Institute at Chicago. He is a member of the science team of Project CETI, an interdisciplinary initiative to understand the communication of sperm whales. He has co-chaired AI and crowdsourcing conferences including COLT (the Conference on Learning Theory), HCOMP (the Conference on Human Computation), and NEML. His honors include the Majulook prize, best paper awards, an NSF CAREER award, and an Alfred P. Sloan fellowship.
About the SRI Seminar Series
The SRI Seminar Series brings together the Schwartz Reisman community and beyond for a robust exchange of ideas that advance scholarship at the intersection of technology and society. Seminars are led by a leading or emerging scholar and feature extensive discussion.
Each week, a featured speaker will present for 45 minutes, followed by an open discussion. Registered attendees will be emailed a Zoom link before the event begins. The event will be recorded and posted online.
Adam Tauman Kalai