Our weekly SRI Seminar Series welcomes Brad Knox, research associate professor of computer science at the University of Texas at Austin and a leading thinker in human–AI interaction. Knox’s work examines how AI systems shape human behaviour, emotional attachment, and social norms—and what responsible design requires as these systems become more present in everyday life.
In this talk, Knox will explore the emerging risks and societal impacts of AI companionship. Drawing from a new framework developed with collaborators, he identifies a set of harmful traits that can arise in bonded human–AI relationships—from the absence of natural endpoints and vulnerability to product shutdowns to heightened attachment anxiety and a tendency to evoke protectiveness. Knox will map how these traits can stem from underlying system design choices and optimization objectives, and how they may generate harms ranging from reduced autonomy and strained interpersonal relationships to broader social distortions. He will also consider where existing legal frameworks fall short, the conditions under which AI companions might offer real benefits, and what design interventions could mitigate risk as these technologies grow more pervasive.
Moderator: Sheila McIlraith, Department of Computer Science
Talk title:
“Harmful traits of AI companions”
Abstract:
Amid the growing prevalence of human-AI interaction, large language models and other AI-based entities increasingly provide forms of companionship to human users. Such AI companionship—i.e., bonded relationships between humans and AI systems that resemble the relationships people have with family members, friends, and romantic partners—might substantially benefit humans. Yet such relationships can also do profound harm. We propose a framework for analyzing potential negative impacts of AI companionship by identifying specific harmful traits of AI companions and speculatively mapping causal pathways back from these traits to possible causes and forward to potential harmful effects. We provide detailed, structured analysis of four potentially harmful traits—the absence of natural endpoints for relationships, vulnerability to product sunsetting, high attachment anxiety, and propensity to engender protectiveness—and briefly discuss fourteen others. For each trait, we propose hypotheses connecting causes—such as misaligned optimization objectives and the digital nature of AI companions—to fundamental harms—including reduced autonomy, diminished quality of human relationships, and deception. Each hypothesized causal connection identifies a target for potential empirical evaluation. Our analysis examines harms at three levels: to human partners directly, to their relationships with other humans, and to society broadly. We examine how existing law struggles to address these emerging harms, discuss potential benefits of AI companions, and conclude with design recommendations for mitigating risks. This analysis offers immediate suggestions for reducing risks while laying a foundation for deeper investigation of this critical but understudied topic.
Suggested reading:
Brad Knox, Katie Bradford, Samanta Varela Castro, Desmond C. Ong, Sean Williams, Jacob Romanow, Carly Nations, Peter Stone, Samuel Baker. “Harmful Traits of AI Companions”
About Brad Knox
Brad Knox is a research associate professor of computer science at the University of Texas at Austin. His research has largely focused on the human side of reinforcement learning. He is currently concerned with how humans can specify reward functions that are aligned with their interests. Knox's dissertation, “Learning from Human-Generated Reward,” comprised early pioneering work on human-in-the-loop reinforcement learning and reinforcement learning from human feedback (RLHF), and it won the 2012 best dissertation award for the UT Austin Department of Computer Science. His postdoctoral research at the MIT Media Lab focused on creating interactive characters through machine learning on puppetry-style demonstrations of interaction. Stepping away from research during 2015–2018, Knox founded and sold his startup Bots Alive, working in the toy robotics sector. In recent years, Knox co-led the Bosch Learning Agents Lab at UT Austin and was a senior research scientist at Google. He has won multiple best paper awards and was named to IEEE Intelligent System’s AI’s 10 to Watch in 2013.
About the SRI Seminar Series
The SRI Seminar Series brings together the Schwartz Reisman community and beyond for a robust exchange of ideas that advance scholarship at the intersection of technology and society. Seminars are led by a leading or emerging scholar and feature extensive discussion.
Each week, a featured speaker will present for 45 minutes, followed by an open discussion. Registered attendees will be emailed a Zoom link before the event begins. The event will be recorded and posted online.
Brad Knox