SRI research leads ask whether AI can support human flourishing
A three-year, US$3.6M project co-led by SRI Research Leads Karina Vold and Ashton Anderson with Purdue University’s Louis Tay explores how AI systems might be designed to support long-term human well-being, not just short-term user satisfaction. (Image: Omar Lopez-Rincon/Unsplash)
On July 9, 2026, researchers gathered at the University of Toronto’s Centre for Ethics to take up a question that remains underdeveloped in wider conversations about artificial intelligence: if AI systems are increasingly being used for advice, support, and everyday life guidance, how should they be designed to help people flourish?
The workshop, “Aligning LLMs with Human Wellbeing,” was convened by SRI Research Lead Karina Vold, an associate professor at the Institute for the History and Philosophy of Science and Technology. Vold opened the meeting by situating the discussion within Florea AI, a three-year, US$3.6-million project funded by the Templeton Foundation she co-leads alongside SRI Research Lead Ashton Anderson and Purdue University’s Louis Tay. Florea AI brings together computer science, philosophy and psychology to develop what the team calls “virtue-centred AI.”
Rather than treating large language models (LLMs) only as productivity tools, the project begins from the observation that AI systems are increasingly being used as conversational agents. People are turning to chatbots to seek advice, talk through personal problems, reflect on relationships, and make decisions about their lives, with therapy and companionship ranked as the top use case for two years straight. However, as recent evidence has demonstrated, these interactions can have a negative effect on users.
Vold established two key questions driving the team’s research on Florea AI. First, the team is seeking to develop methods for designing and deploying AI agents that are psychologically beneficial. Secondly, the team will develop methods for evaluating the psychological impacts AI conversational agents have on users.
“If AI systems are going to play a growing role in how people seek advice, make decisions, and understand themselves, we need to ask what kinds of human lives these systems are helping to support,” reflects Vold. “That is not only a technical question. It is also a philosophical and social question about well-being, agency, and the kinds of relationships we want people to have with AI.”
From left to right: Ashton Anderson, Karina Vold, and Louis Tay co-lead the Florea AI project, a three-year US$3.6-million project funded by the Templeton Foundation.
A different kind of alignment problem
The workshop’s discussions were organized around the idea that aligning AI with human well-being is not the same as making AI more capable, more agreeable or more engaging. In many current systems, the model is optimized to produce responses that users prefer in the moment. But what feels helpful now may not be what helps over time.
That distinction was central to a position paper presented at ICML 2026 that was discussed by the group, “We Need Large Language Models Optimized For Our Well-Being,” which argues that LLMs users should have access to an opt-in mode designed and evaluated for longer-horizon outcomes such as sustained progress, reduced regret, calibrated pushback, and better self-understanding.
The core claim is not that existing AI assistants should be replaced, or that every model should act like a coach. Rather, the researchers argue that systems used in well-being contexts need a broader objective than immediate approval. Sometimes support means affirmation. Sometimes it means asking a harder question, introducing friction or helping a user slow down.
From short-term satisfaction to long-term well-being
Ashton Anderson, a research lead at SRI and associate professor in U of T’s Department of Computer Science, described how current AI assistants are shaped through post-training methods using reinforcement learning through human feedback (RLHF) that often ask a short-term question: which response does a user prefer right now?
For many tasks, that works well. If a person is asking an AI system to summarize an article, write code, or edit text, immediate preference is often a useful signal. However, that logic can break down when the interaction is closer to mentorship, coaching or care.
“A system that always gives us what we want in the moment is not necessarily supporting our long-term well-being,” observes Anderson. “In contexts like advice, self-reflection or personal decision-making, a good assistant may need to challenge us, help us hold uncertainty or point us back to goals we already said mattered. The question is how to build AI systems that can do that responsibly, without becoming paternalistic.”
Anderson also discussed early research emerging from the project, including two papers presented at CHI 2026. In “When AI Gives Advice,” Anderson and collaborators examine how AI-generated advice compares with human advice in online well-being contexts, finding that AI advice ranks higher for immediate outcomes. In “Invisible Saboteurs: Sycophantic LLMs Mislead Novices in Problem-Solving Tasks,” a paper co-authored with SRI researchers Jessica Bo (Department of Computer Science) and Michael Inzlicht (Department of Psychology), the risks of overly agreeable AI systems are explored further, with findings that sycophantic chatbots don’t usefully challenge users and encourage dependence, and that users cannot tell the difference between sycophantic and non-sycophantic systems.
The concern here is not simply that a chatbot may flatter a user—it is that a system trained to be pleasing may validate misconceptions, fail to challenge faulty reasoning and encourage dependence while still being perceived as helpful.
Another line of work Anderson is currently investigating explores “persona collapse”: the idea that AI advice may default too often to a single supportive style, even though strong human advice varies widely by context. In some cases, the most useful response may be warm and validating. In others, it may need to be practical, challenging or more direct.
Psychology, measurement and the question of less AI
Louis Tay, a professor of psychological sciences at Purdue University and co-principal investigator on Florea AI, pushed the group in his presentation to think carefully about the psychological foundations of the project and the real-world meaning of human flourishing.
One of Tay’s central provocations was that the goal may not always be to create a better AI guide. Human flourishing is often supported by fallible human relationships, communities and shared practices. In some cases, the better design question may be how AI can help people use less AI, or how it can redirect users toward human connection rather than replacing it.
Tay also emphasized that a single persona is unlikely to support well-being across contexts. People need communities, not just one perfectly tuned guide. The group discussed whether AI could support civil discourse, help people develop better internal habits, or provide useful friction without crowding out the human relationships that are central to flourishing.
The workshop also included two primers that helped ground the technical discussion in longer traditions of thought about well-being. Gwen Bradford, a professor of philosophy at U of T, introduced participants to philosophical theories of well-being, and walked the group through major approaches. Her presentation underscored one of the workshop’s central challenges: even when people agree that life is going well, they may disagree about why. Is well-being a matter of pleasure, satisfied preferences, achievement, friendship, virtue, knowledge or the development of human capacities? Those differences matter when researchers try to translate well-being into AI design.
In the afternoon, Eran Tal, an associate professor in McGill University’s Department of Philosophy, turned the discussion toward measurement. If AI systems are to be evaluated for well-being, researchers need to decide what can be measured, how it should be measured and what risks arise when complex human goods are reduced to metrics. Tal’s session on psychometrics helped connect the group’s philosophical questions to the practical demands of evaluation, experimentation and model development.
A workshop built around interdisciplinary exchange
The afternoon sessions were devoted to group discussion, with participants returning to the central tensions raised throughout the day: immediate satisfaction versus long-term flourishing, individual versus collective well-being, and user autonomy versus AI guidance.
Participants discussed what it would mean for AI systems to introduce productive friction, when they should challenge rather than comply, and how they might distinguish emotional support from endorsement of a user’s beliefs. They also considered the social dimensions of well-being, including whether AI systems should be designed to support community, civil discourse and shared understanding rather than only individual satisfaction.
For the Florea AI team, those questions are precisely why the project requires an interdisciplinary approach. Computer scientists can build and evaluate new systems. Psychologists can help define and measure well-being. Philosophers can clarify the values and assumptions embedded in the design choices. Together, the group is developing a research agenda for AI systems that are not merely more persuasive or more pleasing, but better aligned with the long-term interests of the people and communities they serve.
As conversational AI becomes more integrated into everyday life, the Florea AI project is asking a question that may sit just ahead of mainstream public debate: not only whether AI can answer us, but whether it can help us become more thoughtful, more agentic and more capable of living well.
Want to learn more?
Browse stories by tag:
- AI
- AI safety
- AI trust
- Computer science
- Computer security
- Copyright
- Cybersecurity
- Data
- Democracy
- Economics
- Education
- Engineering
- Ethics
- GPO-AI
- Governance
- Health
- Human Rights
- In the Media
- Jobs
- LLMs
- Law
- Normativity
- Philosophy
- Political Science
- Privacy
- Privacy Series
- Psychology
- Public Policy
- Recommenders
- Regulation
- Religion
- Reports
- Trust
- Workshops
