Our first installment of the weekly SRI Seminar Series for the 2021-22 academic year welcomes Roger Grosse, assistant professor at the University of Toronto’s Department of Computer Science, faculty affiliate at the Schwartz Reisman Institute, and founding member of the Vector Institute.
Talk title:
“How can deep learning research inform long-term AI safety?”
Abstract:
Once AI systems become far more capable than humans across a sufficiently broad range of skills, we may no longer be in control if they are pursuing objectives contrary to our own. We can't simply deal with the problem as it happens, as we do with most new technologies; in order to ensure a positive outcome for humanity, we need to prepare in advance. It may appear impossible to prepare for a scenario that's probably decades or more into the future. In this talk, I'll outline ways in which current-day AI research can contribute to the long-term safety of AI systems. My talk will summarize our current state of knowledge and will primarily focus on the work of others, but will include a few tidbits from my own research.
In the first part, I'll consider the question of forecasting timelines to human-level and superhuman AI. Analogously to climate models, our beliefs about timelines have important implications for how we should prepare. I'll overview empirical findings about the scaling of AI capabilities with resources (e.g., hardware and data) and how this information can fit into models of AI timelines.
In the second part, I'll introduce the (still hypothetical) problem of mesa-optimization, whereby an AI chooses to solve a problem by implementing an algorithm (the mesa-optimizer) to optimize some other objective. This is analogous to how evolution produced generally intelligent humans who can now pursue goals and desires at odds with genetic fitness. Even if the system as a whole has well-understood safety properties, the mesa-optimizer might not. I'll outline some ways deep learning research can inform our understanding of when mesa-optimization is likely to occur.
Suggested readings:
N. Bostrom, 2012. "The superintelligent will: Motivation and instrumental rationality in advanced artificial agents." (PDF)
A. Cotra, 2020. "Forecasting TAI with biological anchors." (Note: The full report is over 100 pages, but see an excellent summary by Rohin Shah in the comments section.)
E. Hubinger et al., 2019. "Risks from Learned Optimization in Advanced Machine Learning Systems."
About Roger Grosse
Roger Grosse is an assistant professor of computer science at the University of Toronto, a founding member of the Vector Institute, and a Canada CIFAR AI Chair. His research group focuses on machine learning, especially deep learning and Bayesian modeling, with the aim to develop architectures and algorithms that train faster, generalize better, give calibrated uncertainty, and uncover the structure underlying a problem. He is especially interested in scalable and flexible uncertainty models, the automation and configuration of ML systems, and ensuring that AI systems align with human values. Grosse received a BS in symbolic systems from Stanford in 2008, an MS in computer science from Stanford in 2009, and a PhD in computer science from MIT in 2014, studying under Bill Freeman and Josh Tenenbaum. From 2014 to 2016, Grosse was a postdoctoral fellow at the University of Toronto, working with Ruslan Salakhutdinov. Along with Colorado Reed, he created Metacademy, a web site which uses a dependency graph of concepts to create personalized learning plans for machine learning and related fields. Recently, Grosse was a faculty advisor for undergraduates in a global AI competition to address climate change, and also mentored a group of U of T students decoding ciphers with AI.
About the SRI Seminar Series
The SRI Seminar Series brings together the Schwartz Reisman community and beyond for a robust exchange of ideas that advance scholarship at the intersection of technology and society. Seminars are led by a leading or emerging scholar and feature extensive discussion.
Each week, a featured speaker will present for 45 minutes, followed by 45 minutes of discussion. Registered attendees will be emailed a Zoom link approximately one hour before the event begins. The event will be recorded and posted online.
Roger Grosse