The art and science of recommender systems: Insights from Spotify
Spotify has 500 million monthly active users, and its ability to curate playlists that magically fit each user’s taste is far from accidental. In a special event hosted by SRI Research Lead Ashton Anderson, Spotify’s Senior Director of Research Mounia Lalmas shared insights into how the platform crafts personalized listening experiences.
Have you ever clicked on a curated Spotify playlist and asked yourself, “How does it understand me so well?” There is a reason why the platform has 500 million monthly active users: their ability to curate content that magically fits each user’s taste is far from accidental, but the result of sophisticated systems built on complex machine learning.
With an impressive song library of over 82 million tracks, one might wonder how Spotify decides which songs to suggest. The answer lies in the platform’s recommender algorithms, which develop custom content relevant for a specific user based on their past behaviour. Recommender systems are so ubiquitous today that we have come to expect personalized content on nearly every digital platform that we interact with.
In a special event hosted by the Schwartz Reisman Institute for Technology and Society (SRI) at the University of Toronto’s Rotman School of Management, SRI Research Lead Ashton Anderson conversed with Mounia Lalmas, senior director of research at Spotify, who explored some of the behind-the-scenes techniques used by the world’s largest music streaming platform.
An assistant professor in U of T’s Department of Computer Science, Anderson’s research in the field of computational social science led him to Spotify in 2018 to explore how algorithms impact user journeys, where he collaborated with Lalmas, who leads a team of interdisciplinary researchers working on personalization. In addition to her work at Spotify, Lalmas holds an honorary professorship at University College London and is a distinguished research fellow at the University of Amsterdam.
SRI Research Lead Ashton Anderson and Spotify’s Senior Director of Research Mounia Lalmas have collaborated on a number of research projects that examine how recommender systems impact the listening habits of users.
Creating personalized listener experiences
At the outset of her presentation, Lalmas emphasized the importance of “creating personalized listening experiences.” She delved into how Spotify processes data to match users and tracks, providing insight into the types of data used, which includes not only the taste preferences of users, but also their interactions with the app, time of day, and history.
Rather than seeking to predict the next click, Lalmas emphasized the goal at Spotify is to develop algorithms that guide users’ long-term journeys, and that this necessarily shifts what factors determine a “good recommendation.” She rounded out her presentation by sharing recent methods her team has developed, including how the platform is supporting users to explore diverse content and helping to connect users with new artists.
At the heart of Spotify’s listening experience lies a sophisticated recommendation engine composed of three layers: data, models, and experience.
Beginning with data, Lalmas highlighted the role of user playlists and listening history, as well as instrumentation—the actions taken on the platform—in shaping recommendations. For instance, a user who spends a lot of time scrolling is likely seeking out new content rather than something they are already familiar with. Spotify uses metadata provided by the music label, as well as the audio profile of the song, to further factor into recommendations.
User and track data are then mapped onto an embedding space that enables the platform to assess relationships between content. In natural language processing, embeddings enable systems to grasp the relative proximity between words—for instance, the word “dog” would be closer to “cat” than to “tree.” Every track, artist, and user can be represented in this space, which serves as a backbone for the platform’s models, and proximity within these spaces determines similarity and cohesion.
The last layer of Spotify’s recommendation is experience, which consists of the app that users interact with. Recommendation engines power tailored playlists in this layer, such as Discover Weekly, and also determine how content is surfaced on a user’s homepage. This layer provides crucial feedback (for example, if someone skipped a suggested song), which helps the engine to continuously refine its recommendations.
Balancing wants and needs
Lalmas highlighted that recommendation systems should deliver a lifetime of content, with a goal to build trust. To this extent, the aim is to fine-tune algorithms to strike a balance between satisfying users’ wants and needs.
Imagine that you opened a newly recommended playlist, and you skipped the first five songs because they weren’t your style—feeling frustrated, you would close the playlist. On the other hand, if the app continually suggests only familiar songs, you’d soon become bored. The key to engaged listeners, Lalmas emphasized, is striking a delicate balance, which is the goal of all content recommendation systems.
Mounia Lalmas speaks at Rotman School of Management. (Photo by Dan Browne.)
Diversifying listening experiences
Lalmas capped off her talk by sharing some of the progress her team has made towards understanding how to diversify listening experiences, including a survey of recent research publications.
In a paper presented at the 2022 ACM Conference on Web Search and Data Mining, her team proposed and evaluated a machine learning algorithm that models a user’s recent sequential listening patterns (which change quickly) and their all-time listening profile (which changes slowly). The result is a more accurate user profile developed over time, improving recommendations.
In a collaboration with Anderson, Lalmas and team modelled the evolution of user preferences over time. Their findings showed that changes in musical preferences were directional, meaning that those who listened to hard rock may eventually develop an interest in blues rock, but those who started with blues rock would not shift to hard rock. By identifying these preferential pathways, it can become possible to introduce users to new music different enough to diversify their listening—but not different enough for engagement to drop.
Another collaborative project between Ashton and Lalmas analyzed the long-term consequences of Spotify recommendations on users’ listening habits. They labelled users as “generalists” or “specialists” based on the diversity of songs they listen to, and found that recommender systems can be fine-tuned to encourage diverse listening without affecting engagement, by steering consumption towards less popular content. This study highlights a common challenge with recommender systems: the tendency to promote the most popular items, which can lead to “rich-get-richer” scenarios.
The influence of algorithmic recommendations on consumption has motivated researchers to investigate their technical design and governance, and finding new ways to align recommender systems with social values has emerged as an important area of research. SRI Director and Chair Gillian Hadfield is currently engaged in a multi-stakeholder project exploring how to improve user satisfaction with Facebook’s recommender systems, which proposes that partnerships between academia and Big Tech are vital to shed light on the social impacts of algorithms. Recommender systems were also the focus of a session at SRI’s 2022 Absolutely Interdisciplinary conference, which discussed the importance of policy to shape the design of algorithms in the context of democratic rights.
As Lalmas highlighted, diversity is key when creating personalized experiences, and “a diverse and dynamic diet” generates long-term benefits for users and platforms alike. So, the next time you open Spotify, take a moment to appreciate the work that goes into creating an experience that expands your musical horizons, one song at a time.
Watch the talk:
About the author
Sharon Ferguson is a PhD student at Ready Lab in the University of Toronto’s Department of Mechanical and Industrial Engineering. Ferguson holds a BASc in Industrial Engineering from the University of Toronto. Her interests lie in computational social science and human-computer interaction, with her current work using enterprise social networking data and natural language processing methods to learn about engineers’ design processes and collaboration practices. She also investigates how discrimination occurs on these platforms and how we can build tools that nudge participants towards inclusive practices. Ferguson is an Ethics of AI Graduate Research Fellow (2021-2022) for her work studying student intentional persistence and diversity in machine learning and artificial intelligence. Working with Anastasia Kuzminykh and Rohan Alexander, she also investigates the ability of large language models to produce explanations, and how these differ from human explanations. She is passionate about encouraging underrepresented groups in STEM and is an active member of the Graduate Society of Women Engineers.
