How Spotify AI plans to know what's going on inside your head, and find the right track for it

• Spotify has over 100 million tracks available for listeners, and the streaming audio company is increasingly investing in AI to personalize song recommendations.
• Its recently launched AI DJ aims to help listeners move out of their comfort zone and into new music.
• But knowing when users desire to be in discovery mode versus sticking with established musical preferences remains tricky, according to audio experts.

With about 100 million tracks available and over 600 million subscribers, helping listeners find the music they will love has become a navigational challenge for Spotify. It's the promise of personalization and meaningful recommendations that will give the vast catalog more meaning, and that is central to Spotify's mission.

The streaming audio giant's suite of recommendation tools has grown over the years: Spotify Home feed, Discover Weekly, Blend, Daylist, and Made for You Mixes. And in recent years, there have been signs that it is working. According to data released by Spotify at its 2022 Investor Day, artist discoveries every month on Spotify had reached 22 billion, up from 10 billion in 2018, "and we're nowhere near done," the company stated at that time.

Over the past decade or more, Spotify has been investing in AI and, in particular, in machine learning. Its recently launched AI DJ may be its biggest bet yet that technology will allow subscribers to better personalize listening sessions and discover new music. The AI DJ mimics the vibe of radio by announcing the names of songs and lead-in to tracks, something aimed in part to help ease listeners into extending out of their comfort zones. An existing pain point for AI algorithms — which can be excellent at giving listeners what it knows they already like — is anticipating when you want to break out of that comfort zone. 

The AI DJ combines personalization technology, generative AI, and a dynamic AI voice, and listeners can tap the DJ button when they want to hear something new, and something less-directly-derived from their established likes. Behind the dulcet tones of an AI DJ there are people, tech experts and music experts, who aim to improve the recommendation capacity of Spotify's tools. The company has hundreds of music editors and experts across the globe. A Spotify spokesperson said the generative AI tool allows the human experts to "scale their innate knowledge in ways never before possible."  

The data on a particular song or artist captures a few attributes: particular musical features, and which song or artist it has been typically paired with among the millions of listening sessions whose data the AI algorithm can access. Gathering information about the song is a fairly easy process, including release year, genre, and mood — from happy to danceable or melancholic. Various musical attributes, such as tempo, key, and instrumentation, are also identified. Combining this data associated with millions of listening sessions and other users' preferences helps to generate new recommendations, and makes the leap possible from aggregated data to individual listener assumptions.

In its simplest formulation, "Users who liked Y also liked Z. We know you like Y, so you might like Z," is how an AI finds matches. And Spotify says it's working. "Since launching DJ, we've found that when DJ listeners hear commentary alongside personal music recommendations, they're more willing to try something new (or listen to a song they may have otherwise skipped)," the spokesperson said. 

If successful, it's not just listeners that get relief from a pain point. A great discovery tool is as beneficial to the artists seeking to build connections with new fans.

Julie Knibbe, founder & CEO of Music Tomorrow — which aims to help artists connect with more listeners by understanding how algorithms work and how to better work with them — says everyone is trying to figure out how to balance familiarity and novelty in a meaningful way, and everyone is leaning on AI algorithms to help make this possible. Be she says the balance between discovering new music and staying with established patterns is a central unresolved issue for all involved, from Spotify to listeners and the artists.

"Any AI is only good at what you tell them to do," Knibbe said. "These recommender systems have been around for over a decade and they've become very good at predicting what you will like. What they can't do is know what's in your head, specifically when you want to venture out into a new musical terrain or category." 

Spotify's Daylist is an attempt to use generative AI to take into account established tastes, but also the varying contexts that can shape and reshape a listeners' tastes across the course of a day, and make new recommendations that fit various moods, activities and vibes. Knibbe says it's possible that improvements like these continue, and the AI gets better at finding the formula for how much novelty a listener wants, but she added, "the assumption that people want to discover new music all the time is not true."

Most people still return, fairly happily, to familiar musical terrain and listening patterns. 

"You have various profiles of listeners, curators, experts … people put different demands on the AI," Knibbe said. "Experts are more difficult to surprise, but they aren't the majority of listeners, who tend to be more casual," and whose Spotify usage, she says, often amounts to creating a "comfortable background" to daily life.

Technology optimists often speak in terms of an era of "abundance." With 100 million songs available, but many listeners preferring the same 100 songs a million times, it's easy to understand why a new balance is being sought. But Ben Ratliff, a music critic and author of "Every Song Ever: Twenty Ways to Listen in an Age of Musical Plenty," says algorithms are less solution to this problem than a further entrenching of it.

"Spotify is good at catching onto popular sensibilities and creating a soundtrack for them," Ratliff said. "Its Sadgirl Starter Pack playlist, for instance, has a great name and about a million and a half likes. Unfortunately, under the banner of a gift, the SSP simplifies the oceanic complexity of young-adult depression into a small collection of dependably 'yearny' music acts, and makes hard clichés of music and sensibility form more quickly." 

Works of curation that are clearly made by actual people with actual preferences remain Ratliff's preference. Even a good playlist, he says, might have been made without much intention and conscience, but just a developed sense of pattern recognition, "whether it's patterns of obscurity or patterns of the broadly known," he said.

Depending on the individual, AI may have equal chances of becoming either a utopian or dystopian solution within the 100-million track universe. Ratliff says most users should keep it more simple in their streaming music journeys. "As long as you realize that the app will never know you in the way you want to be known, and as long as you know what you're looking for, or have some good prompts at the ready, you can find lots of great music on Spotify."