Using Human-Centric Data Science at Spotify

A data scientist by training, Lillian brings a passion for human-centric machine learning and algorithmic effects to her role leading Spotify’s Personalized Home team. She also talks about the experience of leading her team from home during the pandemic.

Lillian’s team of data scientists and user researchers collaborate with a mixed-methods research approach to try to discern user needs and create the best matches between listeners and creators. They observe and analyze user behaviors to develop product insights. Her team’s user researchers develop qualitative insights about user needs and then work with the data scientists to figure out the scale of this need or behavior. “What are users telling us they actually need in that moment by doing certain actions, by looking and exploring at certain recommended items?”

She sees that Spotify can have a big impact on inspiring new experiences between listeners and the creators of music and podcasts. By sharing music and podcast stories with the world, Spotify often making a match between a listener and a creator for the first time. It could be the first time someone tried a podcast. “These aren’t just pixels and files that you’re moving around, you’re really creating creative experiences for people, and you’re sharing somebody’s creative output with people. There’s a cultural consequence to the things that we do.”

Spotify’s recommendation systems work partly by design and part of it is random. She says it’s the most exciting area of research for her right now because it’s stretching the limits of what a recommendation system can do. She explains that the kinds of recommendations you get from friends are contextualized and teaching a computer to do that is very difficult. It’s even harder to measure if you’re doing a good job, because you have to understand how to interpret the reactions that people are giving.

Lillian also cares deeply about being inclusive and offering opportunities for all types of people to be included in the tech conversation. As a female mathematician and proud Puerto Rican, she has seen how the current tech culture can be difficult for some to adapt to. She believes that instead of trying to adapt, it’s more important for each person to be authentic and that the culture itself needs to adapt to welcome all voices and their technical talents.

While it’s been challenging leading her new team completely over Zoom, she says she is being very intentional about what kind of leader she is and what kind of visible working mom she is. She thinks it’s harder for more junior colleagues because they don’t have the opportunities for workplace mentorship. Overall, she thinks this shift to remote work will be empowering and an opportunity to give people more flexibility about where they can live and work.