Successful Failures: YouTube Music gained its AI smarts from an unlikely source

Before we get started, let me take a moment to state the obvious: Youtube Music is clearly descended from the deceased Google Play Music, and the evolution isn’t remotely ambiguous.  For the most part this article series is about the unexpected origins of successful services, but there’s nothing unexpected about Youtube Music’s origin.  Or is there?

At first blush, Youtube Music would seem to be a standard music streaming service swimming in an ocean of better-equipped, more-experienced competitors.  Apple Music has an incredibly deep catalog and was launched by music industry veterans Jimmy Iovine and Dr. Dre (bona fides don’t get any better than this).  Spotify has been in the game the longest, has the largest market share, and has all the great features users want, including a free service tier.  Until recently, Tidal was owned by Jay-Z, has integrated live concerts and performances, offers CD-quality lossless audio, and has positioned itself as your favorite artist’s favorite music streaming service. 

And there’s still Pandora, Deezer, Amazon Music, Qobuz, and iHeartRadio to name a few more!  Youtube Music isn’t lossless, doesn’t have the largest catalog, nor is it particularly favored by recording artists.  In fact, Youtube Music seems downright ordinary when compared to its peers.  It almost feels like Google is choosing not to compete on the same terms.  Instead of juicing bitrates, or signing early/limited release features from artists, Google has made the Youtube Music experience accessible to the entry-level user with all of the basic features you would expect, and then it turns on the magic.  It’s the sort of magic I’ve talked about before that only Google can deliver, spinning all of that data they have about you into a seamless, incredibly personal music experience.

A Smarter Streaming Service

I’m not here to review Youtube Music.  I have dabbled with Tidal, tried Spotify’s free tier, and I don’t even know what Apple Music looks like.  All I know is Youtube Music, so any review from me would be super biased.  But, I promise that if you’re a regular person who loves music, Youtube Music is literally built for you.  I use the phrase “regular person” very deliberately.  I see a lot of tech reviewers talk about these services and throw out terms like “lossless audio”, “bit perfect”, and “spatial audio”.  They are obsessed with playlists they’ve built and curated over years, and particular versions of obscure albums that almost none of the services seem to have in their catalogs. 

If I’m being perfectly honest, I have one foot in this world.  Half the time I stream music from my own collection of CD-audio quality files because, as one does from time to time, I find myself sitting in my study with headphones on, sipping on an adult beverage, appreciating all the subtleties and complexities of the music (whatever that means, you music snob).  However, there’s another side of me that just wants to hear good music.  I want to hear it without futzing around trying to find a playlist or flipping through menus.  And I want to hear it right now!  In my experience, most “regular people” (i.e. not music snobs) want this, and thankfully this is where Youtube Music shines.

It really dawned on me a few weeks ago before I started working on this article.  I was driving to pick up my daughter from school, and instead of reaching for a well-worn, predictable playlist, I tapped “My Supermix” – the first button you see when you load Youtube Music on any device.  This wasn’t the first time I had tried the feature, but it was the first time I was blown away by it.  Over the next hour, my eardrums were treated to a stream of my favorite artists and songs, interspersed with new ones that I’d never heard before, but nevertheless enjoyed.  I didn’t hit the skip button once.  It was neither the comfortable-yet-repetitive, predictable experience of a Pandora station, nor the hit-or-miss personalized artist radio feature of Spotify. 

Fast-forward to today, and all of a sudden, “My Supermix” has become my go-to music source.  It seems to know exactly what I’m in the mood to hear, wherever I am, and whatever I’m doing.  If you’re a regular person who loves music, this is all that matters.  No one cares about whether a streaming service has 2 million songs or 10 million because predictive playback trumps everything else.  Google has always known this; the power of predictive algorithms is the driving force behind everything they do.  But if you would have asked me a decade ago if it was even possible for AI to predict musical taste, I would have laughed out loud.  Music is emotional, spontaneous, unpredictable: it’s uniquely human.  And I can imagine the Google Assistant laughing right back at me and responding that once you have enough data, it all comes down to math.  Back in the days of Google Play Music, that’s exactly what happened.  Google boiled human intuition down to math.

It’s All About the Music

My quest for an awesome online music experience began in 2004 when streaming radio was all the rage.  The Shoutcast directory was the source, Winamp was the player (yikes I’m old), and everything was free (although it probably shouldn’t have been).  In those days, if you had broadband, a music collection and some Linux smarts, you could be broadcasting your own radio station!   There were some really incredible stations out there that were super-niche, too.  My favorite was the now-defunct smoothbeats.com.  For an underground hip-hop fan like myself, it felt like home from the moment I hit PLAY.  Twenty-four hours a day, I could tune in and reliably hear artists like MF DOOM (RIP, and please follow this link down the internet rabbit hole and learn more about the Super Villain), Aesop Rock, Mos Def, El P, Atmosphere, or supergroups who normally trafficked in mixtapes like The Justus League and Hieroglyphics. 

Even with this narrow focus, there were still stale tracks that you wanted to skip but couldn’t due to the live nature of the station.  So, my quest continued to the world of personalized music content.  Back in 2005, Last.fm and Pandora were some of the first popular services that let you select artists you liked, and listen to similar content.  But Last.fm could do something Pandora couldn’t.  It could hook into other services like internet radio or your personal mp3 collection, analyze what it was hearing, and mix more of this content into your personal station.  I alternated between Last.fm and Pandora for years until the repetition and lack of fresh content made me restless again.

In 2011, Google Play Music launched and like all things Googly in the 2010’s, I was absolutely on-board.  After using it for a few months, the shine wore off and I found myself back on Pandora more often.  Google’s new music service was nice, and you did get more content from the actual artist on their Artist Radio, but it didn’t really scratch the itch for me.  I was disappointed, quite honestly.  I remember telling a buddy about this during a road trip while we were fussing over what to listen to.  He told me that I was using the wrong music service.  If I wanted the best personalized radio, it needed to be something that actual humans created, and the best app for doing that was Songza.

Sometimes AI needs a human touch

While everything else (Pandora, Slacker, et al) was algorithm-driven, Songza took a different approach, merging artist selections with human-curated playlists.  You logged in, tagged the artists that you liked, and then you had access to a dizzying array of mood or activity-based playlists that Songza called its Concierge service.  And it was awesome.  You could click “Just waking up”, “Roadtrip”, or “Relaxing with a bottle of wine” and the playlist just nailed it!  For close to a year, I was using Songza daily, and I sort of accepted that AI was probably not the future of music, and that was fine. 

When Google purchased Songza in 2014, it was a bit of a head-scratcher for me.  Why would Google, of all companies, employ humans to create Songza-style playlists for Google Play Music?  It didn’t dawn on me until years later that the human-curated playlists were not just for us; they were for Google’s AI.  Every time I clicked thumbs up or down in Songza, or didn’t (or continued listening for hours, or didn’t), or selected a mood or an activity, it was all saved as data.  Google wasn’t so much purchasing Songza as it was purchasing Songza’s data about listener choices and reactions.  And as it integrated the concierge service into Google Play Music, Google was continuing to train its AI in real-time.  The immediate result was that Google Play Music gained superpowers!  Mood and activity stations were better than ever, and I stopped using Songza well before the app was sunsetted.  However, it’s the AI-training that would really have a long-term impact on Google’s music experience.

Human-curated playlists never really got jettisoned, as they remained in Google Play Music for the life of the service, and live on inside Youtube Music behind familiar mood or activity choices. But right alongside them, the newly trained music AI went online.  In November of 2016, Google Play Music got a new look and new smarts.  It began to truly contextualize and predict what you wanted to hear based on things like weather, location, and time of day, powered by gobs of training data it had inherited from Songza. 

Later, as Youtube Music went live and began to evolve, that same AI would re-emerge, this time perfected with millions of data points from Songza and Google Play Music, as well as standard video Youtube.  Now, the feature is called “My Mix”, with several daily iterations to choose from including a “Supermix”.  This isn’t just a standard mix of music Google thinks you will like.  The mixes change daily, are context aware, combine old artists with others that are new to you, but most importantly it absolutely nails predicting what you want to hear with uncanny accuracy! 

Clearly there’s a lot more that’s special going on underneath the hood of YouTube Music.  I love the access to user-created remixes and DJ content that just doesn’t exist anywhere else other than Youtube.  I also think it’s great that I can stash and listen to a few thousand of my own mp3’s there.  You can cast your tunes to smart speakers or a TV.  It’s supported on Sonos.  These are all great features.  But regular people (remember them?) who want to listen to music, just have to click one button and they get to hear the most advanced and accurate personal radio station they’ve ever encountered.  And although it’s now driven by Google’s powerful AI, it’s worth remembering that it was data from human-curated playlists that made it all possible.