The Rise of AI Music

Voice-cloning wars. Copyright disputes. A potential flood of nonhuman music on streaming. AI is already a musical battleground. Let’s start with a brief history of AI Music to get us up to speed.

In 2015, Sony’s Paris-based Computer Science Laboratory, developed an AI-composed Beatles pastiche “Daddy’s Car,” which made worldwide headlines as a technological milestone. Well, it’s a start.

Next they fed sheet music for different jazz standards into AI software which produced (among others) an odd, haunting song called “Ballad of the Shadow”. Its creator decided the AI and him had fused together into a new artist — whom he named Skygge. I’m sure we could have come up with a better one.

In 2022, a group of researchers began work on an open-source tool that allowed users to generate raps in famous voices by typing words into an interface (SVC). I’d have called it C-RAPS.

AI’s breakthrough hit arrived on April 4, 2023. “Heart on My Sleeve,” a song by the anonymous songwriter Ghostwriter-977, featured synthetic Drake and the Weeknd vocals. However, it turns out it was written, produced, and sung entirely by a human, with the only AI intervention arriving via the transformation of the vocals into Drake’s and the Weeknd’s voices. Shame!

The song garnered more than half a million streams before Universal Music managed to take it down, highlighting the still-unsettled territory of whether a singer’s voice is copyrighted. That’s gonna be a shit storm.

Paul McCartney announced in June this year that a final, “Free As A Bird”-style Beatles song was coming, using neural-net-powered John Lennon’s voice taken from a demo. Well that’s certainly progress and that’s pretty much where we are at now.

So what are the issues and what have we learned, if anything?

Clearly AI is not yet able to produce full, convincing pop songs with vocals and lyrics. This is largely because of the sheer number of subtle intricacies in a piece of recorded music, from the underlying composition to vocal inflections down to, say, the reverb tail on the snare drum. The results lack real soul.

However, there can be little doubt that there is a world of possibilities (and concerns) in voice-cloning technology alone. Take for example, resurrection dead artists voices. Kurt Cobain was recently made to sing songs by Seether. Even more sacrilegious was Jeff Buckley singing Lana Del Rey. Whatever the path it needs to be amenable to, and profitable for, artists.

But what about giving Elvis fans the ability to sing Elvis songs karaoke-style — with their favourite singers own voice? Duets with The King. An artist called Grimes has allowed unlimited use of digital clones of her voice and simply asks to split any revenue 50-50 with her. Pioneering stuff.

Universal Music Group has described AI music as “fraud” and said that it will “harm artists.” They envision a system of monitoring and monetization (or course) AI music, ideally with the ability for artists to opt out.

There’s also talk of compulsory licensing for voices, the same way anyone can cover a song without getting the writer’s permission.

There’s another world where voice-cloned songs could stay mostly underground, as a fun, non-monetizable gimmick. That could benefit from the hands-off approach labels had toward rap mixtapes. Like mash-ups, it’ll probably have its heyday and then get old. Real quick.

Already, online amateurs are replacing Paul McCartney’s old vocals on his latter-day songs with clones of his Beatles-era voice, with promising results. Tech-savvy fans have given the same rejuvenating treatment to recordings of Axl Rose’s recent live performances with Guns N’ Roses. Ahem.

Right now, there’s a slight delay in even the best processing scenario, but once that goes away, artists could start using AI clones of their own voices live onstage, as a sort of supercharged AutoTune. AI is moving quickly, and the most consequential use cases for voice-cloning may have not yet emerged.

In the studio, they could use Melodyne to tune their voice up to notes they can’t quite hit, and then bring in the vocal clone to smooth it out. Sounds like cheating. But what about burying layers of Brian Wilson’s and Marvin Gaye’s voices deep in the backing vocals? Or even making the most beautiful synthetic voice from a combination of several others? That sounds considerably more interesting.

Universal Music CEO Lucian Grainge recently noted that “consumers are increasingly being guided by algorithms to lower-quality functional content that in some cases can barely pass for ‘music.’ Hard to disagree.

Boomy, which lets users quickly and easily make simple songs on their phones with AI tools and upload them to streaming services, boasts on its website that its users have recorded “15,477,480 songs, around 14.7 percent of the world’s recorded music.” One can only imagine what a sea of shit that is.

In May, Spotify pulled down some Boomy songs and temporarily barred the company from posting new songs, citing evidence of suspicious, artificial “streaming farm”-type activity. Thank you Spotify.

DJ/Producer David Guetta sees AI as another tool for us to make better records, make better demos, and that it will continue the decades-long, computer-driven democratization of music-making. Here’s hoping. Ultimately taste is the gate keeper. “If you have terrible taste, your music is still gonna be terrible, even with AI,” he adds. Word

So what is Big tech doing about all this? Google announced the existence of MusicLM, which attempts to do for music what ChatGPT does for text, back in January. In May, Google released MusicLM in a beta version, open to test users only.

It’s fascinating. Ask it to approximate 1960’s British Invasion rock and it will give you something fractured and bizarre, as if sourced from old vinyl in your dad’s garage that’s warped and scratched. But it’s pretty decent at generating generic trap beats, and nearly spectacular at conjuring funky blasts of retro-futuristic dance music. So struggles with good stuff but spits out crap easily. Jesus.

Consider also what an artist MIGHT have gone on to write had they not died. If AI is all about data input and as long as you loaded in the complete catalogue of tunes and demos the AI would learn the artist’s structures? Maybe, but if it hasn’t experienced the feelings of love, how can it create a genuinely real love song?

Somewhere deep in some distant server farm, something incalculably smart and eerily powerful is readying itself to sing. Maybe, but until it can create a riff like Whole Lotta Love, I aint listening.