Welcome to the future: AI-generated vocal clones of superstars now available on YouTube Shorts – thanks to Google’s ‘most advanced music generation model to date’
We all knew it was coming – now, suddenly, it’s here.
On Thursday (November 16), Google unveiled what it calls its “most advanced music generation model to date”.
Dubbed Lyria, and built by Google DeepMind, the tech giant says that the AI model, “excels at generating high-quality music with instrumentals and vocals, performing transformation and continuation tasks, and giving users more nuanced control of the output’s style and performance”.
Google also released details of two new AI ‘experiments’ in partnership with YouTube, the first of which is dubbed Dream Track, an experiment in YouTube Shorts.
The other experiment is centered around a set of Music AI tools that Google says it is designing with artists, songwriters, and producers “to help bolster their creative processes”.
To develop these projects, Google says that it has “brought together technical experts from across Google with a diverse range of world-renowned artists and songwriters to explore how generative music technologies can responsibly shape the future of music creation”.
Within the Dream Track experiment, a limited set of YouTube Shorts creators in the US will be able to use the tool to produce “a unique soundtrack” featuring the AI-generated voice and musical style of artists including Alec Benjamin, Charlie Puth, Charli XCX, Demi Lovato, John Legend, Sia, T-Pain, Troye Sivan and Papoose.
According to a blog post penned by YouTube’s Global Head of Music Lyor Cohen and Toni Reid, VP of Emerging Experiences and Community Products, these artists “have chosen to collaborate in this experiment and work with us to shape the future of AI in music”.
Interestingly, none of the artists in this first project appear to be actively signed to Sony Music – they’re either signed to (labels of) Warner Music or Universal, or outside of the ‘major’ music companies.
Dream Track works by typing in a topic for a song and then choosing from a participating artist featured in the carousel to generate a 30-second soundtrack for a Short.
Powered by Google’s Lyria AI model, Dream Track will then simultaneously generate the lyrics, backing track, and AI-generated voice in the style of the superstar artist selected by the user.
You can see how this works in the video below:
This news follows the announcement earlier this year of YouTube’s Music AI incubator – a program by which new tools and innovations are being developed at the company in close conjunction with artists and the music business.
UMG Chairman and CEO Sir Lucian Grainge issued the following statement in response to today’s news: “We have a fundamental responsibility to our artists to ensure the digital ecosystem evolves to protect them and their work against unauthorized exploitation, including by generative AI platforms.
“At the same time, we must help artists achieve their greatest creative and commercial potential – in part by providing them access to the kind of opportunities and cutting-edge creative tools made possible by AI.”
Credit: Sthanlee B. Mirador/Sipa USA/Alamy
“In this dynamic and rapidly evolving market, artists gain most when together we engage with our technology partners to work towards an environment in which responsible AI can take root and grow.”
Sir Lucian Grainge, Universal Music Group
Grainge added: “In this dynamic and rapidly evolving market, artists gain most when together we engage with our technology partners to work towards an environment in which responsible AI can take root and grow.
“This is not a time for passivity. Only with active, constructive and deep engagement, can we build a mutually successful future together.”
Credit: Warner/press
“YouTube is taking a collaborative approach with this Beta. These artists are being offered the choice to lean in, and we’re pleased to experiment and find out what the creators come up with.”
Also commenting on the news, Warner Music Group’s CEO Robert Kyncl, said: “With each major leap in technology, the music industry navigates a fresh set of challenges and opportunities.
“It’s not always the case that, from the outset, tech platforms partner with artists, songwriters, labels, and publishers to experiment, iterate, and find possible solutions.
“YouTube is taking a collaborative approach with this Beta. These artists are being offered the choice to lean in, and we’re pleased to experiment and find out what the creators come up with.”
In addition to the Dream Track experiment, Google and YouTube say that their researchers have been working with artists, songwriters, and producers in YouTube’s Music AI Incubator to design a suite of music AI tools.
In Lyor Cohen and Toni Reid’s blog post, they write that “it was clear early on that this initial group of participants were intensely curious about AI tools that could push the limits of what they thought possible”.
According to Google, the music AI tools it’s testing out will allow users to “create new music or instrumental sections from scratch, transform audio from one music style or instrument to another, and create instrumental and vocal accompaniments.”
In practice, this means that you could turn a recording of your singing into an orchestral score, or hum a melody and turn it into a guitar or trumpet track.
In an example provided by Google and YouTube, producer and songwriter Louis Bell builds a track by humming a melody:
“We’re at a crucial crossroads in human history, and participating in determining our direction is humbling,” said Bell.
“Ultimately, technology can’t substitute for personal taste; therefore, a chef with access to infinite ingredients must still rely on his own intuition in the end.”
The final element of YouTube and Google’s announcement involves Watermarking AI-generated audio with a new tool called SynthID.
Any content created and published with Google’s Lyria model will be watermarked with SynthID, which the company explains, “embeds a watermark into AI-generated audio content that’s inaudible to the human ear and doesn’t compromise the listening experience”.
Google’s latest AI announcements in partnership with YouTube follow the news from Tuesday (November 14) that YouTube is developing a system for rightsholders to request the removal of content on the platform that “mimics an artist’s unique singing or rapping voice.”
This new takedown request system will initially be available to labels or distributors who represent artists participating in YouTube’s AI music experiments. YouTube said that it plans to expand access to other labels and music distributors in the coming months.
Charlie Puth said: “I’m extremely excited and inspired by the realm of musical possibilities that come from allowing the human-mind to collaborate with the non-human mind. YouTube has been a great partner in outlining their approach to AI and understands the need to work together with artists like me to develop this technology responsibly, ensuring it will accelerate creativity instead of replacing it.”
Sia said: “I’m excited to be a part of YouTube’s cutting edge Dream Track experiment and I can’t wait to hear what kinds of recipes all you creators out there come up with!”
Papoose said: “I think this experiment is an amazing ground breaking opportunity for all artists and creators. I am more than honored to be included. I strongly believe A I has a significant place in the future of music. Embracing the future is something we all shouldn’t be afraid of. Its beyond human capabilities will inspire future artists tremendously. While simultaneously being useful to artist of the past.. This is In deed the best example of innovation I’ve seen in a long time.”
John Legend said: “Being a part of YouTube’s Dream Track experiment is an opportunity to help shape possibilities for the future. As an artist, I am happy to have a seat at the table and I look forward to seeing what the creators dream up during this period.”
Demi Lovato said: “The development of AI technology is rapidly changing the way we navigate the landscape and I believe as artists we need to be a part of shaping what that future looks like. My career has been about pushing boundaries of and creating the most interesting music for my fans. I am open minded and hopeful that this experiment with Google and YouTube will be a positive and enlightening experience.”
Charli XCX said: “When I was first approached by YouTube I was cautious and still am, AI is going to transform the world and the music industry in ways we do not yet fully understand. This experiment will offer a small insight into the creative opportunities that could be possible and I’m interested to see what comes out of it.”
Alec Benjamin said: “I am proud to be partnering with YouTube and Google to help explore the future of AI technology and music. It’s exciting to be on the forefront of this new chapter in music history.”Music Business Worldwide
