Google created an AI that can compose music from word descriptions, but won’t be released
- ByStartupStory | January 28, 2023
Google‘s remarkable new AI system can make music in any genre based on a text description. However, because to the hazards, the corporation has no imminent plans to release it.
Google‘s MusicLM isn’t the first generative AI system for songwriting. Other attempts included Riffusion, an AI that composes music by picturing it, Dance Diffusion, Google’s own AudioML, and OpenAI’s Jukebox. However, due to technical constraints and inadequate training data, none of them have been able to produce songs that are exceptionally sophisticated in composition or high-fidelity.
MusicLM may be the first to do so.
MusicLM was trained on a dataset of 280,000 hours of music to learn to generate coherent songs for descriptions of “significant complexity,” as the creators put it (e.g., “enchanting jazz song with a memorable saxophone solo and a solo singer” or “Berlin ’90s techno with a low bass and strong kick”). Its songs, surprisingly, seem like something a human artist might write, albeit not as innovative or musically harmonious.
Given that there are no musicians or instrumentalists in the loop, it’s difficult to stress how fantastic the samples sound. MusicLM captures details such as instrumental riffs, melodies, and emotions even when fed rather extensive and meandering descriptions.
MusicLM’s powers go beyond only creating song clips. The researchers at Google demonstrate that the algorithm can build on existing melodies, whether hummed, sung, whistled, or played on an instrument. Furthermore, MusicLM may take a series of descriptors (for example, “time to meditate,” “time to wake up,” “time to run,” “time to give 100%”) and construct a melodic “story” or narrative lasting up to several minutes – ideal for a movie soundtrack.
That isn’t everything. MusicLM can also be directed using an image and a description, or it can generate audio that is “played” by a specific sort of instrument in a given genre. The AI “musician’s” experience level can also be set, and the system can generate music influenced by locations, epochs, or criteria (e.g. motivational music for workouts).
But MusicLM is far from perfect — far from it, in fact. The altered quality of some of the samples is an unavoidable side effect of the training process. And, while MusicLM can synthesize vocals, including choral harmonies, it falls short of expectations. The majority of the lyrics span from barely English to outright gibberish, and are sung by synthesized voices that sound like mash-ups of other singers.
“We acknowledge the risk of potential misappropriation of creative content associated with the use case,” the co-authors of the paper wrote. “We strongly emphasize the need for more future work in tackling these risks associated with music generation.”
If MusicLM or a similar system becomes available, it becomes certain that serious legal concerns will arise — even if the systems are positioned as tools to aid artists rather than replace them. They already have, although in the form of simpler AI systems. In 2020, Jay-record Z’s company filed copyright strikes against Vocal Synthesis, a YouTube channel that used AI to make Jay-Z renditions of songs like Billy Joel’s “We Didn’t Start the Fire.” After removing the videos, YouTube reinstalled them after discovering that the takedown requests were “incomplete.” However, deepfaked music remains on shaky legal ground.
From the standpoint of the user, Waxy’s Andy Baio speculates that music made by an AI system would be regarded as derivative work, with just the original portions protected by copyright. Of course, it’s unclear what constitutes “original” in such music; exploiting it commercially is an uncharted territory. It’s easier if generated music is used for purposes protected by fair use, such as satire and commentary, but Baio expects courts to make case-by-case decisions.
It might not be long until there is some resolution.Several litigation currently pending in court will almost certainly have an impact on music-generating AI, including one involving the rights of musicians whose material is used to train AI systems without their knowledge or agreement. But only time shall tell.
Follow Startup Story
Related Posts
