Testing the Bounds of Creativity: My Experience with Google’s Latest AI-Powered Music Generation

Quick Links

• What Is MusicFX?
• Prompts Are Awesome for Making Music on the Fly
• DJ Mode Creates Controlled Chaos
• MusicFX Is Perfect for Jamming
• Limitations of MusicFX

Key Takeaways

• MusicFX by Google is a new generative AI tool for making music.
• The DJ mode allows you to mix together different elements to create a continuous tune.
• While lacking lyrics and control in DJ mode, MusicFX is easy to use.

The emergence of generative AI has opened new doors for creatives. These tools can be used to create unique art, which has historically been a way for humans to express emotion. Arguably, the most interesting one is music, which follows certain rules, but how good is AI at making music if it doesn’t even have a pair of ears to enjoy it?

What Is MusicFX?

MusicFX is one of Google’s latest generative AI tools (or, should I say, toys) that allows you to create new music based on a simple prompt. MusicFX was built on top of MusicLM , which was released in 2023. It’s actually part of Google’s AI Test Kitchen, which also has TextFX , ImageFX , and VideoFX . I’ll let you infer what the other tools can generate. There’s also a library for everything you’ve generated with all of these tools. Nice.

It’s worth noting that MusicFX isn’t the first nor the only music AI tool. My colleague Dibakar swears by Suno and Udio , both of which can create convincingly realistic music that’s almost indistinguishable from a real band.

MusicFX isn’t even Google’s first AI music tool, as Dream Track for YouTube Shorts predates it. The two tools are completely different, though. YouTube teamed up with nine artists to use their vocals and create music in their style. MusicFX refuses to add any vocals or copy popular bands and songs, which means you can’t ask it to create a Babymetal version of your favorite Taylor Swift song.

The most unique feature of MusicFX is the AI DJ tool, which lets you mash up different genres, instruments, and styles. The end result is always more harmonious than you’d expect, even if your prompt soup consists of “flute,” “weird, ominous noises,” and “surf rock.” You’ll see some examples later.

Prompts Are Awesome for Making Music on the Fly

In the basic prompts mode, all you have to do is enter a basic prompt in the text box. MusicFX takes a couple of minutes to process your prompt and creates two versions of the prompt. You can download and share either version.

In the “Settings” menu, MusicFX also lets you choose between 30-, 50-, and 70-second durations. You can also set it to loop, which will stitch the ending and beginning together. This is perfect if you just want a continuous relaxing beat for work and study. The settings also have a seed number, which you can modify or lock to keep MusicFX making similar tunes.

I especially love how MusicFX automatically recognizes the keywords in your prompt and gives you a drop-down menu for each, so that you can customize the prompt on the fly. Check out the nifty tempo slider that appears under “builds up,” allowing me to go from “insanely slow” to “REALLY fast” (the emphasis is entirely MusicFX’s).

I know you’re itching to hear an example, so let’s give the floor to MusicFX with the prompt “Upbeat techno song with guitars in the background. It builds up toward the end.”

While the song we got here isn’t necessarily techno, it’s definitely upbeat, has guitars, and slowly builds up. This isn’t to say that MusicFX can’t do techno, though, quite the contrary. Techno music is often very repetitive, with subtle changes that evolve over time; MusicFX lends itself perfectly to creating those kinds of beats. I could ask MusicFX to make subtle changes or variations to the following track and listen to it for hours because it’s that good.

I’ve noticed that MusicFX excels in all genres that don’t rely on vocals, such as most types of electronic music, jazz, hip-hop (in terms of beats), Lo-Fi, ambient, classical, and so on. However, it also does a great job with any other genre you can throw at it. Here are two amazing versions of the prompt “Fast thrash metal song with a heavy guitar riff and a lead guitar solo over it. Double bass drumming and a prominent bass line:”

vMix Pro - Software based live production. vMix Pro includes everything in vMix 4K plus 8 channels of Replay and 8 vMix Call
This bundle includes Studio 200 for vMix from Virtualsetworks, HTTP Matrix 1.0 automation scheduler, and 4 introductory training videos from the Udemy vMix Basic to Amazing course.

DJ Mode Creates Controlled Chaos

DJ Mode is the newer and wilder feature in MusicFX, which lets you harmonize a mix of instruments, genres, and styles in real time using “chips.” Instead of making tracks, you use DJ mode to create a continuous virtual DJ set. You can add and manage how much of a chip you want on the left, and the temperature slider on the right determines the diversity that the AI will output. It’s much more messy than prompts mode, but it’s also more engaging and fun.

Your browser does not support the video tag.

The music you can create within DJ Mode almost always sounds fantastic, regardless of how many chips you add. The catch is that you don’t have that much control over the song and can’t fine-tune it to capture what you had in mind. I feel like MusicFX treats your adjustments to the chip sliders as mere suggestions rather than instructions, so the music doesn’t always correspond to what you see on your screen.

Your browser does not support the video tag.

In this example, I started off with a simple slap bass line and wanted to add an ominous drone, which initially replaced the bass line altogether and later clashed with it, so I decided to ditch it in favor of a droning guitar riff. I later added a steel drum, which worked rather well.

MusicFX Is Perfect for Jamming

While the DJ mode is fun, I feel like you can get the most from MusicFX if you use it to experiment. I play guitar and bass, so maybe I should throw in some ChatGPT guitar riffs to see if they’ll work with a drum beat from MusicFX. I feel like MusicFX is the perfect jam buddy when you want to have fun with friends but are a couple of instruments short of a full band. It’ll lead the way, and you can throw in your tunes on top of the AI base.

Marc Rebillet, an experimental musician and DJ, used a sample from MusicFX at Google I/O and built on top of it to create a catchy, all-original tune. Rebillet only needed a couple of minutes to showcase what can be done with MusicFX with some creativity and musical knowledge.

Still, you don’t have to be a professional musician to make music with MusicFX. The tool will do the heavy lifting for you; your job is simply to come up with unique combinations to create music.

Limitations of MusicFX

The lack of lyrics and music inspired by real-world artists is MusicFX’s most significant limitation. Other AI tools have already conquered this, so we can deduce that Google set that limitation to avoid controversies and copyright issues.

Another limitation is that the tool won’t always do what you think it will. As I demonstrated above, you have to surrender a lot of control over individual elements. For example, if you add a bass line to the DJ tool, it’ll automatically add several other instruments without a chip to manage them.

Even though MusicFX is a random Google AI experiment, it feels quite polished and ready for everyday use. I could easily see myself replacing Spotify with MusicFX for background tunes. Given how much I listen to music, I’m tired of my Daily Mixes , so it’s nice to hear completely original music that can adapt to my mood with a simple prompt.