Combining music and AI art — one of my personal projects
Hello there! I'm Aia. :D
And today, we're going to talk about... music! Well, I'm the one who's going to talk, but it doesn't matter. :P
But before beginning with the actual blog post, I think I'm going to explain why I like music so much. Oh, and I also have to warn you that this is basically my creative process for this project! :P Well, let's get to it.
Music has always held a special place in my heart, and I believe it's one of the most incredible art forms out there. Its power is truly remarkable – it can evoke such a wide range of emotions within us. I find it astonishing that a mere arrangement of sounds and melodies can have such a profound impact on our feelings. Music has the unique ability to relax and soothe us, but it can also stir up intense emotions, bringing us to tears. It can make us feel joy, sadness, nostalgia, excitement, and everything in between.
The beauty of music lies in its versatility and universality. It transcends language and cultural barriers, connecting people from all walks of life. Whether it's the gentle strumming of an acoustic guitar or the booming bass of an electronic track, music has the power to move us. It can transport us back in time to cherished memories or inspire us to envision new possibilities for the future. The combination of lyrics and instrumentation creates a white canvas upon which artists paint their stories, and we, as listeners, interpret and relate to these stories in our own unique ways.
Furthermore, music has an incredible ability to enhance our experiences. It can turn mundane tasks like washing the dishes into enjoyable rituals and elevate our workouts or daily commutes. It can provide comfort during difficult times and serve as a source of motivation when we need that extra push. The right song at the right moment can truly make all the difference in our day-to-day lives.
For me, music is an essential companion on this journey through life. It has been a constant source of joy, comfort, and self-expression. Whether I'm singing along to my favorite tunes in the shower — yep, it's my fault when it suddenly rains in a sunny day :P —, discovering new artists, or creating music of my own, it continues to enrich my life in ways that few other things can. It is a universal language that speaks to the heart, and I am forever grateful for its presence in my world.
And... that was longer than I expected, but I think I'm going to leave it like that. ;P You can skip it tho. :)
Well, where were we? Oh, yes. Today, we'll be talking about combining music and AI art. Sounds like a good topic for me! :D
The idea is combining two things: music and images. And to do so, we have to choose a song and get the lyrics (you can just write them down or copy them from the Internet, up to you). Then, try to imagine those lyrics as images, and divide them into small phrases (usually the verses are good to go, but sometimes you'll stumble upon some long ones and you'll have to divide them). Finally, you just have to open an AI art generator (NightCafé and playground.ai are the ones I'd recommend you for this; with NC you have unlimited base image generations, and with Playground you get 100 daily images, which is a decent amount) and start creating! Well, I forgot to mention thatif you want to see how it looks like, you might want to open a video editor and put the images in the right order (and length!), so good luck with that! 😝 I'm uploading my tries to YouTube, so feel free to give it a look!
And I'm seeing that this post is maybe too short, so here's my full experience with this:
A month or so ago, I was listening to "The Kids Aren't Alright", by The Offspring, when I thought to myself: "Hey, how would this song look like? Oh, what if we tried to see how does AI draw it?" And I got to work. I mean, it's actually a cool concept. Songs are usually full of metaphores, and it could be interesting just seeing if the image the algorithm draws is similar to what I draw in my mind — spoiler: it usually isnt! —, so I sit down, grabbed my headphones and listened to some music. Hey, don't blame me, I had to concentrate!
The first song I "translated" (I think I'm going to use that term from now on, as what I'm doing is basically translating words to images with the help of AI) was that song, "The Kids Aren't Alright" (The Offspring). It's a powerful song with interesting lyrics, which you most probably have listened at least once (even if you still don't know it ;P). I opened NightCafé, but then remembered I'm poor and closed it and opened playground.ai and began working. I just copied the song lyrics and put them as the prompt, and then all I had to do was downloading the images, putting them in CapCut... et voilà! The first song was done.
And it was horrible.
What had happened there? Weird pictures with no coherent context, off-topic images... I was doing something wrong there... So I thought: "What are we doing wrong? Wait. We can't just give a phrase to an AI algorithm, it needs some context!" So I tried again, the next song being "Uprising" (Muse), another awesome piece. However, this time, I added some modifiers after the song lyrics to try to get better results. And it worked, at least to some point. The images this time had a decent composition. Not really on-topic, but at least there wasn't a random glowing orb in any of them.
The third song I did was "Livin' on a Prayer" (Bon Jovi). I love that song. :P This time, I retouched the modifier prompt, and the results had nothing to do with the previous ones. Finally! :D
And the last one was "Coraline" (Måneskin), a really beautiful song. This one was a challenge, as the lyrics are in Italian and it looks like the AI didn't understand them really well, so I had to add a brief description in English for some parts. Still, the result was decent.
And now, I'm doing this in a much bigger scale... I'm creating a full album from scratch! Entirely with AI tools: Mistral for the lyrics, Suno for the music, Playground for the images (if you work there don't worry, I'm not trying to monetise it lol). If you want to see it when I finish it, feel free to subscribe to my other YouTube channel!
I'll probably create a tutorial for Suno soon, in case anybody's interested (let me know if you want!), and feel free to check my other posts!
Well, that’s all for now. Thanks for sticking around, and don’t forget to leave a comment or send me a message. I’d love to hear from you!
Until next time, keep dreaming, keep creating, and keep having fun! 🚀
Bye for now!
Aia
What do you think?
Nice post Aia! :D
It was good, but I'm not interested in your life. :/
Horrible. Please, don't do this EVER again. :(
Comments