Instant Genius - How AI and androids could shape the music of the future

Episode Date: May 24, 2021

While Daft Punk may have sadly split, machine-created music may be about to skyrocket in popularity. Not only are artificial intelligence neural networks now capable of creating original melodies, but scientists are also developing robots capable of playing – and improvising – live music. So, will AI and androids soon top the charts? And could they even replace human musicians entirely? On this week's episode of the Science Focus Podcast, Prof Nick Bryan-Kinns, director of the Media and Arts Technology Centre at Queen Mary University of London, joins staff writer Thomas Ling to explain groundbreaking new music technology. Let us know what you think of the episode with a review or a comment wherever you listen to your podcasts. Subscribe to the Science Focus Podcast on these services: Acast, iTunes, Sticher, RSS, Overcast Listen to more episodes of the Science Focus Podcast: Could these gloves be the future of music? – Imogen Heap Why do humans make music? The psychology of the sea shanty: Why work songs are such earworms Meet the computer scientist teaching an AI to play Dungeons and Dragons The creator of Bellingcat on using the internet to investigate global affairs The future of human flight, with real-life Iron Man Richard Browning Hosted on Acast. See acast.com/privacy for more information. Learn more about your ad choices. Visit podcastchoices.com/adchoices

You said this place was steps from the water. We just haven't found the steps yet. How much did we save? Enough. Enough to get lost. Or you could book a stay with Hilton. Welcome to your oceanfront room. Just steps from the water.

The Hilton sale is on now. Book on Hilton.com or the Hilton app and save up to 20% to get the stay you expected. When you want savings, not surprises. It matters where you stay. Hilton, for the stay. Study And play.

Come together on a Windows 11 PC. And for a limited time, college students get the best of both worlds. Get the Unreal College deal, everything you need to study and play with select Windows 11 PCs. Eligible students get a year of Microsoft 365 premium

and a year of Xbox GamePass Ultimate with a custom color Xbox wireless controller. Learn more at Windows.com slash student offer. While supplies last, ends June 30th, terms at AKA.m.S. College PC. Starting a business can seem like a daunting task unless you have a partner like Shopify.

They have the tools you need to start and grow your business. From designing a website to marketing to selling and beyond, Shopify can help with everything you need. There's a reason millions of companies like Mattel, Heinz, and all birds continue to trust and use them. With Shopify on your side, turn your big business idea into... Sign up for your $1 per month trial at Shopify.com slash special offer.

This podcast is sponsored by name, audio and focal. Streaming has made music more accessible than ever, but true listening is about more than ease. It's about quality. British audio experts name audio, alongside French acoustic specialist focal, combine handcrafted tradition with cutting-edge innovation and high-end materials,

delivering digital precision with analogue warmth. So you can experience exceptional sound at home. Music just as the artist intended. Visit name audio.com to live. learn more. You're listening to the Science Focus podcast from the BBC Science Focus magazine team, with the UK's best-selling science and technology monthly, available in print and in several digital formats throughout the world. Find out more at ScienceFocus.com or look out for us in your app store. Hello and welcome to the Science Focus podcast. I'm Thomas Ling, staff writer at BBC's Science

Focus magazine. Now while Darth Punk may have sadly split, Machine-created music may be about to skyrocket and popularity. Not only can artificial intelligence now create original melodies, but scientists are also developing robots that can play music live on stage. Will AI and Android soon top the charts? And could they even replace human musicians entirely? Here to explain groundbreaking new music technology is Professor Nick Brian Kins, director of the Media and Arts Technology Centre at Queen Mary University of London.

So to most people, I think generating music with the help of AI, it seems like quite a futuristic concept. But hasn't this technology actually been around for quite a few years? Yeah, it has, actually. If you look at the history of computers, making music with computers is one of the earliest things that people did back in the 1950s. people were creating music using computers. And you can even trace it back to, I think there was a quote from Ada Lovelace in 1843,

talking about how you may be able to use computers to generate some sort of music or make elaborate musical things using computers. So pretty long history there. How can AI be used to create new music on scratch? Yeah, now that's a good question. So personally, at the moment, I see. AI being used to not really create new music, but more to create like a pastiche or a copying or a derivative of existing music.

So you would take an AI algorithm and you would chuck loads and loads of music at it and then it could make versions of that, derivative versions of that kind of music. And so thinking about that, it's not really creating a whole new kind of music or a new kind songs or something like that, it's making new versions of existing kinds of music. So one of the big things people worry about is, oh, the AI is going to take over the creativity process and all that. But it's still relying on people to make the initial sets of music that it's trained on, right? So sort of coming back to the basics, though, so there seems to be a lot of terms banded

around like artificial neural network machine learning in terms of AI music. What actually are they? Yeah, so in terms of how the AI works, so you've got these neural networks that are really large sets of bits of computers that try and mimic how the brain works. So you basically throw lots of music at this neural network and it learns patterns. So much like the human brain learns patterns by repeatedly being shown things, it's the same with these neural networks. And what's tricky about the neural networks is these days, is they're getting bigger and bigger, so they're using more and more computer power. But it becomes harder and harder for humans to understand what they're actually doing. What has that AI learned? Difficult to know nowadays, because it's become so complex that it's really difficult for people to explain what the AI is doing.

The other thing that's interesting about these neural networks these days and the really big AIs that are like Google or so on would be putting out there is that they consume a lot of energy. So if you're trying to train AI to analyze the last 20 years of pop music, you're chucking all that data in there and then you're using a lot of electricity to do the analysis and to generate a new pop song based on that. So at some point there's going to become a balance where you say, you know what, is this actually causing too much environmental impact through the amount of energy we're using? Or is it got to the point now where I simply don't understand what this is doing? Okay, it's making some nice music, but I don't understand how it's working anymore. So there's going to be a balance there where we start to think, well, have we reached the boundary of what we can do with these neural networks, given the way that they work? I was going to say, I guess we have to sort of judge how good the music is, whether it's work. sort of ending the world for then.

That's right. Yeah. I mean, it's the same thing with like the Bitcoin and all that, where it's using so much energy to do the mining of the coins now. It's like, well, is this really a viable way to do a monetary system? So you could imagine, if you extrapolate that same way, you get to the point where you say, is it really worth putting all this energy into generating these pop songs when we could, you know, be encouraging humans to make music, right?

Can you give me sort of a ballpark figure of how much energy it does actually use, or how long it takes a computer or a supercomputer to generate new AI music? So it's really a question of how much data are you training it on and how complex is your model. The IEI models use really simple techniques, and you could actually do it by hand as a human you could go through and figure out the numerical values that you had to put into that model as possible. But as it got more and more complex, you know, as the computer's got faster, we started using more and more complex systems until we get to this point now where we have these essentially black boxes that we chuck music into and a nice new music comes out where we don't really understand the details of what it's doing.

So recently AI has been used to create quotes and quotes sort of new tracks from the likes of Amy Winehouse and Nirvana. is this an easy thing to do? Can you just sort of shovel of Amy Winehouse's back catalog into an AI and it could easily generate a new piece of her music? Yeah, I mean these days that's a pretty sort of straightforward thing that people do.

The trick is in some ways it's like a magic trick, right? Because there's a little bit of smoke and mirrors going on whereby you do chuck a load of music, Amy's music in, and you get a load of music that comes out. But somebody has to go and edit. that and say, oh, which of these do I like or which do I not like or which parts of these songs

do I think are good and I want that AI to work a bit harder on that part of the song and so on. So there's still a lot of human editorial control going on at the moment. So it's not as simple as, oh, let's put all of Amy's back catalog into this AI and it'll produce a new song. You put all the back catalog in. You'll get 100 or 1,000 new songs and you have to go through and decide which one is actually the one that you then want to put out as a new track. Yeah, I think that's really interesting. Say last year, AI, Your Vision was launched.

It's basically like Eurovision, but with two main differences. The first one, the UK, didn't finish bottom. It was six out of 13th. And the second, all the songs were made with the assistance of AI. It was just the assistance. It's so had a major human element to it. It's not simply just pressing a button on a computer. and at producing a song.

So do you think maybe there could be an AI within the next 10 years capable of generating a hit single all by itself about this human input? I'm not sure about that. I'm a skeptic on this side of things. Yeah, I would be saying, I think it could generate a hit single, but not on its own, yeah.

I think they already were some researchers who said that they've done this, where they'd generated some tracks, and then they just put all those tracks in, you know, put them out, and then people listen to them, and then it generated a number one track that way. But the point is it's still some human selection in there, right? Because even if you put all the 100 tracks or 1,000 tracks

that the computer makes out there, then people listen to the ones they like. It's still a human selecting which ones they think are nice or enjoyable. So, you know, there's always going to be some element of human selection or curation. because, to my mind, if the computer was able to generate the tracks on its own and had its own idea and music like aesthetic and said, oh, these are the kind of music I like, maybe we wouldn't like that kind of music, right? Maybe the computer likes the different kind of music than we do. Maybe it would just be all the AIs sitting listening to music together, and we would be like, what's this?

So, you know, the problem is we're trying to train the AI to make music that we like, But we're not allowing the AI to make music that it likes. That's really interesting how there might be a sort of translation difficulty between AI and humans. So in that respect, do you think it might be impossible for an AI to create lyrics that may sort of inspire certain emotions in humans? Yeah. I'm also kind of sceptic on that one as well. I have to say because, yeah, so that AI can generate a lot of lyrics that are interesting and, you know, they have an interesting narrative flow to them and that kind of thing.

The point is, though, that lyrics for songs are typically based on people's life experiences, like what's happened to them, you know, people they fall in love with, you know, things that have gone wrong in their life or watching the sunrise in the morning. You know, all these kind of things are not things that AIs do every day. So in terms of thinking about what can the AI generate in terms of its lyrical content, if it's actually trying to communicate so emotion, to me I'm a little bit skeptical that it would have that life experience to be able to communicate something meaningful to people

that isn't just a pastiche of previous songs. But how could AI music sort of move music as a whole forward? Like could it potentially create new genres? Yeah. So that's why I think the big thing will be where there'll be these mashups between different kinds, of musical styles, and this is quite a big research area at the moment, style transfer, like changing the style, taking the content of one kind of music and putting it in the style

of another kind of music, and exploring those different mashings together of maybe three or four different genres. So the kind of things that would be kind of difficult to do for people to do in a music studio, you know, it would be difficult, oh, how are we going to get four different genres of musician in a studio together and learn to play together and then create a new piece of music, that'd be something much easier for an AI to try a million different combinations of genres until one pops up, which happens to be something that we kind of like. He's saying to basically to create these new genres of music, we might have to listen to sort of 990, whatever terrible pieces, and then there'd be one genre as a defining piece, maybe.

Maybe, or maybe we have one of these systems set up where the AIs are sort of competing against to each other and they're able to, there's one AI that's judging saying, well, this one is probably the kind of things that humans would like. So there's a million different genres that they invent. And there's a different AI that's sort of looking at them going, well, maybe these five here are the ones that the humans will like. Let's see what they think. I'm instantly imagining an AI Simon Cowell here.

Exactly. That's what I'm thinking. Yeah, that's the way. So what would you say to people worried that introducing AI into music production? could kind of sap creativity from the entire industry? Yeah, I mean, I can see that in the short term, where people getting concerned about, you know, losing work in production or engineering,

these kind of things, or people whose job was to write pop music or tunes or something like that. But, you know, people say this with every introduction of new technology into music. So, you know, the invention of the gramophone record, for example, every was worried, oh, no, this is going to be terrible for music. And of course, then it launched the whole thing about recorded music sales, which is basically what we enjoyed today,

or the introduction of video to music, you know, MTV coming out. Everybody said, oh, this is going to be the end of music. But of course, it wasn't. It was a different way of consuming music. So I think it will have an effect. And what it will partly do will, it might allow more people to make music, because it's now much easier to make,

professional sounding music using just even your phone. So to what extent should we see AI as a tool in musical production rather than just giving us a finish result? A tool would be a good way of thinking of it. But I think in the future, in the very soon future, it'll be more thinking of AI as a co-creator, so a partner in making the music. So, you know, if it's somebody,

I'm talking about it as a person now, If it's something that you create music with, maybe it's able to help you out by suggesting some bass lines, or maybe it gives you some ideas for different lyrics that you might want to use based on the genres that you like. Or maybe it throws in a bit of an oddball suggestion for a musical riff somewhere based on some different genre that you didn't know. What about the technical challenge is? Which are the biggest sort of technical hurdles which AI needs to overcome? Yeah. Well, the sort of big technical challenge that AI needs to overcome in the moment is to be able to generate across different, how to describe it, different structures in the music.

So at the moment, AI is quite good for generating small riffs or it can do melodies or it can do other elements of a musical content. But it's not very good at being creative across different levels of the music. So, like thinking about the big picture of the piece that you're generating as well as the small motifs. So the big challenge at the moment is really integrating across, you know, the bigger, thinking about the whole piece of the music, you know, all the different pieces playing together, but also the, you know, if it's a classical piece, you know, it could be 10, 15 minutes long. How do you generate at that scale, but also at the note scale? That's the thing that AI is not great at the moment. How good is AI at replicating sort of human voice of singing? Actually, again, it goes down to the training.

So there's quite a few examples of pretty convincing human voice simulation these days. So it's pretty good. And the question is, what do you want it to sing? And why do you want it to sound like a human anyway? Well, you get streaming services like Spotify and Tidal. And they're already using algorithms at the moment to generate suggestions. to users based on their listening history. Do you think in future it would be possible to take this a bit further?

Like, for instance, could AI create music in real time based on someone's mood or someone's heart rate? Yeah, definitely. And this is a really interesting research area at the moment and quite a lot of research looking at how you map between, as you say, heart rate or maybe skin conductivity, like how sweaty you are or maybe the size of your pupils, these kind of things, to the kind of music that you're listening to, and then think about how you can use music to change or influence your mood, either to make you a bit more happy or a bit more sad and relax,

this kind of thing. So it's definitely an area of research in a moment. The main issues is that often music has very personal meanings, right? And so you can do these research studies where you say, okay, out of 1,000 people, 900 people, when they listen to this track, feel a little bit sadder, or, you know, 870 people feel a bit happier or whatever it is. But for each individual track and each individual person, there'll be personal experiences that may be connected with that song. So, for example, it may be that there's some song that most people think it's sad, but you played it at your wedding, so you think it's really happy.

You have happy associations or the other way around. Maybe there's a happy song that somebody played at a funeral, and so every time you hear it, you're a bit sad and nostalgic. So the challenge is that there are many things that can be sort of generalised about how music influences your mood and you can use certain tracks to, on average, increase people's mood. But for each individual person, it's very difficult to learn all those connections between the music. What's the most exciting thing to you personally in the world of AI music at the moment? Oh, robots. Oh, I love musical robots, yeah. So this is like being a dream of everybody for like a hundred years or something, a robot that can play music. So this is a kind of interesting time, I think, where people are now building robots that you can play in real time with them in a sort of band-like situation. And they'll do sort of gestures like a musician would do, like they'll sort of nod or keep a beat by tapping their foot, they'll look at you when it's your turn to do a solo, you know, this kind of stuff. It's really cool. the fun thing is that they're taking, it's not just the music that they're making,

but it's the gestures that go around the music, the stuff that happens when you play in a band, you're nodding, you know, you're getting into the groove and all that kind of stuff. And I think that's really interesting to me is that these combination of stuff has come together where we can really feel like it's a real living thing that we're playing music with. Yeah, I think that's great. So is there the same kind of leaps and bounds in terms of virtual reality as well, or is it mainly just the robots which are really grabbing you at the moment? It's really the robots.

Yeah, I mean, virtual reality is interesting in terms of sound hasn't been used that much in virtuality because people are mostly focusing on the visual side of things and trying to make the interaction more natural. So being able to use your hands to move around in VR, you know, using your hands to interact with things. So the sound and the music has kind of played a second place at the moment in VR. And I think there is a lot potential to use, like you say, how they use music and computer games.

It'd be great to be able to use that more in VR, but not just for sort of background music, but to make the experience feel more immersive, more realistic and try and convey more of emotions rather than space. So have you personally sort of had a jam session with a robot yet or not? No, I haven't had a chance to have a jam session with a robot yet, no.

But we have a number of people that we're working with on different kind of robots. There's some really great ones in the States actually in Georgia Tech. They've got some really great ones that play a variety of instruments. It would be great when we can travel again.

I was planning to go and visit them and have a bit of play. So do you think in future, if you have sort of a a solo musical artist could maybe go on tour a company by a band of robots essentially. Yeah, for sure. I think so, that'd be great, right?

And I think especially as we come out of lockdown and all that and stuff, and people really want to have more physical experiences again, like go to gigs and events and so on. And I think that's why, to me, you know, the VR virtual reality music, I don't find it so compelling, especially at the moment where we're all looking at a screen every day. I think the physical stuff where you actually go and you physically see or you play

music with other beings, let's say that, I think that would be the real thing that's going to catch people's imagination. But won't sort of AI be working alongside these robots, say if you needed to, if you're having a live gig and you had your band of robots there, wouldn't a certain element of AI be needed to sort of jam on the stage? It's come to a point where the AI and the robots and the sort of computer vision tracking stuff all seems to be getting to a point where it can work in real time and it can actually feel fluid and realistic in terms of making music and that sort of sensation of jamming. What will be interesting will be, you know, what does the audience look for this?

So at the moment, you know, it's quite fun to play as a musician with a robot. You know, is that fun to watch for an audience? I'm not sure if it is. maybe this. I think judging by Darth Punk, I think it would be fine. That's true. So in that way, do you think there's still quite a little prejudice about AI music and robot music? Yeah, I think the expectations would be that the robot is going to play basically some sort of electronic music or it's going to sound very synthesizery, right? And one of the groups I've been working with in China's been looking very much about how robots can play really traditional instruments, like that's super

traditional instruments and how they can also perform in a way that sort of captures the traditional movements and mannerisms, but in a sort of robot type way, if that makes sense. So they're not trying to recreate, you know, an android that looks like a person that moves in the same way, but how can you create robots that sort of have the same performance feeling? Oh, look, yeah. So I think that will be interesting to see how we can sort of merge the sort of AI and the robot and the tradition to create this sort of hybrid thing going on. Great. Well, thank you so, so much for your time. We're hopefully speak to you again soon. Thank you very much. Thank you for listening to this episode of the Science Focus podcast from the BBC Science Focus team.

If you've enjoyed this episode, please leave a review whatever you're listening to us. You can see more amazing insights into future technology on sciencefocus.com. We can also find the latest science news and stories. Thank you for listening to the Science Focus podcast from the BBC Science Focus magazine team. We're the UK's best-selling science and technology monthly, available in print and in several digital formats throughout the world. Find out more at sciencefocus.com or look out for us in your app store.

This podcast is sponsored by Name, Audio and Focal. The texture and emotional depth of music can be lost through digital sources or poor signal. Name Audio believes you can have digital precision with analog warmth. Alongside French acoustic specialist focal, Name creates high-end audio systems, combining innovation with craftsmanship, so you can listen to music, just as the artist intended. Discover more at name audio.com. Ambition comes in all shapes and sizes.

At First Citizens Bank, we roll with your goals, because we're built for what you're building, fit for your ambition for citizens back there's a new way to sweet green meet wraps handheld, hearty and made for life on the moon with bold chef-crafted flavors fresh ingredients and over 40 grams of protein

they're built to satisfy without slowing you down try wraps today in the app or at order.com available at all participating locations