Y Combinator Startup Podcast - #20 - Making Music and Art Through Machine Learning - Doug Eck of Magenta

Episode Date: July 21, 2017

Doug Eck is a research scientist at Google and he’s working on Magenta, a project making music and art through machine learning.If you want to learn more you can check out Magenta.Tensorflow.org

Hey, this is Craig Cannon, and you're listening to a Wide Combinator's podcast. Today's episode is with Doug Eck. Doug's a research scientist at Google, and he's working on Magenta, which is a project making music and art through machine learning. Their goal is to basically create open source tools and models that help creative people be even more creative. So if you want to learn more about Magenta or get started using it, you can check out magenta.tensaflow.org. All right, here we go. I wanted to start with the quote that you ended your I-O talk with, because I feel like that might be helpful for some folks. So it's a Brian Eno quote, and I will have the slightly longer version.

Yeah, good. So yeah, it goes like this. Whatever you now find weird, ugly, uncomfortable, and nasty about a new medium will surely become its signature. CD distortion, the jitteriness of digital video, the crap sound of 8 bit, all of these will be cherished and emulated as soon as they can be avoided. It's the sound of failure. So much modern art is the sound of things going out of control. out of a medium pushing to its limits and breaking apart. So that's how you ended your IOTalk.

Correct. And what it kind of opened up for me was like what, when you're thinking about creating Magenta and all the projects they're in as new mediums, how are you thinking about like how are you thinking about what's going to be broken and what's going to be created? The reason that I put that quote there, I think, is to be honest with the division between engineering and research and, artistry and to not think that what I'm doing is being a a machine learning artist, but we're trying to build interesting ways to make new kinds of art. And I think, you know, it occurred to me. I read that quote and I thought, you know, that's it, right? No matter how

hard Eastman or whomever invented the film camera, I'm sorry if that's the wrong person, right? Like, they clearly weren't thinking of breakage or they're trying to avoid certain kinds of breakage. I mean, you know, guitar amplifiers aren't supposed to distort, you know. And, you know, I thought, well, what if we do that with machine learning? Like, like, models that are, like, the first thing you're going to do, if you think, if someone comes to you and says, here's this really smart model that you can make art with, what are you going to do? You're going to try to show the world that it's a stupid model, right?

But maybe the way that, maybe it's smart enough that it's kind of hard to make it stupid. So you get to have a lot of fun making it stupid. Right. I was playing with a quick draw this morning with my girlfriend. And what she was trying to do was make the most accurate picture that the computer wouldn't recognize, like immediately out of the gate. She works in art and, like, yeah, doesn't want to believe. That's right.

It's a good intuition. I mean, you know. Yeah. So maybe the best ways to start is then talk about, like, what are you working on right now? What are you guys making? So right now we're working on, kind of let me think. It's a good question. We have this project called Ensynth, which is trying to get deep learning models to generate

new sounds. And we're working on a number of ways to. make that better. I think one way to think about it is we have this latent space. We have a, so to make that a little bit less a buzzword, we have a kind of compressed space, a space that doesn't have the ability to memorize the original audio, but it's set up in such a way that we can try to regenerate some of that audio. And in regenerating it, we don't get back exactly what we started with, but hopefully we get something close. And that space is set up so that we can move around in that space and come into new points in that space and actually listen to what's there.

Right now, it's quite slow to listen, so to speak. We're not able to do things in real time. And we also would love to be at kind of a meta-level building models that can generate those embeddings, having trained on other data, so that you're able to move around in that space in different ways. And so we're kind of moving, we're continuing to work with sound generation for for music. And we also are spending quite a bit of time on rethinking the music sequence generation work that we're doing. We put out some models that were, you know, by any reasonable account primitive.

I mean, kind of very simple recurrent neural networks that generate MIDI from MIDI and that maybe use attention, that maybe, you know, have a little bit smarter ways to sample when doing inference, when generating. And now we're actually taking seriously, wait a minute, what if we really look at large data sets of performed music? What if we actually start to care about expressive timing and dynamics, care deeply about polyphony, and really care about not putting out kind of what you would consider

a simple reference model, but actually what we think is super good. And I think those are the things we're focusing on. I think we're trying to actually make things really pull up quality and make things that are better and more usable for people. And so with all of that supervised learning, are you going to create a web app that people will evaluate how good the music is? Because I heard a couple interviews with you before where that was the issue, right?

Like, how do you know what's good? Yeah. So that is like, I'm pausing because that's the big question I think in my mind is how do we evaluate these models? at least for magenta i haven't felt like the quality of of what we've been generating has been good enough to bother so to speak like like you find it you cherry pick you find some good things like okay this model trains and it's interesting and now we kind of understand that the API the input output of what we're trying to do um i would love yeah i don't know how to solve this like i conceptually what we do here's what we do right we build we build a mobile

app and we make it go viral. That's what we do. Right. And then once it's viral, we just keep feeding all of this great art and music in. And I used to do music recommendation. We just build a collaborative filter, which is a kind of way to make, you know, recommend items to people based upon what they like. And we start giving people what they like, and we pay attention to what they like, and we make the models better. So all we need to do is make that app go viral. There's just one simple thing. In fact, maybe someone in the Y Combinator world can help do that. It's like sliding between cow and trombone.

Exactly. Exactly. Yeah, right. Maybe that particular web app is not the right answer. Now, I mean, I'm saying that as a joke, but I think, look at it this way. If we can find a way, or the community in general can find a way for machine-generated media to be sort of out there for a large group of interested users to play with, I think we can learn from that signal, and I think we can learn to improve.

And if we do, we'll make quite a nice contribution to machine learning. We will learn to improve based upon human feedback to generate something of interest. So that's a great goal. But I'm totally like today in this room, you know, I wish I could tell you we had a secret. We had like a secret plan, you know, like we're, you know, oh, he's figured it out. The app's going to launch tomorrow. It's like, it's really hard to work. Like bleep cut.

Yeah, yeah. Exactly. Sorry. Interesting. Okay. Because I was wondering what kind of data you were getting back from artists. Do people just use all of your projects, you know, all of the repos to create things of their own interest? Are they pushing back valuable data to you?

So we're getting some valuable data back. And I think what we're getting back, some of the signals that we're getting back are giving us such an obvious direction for improvement. Like, why would I want to run a Python command to generate a thousand MIDI files? That's not what we do. You know, like, you get that kind of feedback. You're like, okay, we wanted this command line version because, you know, we need to take. to be able to test some things. But if musicians are really going to use the music part of what we're doing, we have to provide

them with more fluid and more useful tools. And there, I think we're still sitting with so many obvious hard problems to solve, like, integration with something like Ableton or like really solid, you know, real-time I-O and things like that that we know what to work on. But I think we'll get to the point pretty quickly where we'll have something that's kind of solves the obvious problems, plugs in reasonably well to your workflow, and you can start to generate some things and you can play with sound. And then we need to be much more careful about the questions we ask and how good we are at listening to how people use what we're doing.

And so what are artists using it for at this point? Right now, so we have most of what we've done so far as it had to do with music. If we look for a second away from music and look at SketchRN, which is a model that learned to draw, we've actually seen quite a bit of – so first, at a more – at a higher level, sketch R&N is a recurrent neural network trained on sketches to make sketches. And the sketches came from a game that Google released called QuickDraw, where people had 20 seconds to draw something to try to win at Pictionary with a computer, you know, a classifier counterport. And so, you know, we trained a model that can generate new cats or dogs or whatever. There's some really cool classes in there. A cruise ship is nice. Yeah. The one that always threw me was camouflage. Like it was, like it. out camouflage all the time. Like, I've never, as if by definition, you can't draw it, right? Yeah. Nothing. Pause for 20 seconds.

Right. Yeah, I actually won a dictionary round with the word white. And I just pointed at the paper, I'm like, no way. And she said, white. I'm like, you've got to be kidding. Yeah. Okay. So anyway, it's kind of like a corollary to camouflage. So we've seen artists start to like sample from the model. We've seen artists

using the model as a distance measure to look for weird examples, because the model has an idea of what's probable in the space. We've also seen artists just playing around with the raw data. And so there's been a nice explosion there. I think, you know, I'm not expecting that artists really do a huge amount with this quick draw data, because as cool as it is, these things were drawn in 20 seconds, right? There's kind of a limit to how much we can do with them. On the music side, we've had a number of people playing with N-Synth with just like dumps of samples from N-Synth. So basically like a rudimentary synthesizer. And there I've been surprised at the kind of, I would expect that if you're really good at this,

so like your AFX twin or you want, how about this, you want to be Apex Twin, right? That you look at this and go, yeah, whatever. There are 50 other tools that I have that I can use. But those are the people that we found have been the most interested. because I think we are generating some sounds that are new. I mean, so first, as someone pointed out on hacker news, you can take a few oscillators on a noise generator and make something new. But I think these are new in a way,

when you start sampling the space between a trombone and a flute or something like that, that these are new in a way that capture some very nice harmonic properties, capture some of the essence of the Brian Eno quote, are kind of broken and glitchy and edgy in a way. But that glitchiness is not the same as you would get from like a digital clipping. The glitchiness sounds really harmonic. And so like for example, Jesse on our team, Jesse Engel, he built some Ableton plugin where you're listening to these notes, but you're able to change the, like erase the beginnings of the notes.

So you like erase the onsets, which is usually where most of the information is. Like most of the information in a piano note is kind of that first percussive onset. But it's the onsets that the model is doing such a great job of reproducing because it gradually kind of moves away from in time. It's a temporal embedding and the noise kind of adds up as we move through the embedding in time. So it's the tails of these notes that start to get ringy and they'll detune and you'll hear these rushes of noise come in or there'll be this little weird whip at the end. And so like we found that that musicians who've actually played with sound a lot,

find these particular sounds immensely fascinating. I think that the kinds of sounds that sound interesting in a way that's hard to describe unless you've played with them. I think they're interesting because the model has been forced to capture some of the important sources of variance in real music audio. And even when it fails to reproduce all of them, when it fills in with confusion, so to speak, even that confusion is somehow driven by musical sound. And which, you know, you see by the corollary, you know, if you look at something like Deep Dream

and you see what models are doing when they're sort of showing you what they've learned. It may not be what you expect from the world, but there's something kind of interesting about them, right? Anyway, it's a long answer, but the short version of the answer is we found that working with very talented musicians has been really fruitful. And our challenge is now to be good enough at what we do and make it easy enough and make it clean enough that even someone who's not an Afex Twin. And I'm not saying we worked with Apex Twin. We didn't work with AVEX Twin. But like, you know, that kind of artist. Yeah.

That we can also be saying, hey, this is really like genuinely, musically engaging for a much, much larger audience. That's surprising. So it's not necessarily generating melodies for people so much as it is generating interesting sounds. That's what's brought them in. That's what's brought them in. Though the parallel has existed for the sequence generation stuff.

And what I noticed, even with AI duet, which is this, like, web-based, like, it's a simple R&N, it's like, I can lay claim. It's, it's technology that was published in 2002. It's really a very simple, really simple. But like, so like, so this model, if you haven't, you know, if your viewers haven't seen it, you play a little melody and then the model thinks for a minute and the AI genius, which is an LSTM network comes back and play something back to you, right? If you play for release, you know, da-da-da-da-da-da-da-da-da-da-da-da-da-da-dun, right, and you wait, you're expecting maybe, you know, that it'll continue the tune. It's not going to, right? It's going to go So this idea of like expecting the model to like carry these long arcs of melody along is not really understanding the model.

What we saw was like like especially jazz musicians, but like musicians who listen, the game they play is to follow the model. And so they'll like, I'd see guys or people, women too, sit down and go like, you know, dum, dum, dum, dumb, dumb, dumb, and just wait. And it's almost like pinging the model with an impulse response. Like, what's this thing going to do? And then instead of trying to drive it, it comes back and goes, dumb, dumb, dumb, right? Yeah. And then the musician says, oh, I see, let's go up to the fifth. And then you get this really, it's almost like follow the leader, but you're following the model.

And then it's super fun. And like, it's basically a challenge for the musician to try to understand how to play along with something that's so primitive. Right. But if you don't have the musical, so basically it's the musician bringing all the skill to the table. Right. So, like, even with the primitive sequence generation stuff, like, it's still been interesting to see that it's musicians with a lot of musical talent and particularly the ability to improvise and listen that have managed to actually get what I would consider interesting results out

of that. So, yeah, so it's become more of like a call-and-response game than a tool. Yeah, I think so. I think it, and that's partially because the model, the model's pretty primitive. I think that we can, you know, if we can get the data. pipelines and orders so that we know what we're training on and we can actually do some more modern sequence learning, you know, having like generative adversarial feedback and things like that. We can do much better. And even we have some stuff that we haven't released yet that I think is better.

But yeah, I think as we make it better, it'll be more of a this model is going to give me some more ideas from what I've done. Right now it's more of a, this model's kind of weird, but I'm going to try to understand what it's doing. Both are fun modes, by the way. Yeah. They're both cool modes. Yeah. I mean, I've enjoyed it. Like, I'm definitely not a pianist. I mean, I've played guitar before. And I tried to get a song going, but I had trouble with it. We're sorry. It's okay. Yeah, I think it's mostly my fault, to be honest. I love the YouTube video. Blame the user, right? Yeah. Yeah. The video where some, where that guy played a song with it, that was amazing. Yeah, that was cool. It was very cool. Have you seen a lot of that stuff as well? Yeah, we've seen, we saw, like, we haven't, I haven't, well, we haven't pushed the sequence generation stuff much because we really wanted. to focus on tamper.

But when we have released things and kind of try to show people we're there, yeah, we've gotten, if you look on them, there's a magenta mailing list that's just like it's linked in G.coma magenta. And if you look around, there's like a discussion list, which is like as flammy and spammy is some discussion lists, but a little bit less so. It's pretty, you know, every couple weeks someone will put up some stuff they composed with magenta. And usually they're more effective if they've layered their own stuff on top of it or they've, you

know, taking time offline rather than in performance to generate. But some stuff's actually quite good. I think it's fun. It's a start. Yeah. I mean, I think it's great. And so how, like, you compared it to, you know, the work you did in 2002. Where is LSTM gone since then?

Like, you know, you talk about like, you ended up doing this project. I saw in your talk that because you kind of like failed at it a while ago. Failure is good. Yeah. Yeah. So the, there was a point in time. I was at, um, in a, in a, at a lab called Idzia, the Dalai Moli Institute for Artificial Intelligence,

and I was working for Juergen Schmidt-Huber, who's one of the co-authors. He was the advisor to SEP Hockrito, who did the, who did LSTM. And there was a point in time where there were three of us in a room in Mano, Switzerland, which is a suburb of Lugano, Switzerland, who were the only people in the world using LSTM. It was myself, Felix Gares, and Alex Graves. Among the three of us, by far, Alex Graves, has done the most with LSTM. So he continued after he, he finished his Ph.D. and he continued doggedly to try to understand how recurrent neural networks worked,

how to train them, and how to make them useful for sequence learning. I think more than anybody else in the world, including SEP, the person who created LSTM, you know, Alex just stuck with it and finally started to get wins, you know, in speech and language, right? And I more or less put down LSTMs. I started working with audio stuff and other more like cognitively driven music stuff at University of Montreal. grow. But like, it worked finally, right? And it was, you know, like there's this thing in music, a 20-year overnight success, right? Somebody else's like, this worked because he stuck with it. And now, of course, it's, you know, become like sort of the touchstone for, you know, recurrent

models in time series analysis. It forms, some version of it forms the core of what we're doing with translation. I mean, these models have changed, right? They've evolved over time. But basically, you know, recurrent neural networks as a family of models. is around because because of that effort of like it's interesting right it's really there really were three of us and felix went on with his life and um i went on with my life and alick stuck with this kind of really one person carrying forward but you may get letters from people saying hey wait you forgot about me you forgot about me this is this is a little bit reductionist obviously there were more but it felt that way at the time right what was the breakthrough then that like got people interested

I think it was the same breakthrough that got people interested in deep neural networks and convolutional neural networks. It's that these models don't work that well with small training sets and small models. So like you mentioned that. Yeah, that they're data absorptive and meaning that they can absorb lots of data if they have it. And, you know, neural networks as a class are really good with high dimensional data. And so as machines got faster and memory got bigger, they started to work. So, you know, we were working with really small machines and, like, working with LSTM networks that were, you know, maybe had like 50 to 100 hidden units and then a couple of gates to control them and trying things that had to do with like the dynamics of how these things can count and how they can like follow time series. So you try to scale that to speech or you try to scale that to, you know, speech recognition was one of the first things.

This is really hard to do. So I think a lot of this is just due to having faster machines and more memory. It's kind of weird, right? It's surprising that that would be it. Yeah, I think it surprises everybody a little bit. Yeah. Now the running joke, like having coffee here at Brain is sort of like, like, what other technology from the 80s should we rescue? You know, it's just like, I mean, exactly right.

AI's back. Exactly right. How far have you pushed LSTM? Like, you know, obviously there's some amount of like text generation that people are trying out. You know, have you let it create an entire song? No, we haven't because we haven't. got the conditional part of it right yet. So I think, like LSTM in its most vanilla form, I think everybody's pretty convinced that it's not going to handle really long time scale

hierarchical patterning. And I'd love it if someone comes along and says, no, you don't need anything but vanilla LSTM to do this. But, you know, I think what makes music interesting over, you know, even after like five seconds or 10 seconds, is this idea that, you know, you're getting repetition, you're getting, you know, harmonic shifts like chord changes. there's a there there right and um one way to talk about that there there is that you know you have some lower level pattern generation going on but there's some conditioning happening oh now continue generating but the condition shifted we just shifted cords for example and so um i think if we start talking about conditional models if we talk about models that are explicitly hierarchical um if we

talk about models that we can sample from in different ways we can start to get somewhere but i think um you know only a recurrent neural network is is is is, you know, it's, it's not, it's, it would be reductionist to stay that it's the whole answer and it's in fact true. It's not the whole answer. I was thinking about how you were, was it the TensorFlow or the IOT talk where you're talking about Bach? Oh, probably, oh, that we did stuff that was like more Bach than Bach. Yeah, we nailed it. Yeah, that's great. Like, you start making things that like are, you know, more palatable. It's like, I'll make the best Picasso painting for you. Sure. But it's not necessarily a Picasso painting because it's not necessarily saying it. anything.

Precisely. Right. So I think by analogy, so first, in case it's not clear, I don't believe that we made something that was better than Bob. But when we put these tunes out for like untrained listeners to listen to, they sometimes voted them as sounding more bocky. And I think it's, imagine like what these models are learning, right? They're learning kind of the principal axes of variance.

They're learning what's most important. They have to, right? Because they have a limited memory. They're compressed, right? So if you sample from sketch R&N with very low temperature, meaning without a lot of noise in the system, you actually get what, like, if you want to squint your eyes and break philosophy is like the platonic cat, you know, you get a cat that looks more like a cat anyone to draw, sort of the average cat. And I think that's sort of what we're getting from these time series models as well.

They're kind of giving you something that's more a caricature than a sample, right? So then in the creation of art, like what are you predicting is going to happen, like as Magenta progresses? I think in, can I make predictions that are on the time frame of like 28 to 40 years? No one will ever test. In a thousand years, Magenta is going to be the only, no. Joking aside, I do believe that the machine learning and AI will continue, like will become part of the toolkit for communicating and for expression, including art.

I think that in the same way, I think that it's healthy for me to admit that those of us who are doing this engineering won't, almost by definition, know where it's going to go. We can't and we shouldn't know where it's going to go. I think our job is to build smart, you know, AI smart tools. At the same time, I want to point out, like, some people find that answer boring, like it's hedging. but I do think there are directions. I can imagine directions we could go on that would be really cool. For example, thinking of literature, right?

I think plot is really interesting in stories and that you can imagine that we have a particular way as humans, like the kind of cognitive constraints that we have, like kind of limitations and how we would draw plots out, you know, as an author. And you're not going to do it in one past, left. to write, like, in a recurrent neural network, it's going to be, like, sketching out the plot, and do we kill this character off? But I kind of can imagine that generative models might be able to generate plots that are,

that are really, really difficult for people to generate, but still make sense to us as readers, right? Like, you know, it's, think of it. Think of it. If you flip it around, like, I think jokes are hard because it's really hard to generate the surprising turns and the, the, like, kind of like, you go, in one direction and you land over here, but it still makes sense. And I can imagine that the right kind of language model might be able to generate jokes that are super, super funny to us and that actually

might have a flavor to them of being like, yeah, I know, this joke must have been machine generated because it fits in so many different ways, right? Right? It like this high, it like fits in so many different ways in a math way, like in a high dimensional space, but it's super funny to us. Like, like, I don't know how to do that. But I can totally imagine that we would be in a world where we get that. I thought about it in the complete opposite way, but that makes sense. I was thinking about it, you know, training it to create pulp fiction. Like that'd be so simple in my mind. Like, you know, just create these like airport novels. It can just like bang out the plots. I mean, that's probably where we'll start. I mean, I would love it if we could write. So, so everybody understands

that's listening or watching, you know, we can, we can't generate a coherent paragraph. Right. So, I don't mean, we magenta. I mean, kind of we humanity. He's like, I can't write at all. It's really hard. And it all hits, I think it all hits at structure at some level. Like, you know, nested structure, whether it's music or I think there's like, like, art plays with geometry or color or something else. You know, it's, it's meaning. It's, you know, it's nested structure somewhere.

And has the art world or, you know, I guess any kind of artist, any kind of creator, have people push back in the way that they're scared? You know, I imagine when photography came out, everyone was pushing back saying like this, this might end painting. because it's about, you know, photography captures the essence. But then it ended up changing because people realized that that painting wasn't just about capturing something, you know, capturing an exact moment. Certainly, the generative art world, and we've seen lots of that.

Another researcher in London, someone posted on his Facebook something like, he posted to us a tweet that was like, what you're doing is bad for humanity. It's like, really? Like he's making like new folk songs, right? It's like generating folk songs with an LSDM. This is Bob's term. I'm like, it's probably not bad for humanity. So yeah, of course.

But like what I love about that is, you know, it's okay if a bunch of people don't like it. And in fact, if it's interesting, what art does everybody like? Zero. Right. Or it's really boring, right? Right. So you have this idea that like if you want to really engage with people, you're probably going to find an audience. That audience is going to be some slice.

Frankly, it's probably going to be some slice coming up. from the next generation of people that have experienced technology that are taking some things for granted that are still novel to someone like myself, right? You know, and, but it's, you know, it's okay if a bunch of people don't like it. Yeah. Well, when we were talking before, I was, I was surprised that you hadn't gotten more pushback. It seems to be like most people in our world are just like, oh, okay, whatever. It's like, do your thing. It's kind of opening up new territory rather than it is, like, you know, challenging. I think that I've gotten, I've gotten pushback in terms of questions. I think we have, and I say, I think this is as a community in Google and outside of Google and outside

of Magenta. I think people are really clear that what's interesting about a project like this is that it be a tool, not a replacement. It's, and I, I thought, I think more, I think if we, if we presented this as, you know, push this button and you'll get your finished music, it would be a very different story, but that's kind of boring. I think it's funny. It's funny you mentioned Hacker News because I was talking with one of the moderators. We love you, Hackerno. Be nice to us. They're great. It's just impersonal. It's so easy to critique people. But I was talking with Scott, one of the moderators. And he was wondering if you guys were concerned with the actual like cathartic feeling of creating music or if that's just something you don't even consider right now. I mean, as people, we have course. You have to. Yeah. Yeah. And I think there's a couple.

of levels there. I think you lose that if what you're just doing is pushing a button. And like, so I think this is everywhere. The drum machine is such a great, a great thing to fall back on. It is just not fun to just like push the button, like, and make the drum machine do its canned patterns. And I think that was the goal. My sense of the reading that I've done is this will make it really easy, right? But like, what makes the drum machine interesting is people working with it, writing their own, you know, writing their own loops or their own patterns, changing it working against it, working with it. And so, you know, I think this project loses its interest if we don't have people getting that cathartic release, which I believe me, I understand

what you mean. That's the thing. One, the other thing I mention is, if there's anything that we're not getting that I wish we were getting more of is creative, people coding creatively, right? We talk about creative coding in it like this kind of hand-wavy sense, but like, like, I would love to have the right kind of mix of models in Magenta and in open source linking to other projects that you as a coder could come in and actually say, I'm going to code for the evening and add some things. I'm going to maybe retrain. Maybe I'm going to hack data. And I'm going to get the effect that I want. And that part of what you're doing is being an artist by coding. And I think we haven't hit that yet in Magenta. I'd love to like get feedback from whomever,

like in terms of ways to get there. The point is there's a certain catharsis. for those of us that train the model, you get the model to train. It works. It's like, it's fine. You'll be bored if you just push the button, but it feels good for me to push that button because I'm the one that made that button work, you know? So there's that, right? It's a creative act in its own, right?

And have people been, like, creatively breaking the code? Like, oh, it would be funny if it did this or interesting if it did that. A few, though. I think our code is so easy. Like, most open source projects need to be rewritten a couple times. And I think, you know, we've gone through. we're on our second rewrite, is that if the code is brittle enough that it's easy to break uncreatively, then it's hard to also break it creatively. And listen, I'm being pretty critical.

I'm actually, I'm really proud of the quality of the Magenta Open Source effort. I actually think we have, you know, well-tested, well-thought-out code. I think it's just a really hard problem to do coding for art and music. And that, you know, if you get it wrong a little bit, it's just wrong enough that you have to fix it. And so, you know, we still have a lot of work to do. So then where does that like creative coder world go? I've seen a lot of people that are concerned with like even just preserving. I think Rhizome is doing the preserving digital art project. What direction do you think that's going to go in?

Presumably a number of cool directions in parallel. The one that the one that interests me personally the most is reinforcement learning. And this idea that, you know, models train. So there's a long story or a short story. Which one do you want? Long for sure. Yeah, okay. Well, how long.

It's not that bad. So, you know, generative models 101, you start generating from a model trained just to be able to regenerate the data as trained on. You tend to get output that's blurry, right, or is just kind of wandering. And that's because all the model learns to do is kind of sit somewhere on the big, imagine the distribution as a mountain range and it just sits on the high mountain top. Kind of plays it safe. Yeah, kind of plays it safe. All t-shirts are gray. if you're colorizing because that's safe, you're not going to get punished.

And, you know, one revolution that came along, thanks to Ian Goodfellow, is this idea of a generative adversarial network that is a different, it's a different cost for the model to minimize where the model is actually trying to create counterfeits and it's forced to not just play it safe, right? I don't know how, this is too technical. It's very interesting to me. Yeah, this was part of the talk, right, where you like cut out the square. Yeah, yeah, exactly.

Yeah, I saw that part. So another way to do the, is to use reinforcement learning. And it's slower to train because all you have is a single number, scale or reward instead of this whole gradient flowing than GANS, but it also is more flexible. Okay. So my story here is that the GANS are part of a larger family of models that are at some

level critical. Everybody needs a critic. And they're pushing back and they're pushing you off of your, pushing you out of your safe spot, whatever that safe spot is. And that's helping you be able to do a better job of generating. We have a particular idea that you can, can use reinforcement learning to provide reward for following a certain set of rules or a certain set of heuristics. And this is normally like, if you mention rules at a machine learning dinner

party, everybody looks at you funny, right? Like you're stepping backward. Yeah, you're not supposed to use rules. Come on. We don't use rules. But instead of building the rules into the model, like the, the AI is not rules. The machine learning is not rules. The rules are out there in the world, and you get rewarded for following them. And we had, I thought, some very nice generated samples of music that were pretty boring with the LSTM network. But then the LSTM network trained additionally using a kind of reinforcement learning called deep Q learning to follow some of these rules. The generation got way different and way better.

And specifically got catchier. What were the rules? The rules were like rules of composition for Counterpoint from the 1800s. They were super simple. Now, we don't care about those rules. Okay. But there's a really nice creative coding aspect, which is, think of it this way, I have a ton of data.

I have a model that's trained. I have a generative model, whatever it may be. It may be one trained to draw. It may be one trained for music. And that model has kind of tried to disentangle all the sources of variance that are sitting in this data. And so it's smartly generated, you know, it can generate new things. But now, now think like, as long as I can write a bit of code that takes a sample from the model and evaluates it, providing scale or reward, anything I stuff in that evaluator, then I can get, like,

the generator to try to do a better job of generating stuff that makes that evaluator happy. It doesn't have to be 18th century rules of counterpoint, right? So you could imagine, like, taking something like sketch R&N and adding a reinforcement learning model that says, I really hate straight lines. And suddenly, the model is going to try to learn to draw cats. but without straight lines. The data is telling it to draw cats. Sometimes the cats have triangular ears with straight lines,

but the model is going to get rewarded for trying to draw those cats that it can without drawing straight lines. And straight lines was just one constraint that I picked off the top of my head. It has to be a constraint that you can measure in the output of the model. But like musically speaking,

if I could come up with an evaluator that described what I meant in my mind by shimmery, really fast-changing, small local changes, I should be able to get like a kind music that sounds shimmery by adding that reward to an existing model. And furthermore, the model still retains like the kind of nice realness that it gets from being trained on data. I'm not trying to come up with a rule to generate shimmery. I'm trying to come up with a rule that rewards a model for generating something. Yeah. It's very different, right? Yeah.

So I think that's one really interesting direction to go in is like opening up the ability. If you can generate scalar reward and drop it in this box over here and we'll take a model that's already trained on data and we'll tilt it to do what you want it to do. That kind of underlies a fear that people have, right? Which is like what happens when you can create the best pop song? And what do people do? And do you have thoughts on like, A, is that possible? And B, what would the world look like if that world comes to be? I think that it is, I had an algorithm for doing this, which is, which is the best pop song for me, which is when we used to sell U's CDs, it was usually like a two to one. Okay.

So every time, if you have a thousand CDs, you trade them in and you have 500 that you like better. And you just kind of keep going, right? You finally get that one, yeah, exactly, hill climbing in that space. Yeah, I think that what I, so I'm not sure. Like, part of me wants to say people love the kind of rawness and the variety of things that aren't kind of predictable pop. but let's face it. Like, people love pop music of even the, like, there's a kind of pop music that you'll catch on the radio sometimes that isn't like, like, almost like, most of your listeners are probably in the same camp or viewers. Like, like, there's pop that we love.

Like, you know, like, I love the popiest of Frank Ocean's music. I can listen to it forever. But then there's like, just like, there's like the gutter of pop. Yeah. And so, you know, I mean, you can't even distinguish who the artist is, but like they play at the big festival. So I guess that kind of unasked the perfect pop. I mean, pop is such a broad. the thing.

Yeah. But yeah, I think, like, I can imagine that with machine learning and AI at the table, we will, here's another way to look at it. Like, some things that used to be hard will be easy, right? And so we'll offload all of that. And if people are happy just listening to the stuff that's now easy, then, yeah, it's a problem solved and we'll be able to generate lots of it. But then what people tend to do is go look for something else hard, right?

It's like the drum machine argument. So you've solved the metronomic. You've solved the metronomic, you know, beat problem. And then what you actually find is that artists who are really good at this, they play off of it and they're allowed like when they sing to do many more like rhythmical things than they could do before because now they have this, the scaffolding they didn't have to work with before. They just constantly break it, right?

As soon as you destroyed the electric guitar. But I hope that's an honest answer to your question. I mean, your question was a different flavor. It's like, hey, are we really moving towards a world where we're going to generate the perfect pop song? Yeah, I don't know. I don't think so. I mean, I don't feel like that's going to happen. But, you know, maybe it happens so quickly.

And then as soon as we realize like, okay, this is how we're going to break it. This is how we're going to retrain ourselves. Yeah. It can learn so fast that it's like, okay, now I can do that too. Yeah, that's nice. But then I can do that too. So then what I was wondering is there like, in, you know, the next like handful of years, is there like a holy grail that you're working toward for a majority?

gentle like, okay, now we've hit it. Like, this is the benchmark that we're going for. There are a couple of, there are a couple of things I'd love to do. I think, I think composing, creating long form pieces. Whether they're, whether they're music or art, I think is something we want to do. And this hints at this idea of like not just having these things that makes sense at like 20 seconds of music time, but actually say something more. That direction is really interesting because I think that not only, so let's face it, that would be at least more interesting if you push the button and listen to it. But also, this leads to, like, tools where composers can offload very interesting things.

Like, some people, some people, I'm one of these people. I'm really obsessed with, like, expressive timing. I'm really obsessed with musical texture. Okay. I don't know what that is. Oh, no, I just mean, like, let's say you're playing the piano. Oh, I saw it in the gray art space or the gray talk. You were contrasting the piano played by a computer.

Did your homework. Yeah, exactly. I just watched a bunch of YouTube videos. So, you know, if you listen to someone play waltz, it'll have a little little to it. Or like some of my favorite musicians like Thelonious Monk, if you're familiar, if you're not familiar with Thelonius Monk, homework, go listen to him. He played piano with a very, very specific style that almost sounded lurching sometimes. It was very, he really cared about time in a way.

And so, like, if you, if the way that you're thinking about music and composition is really, caring about kind of local stuff, right? it would be very, very interesting if you had a model that would handle for you some of the decisions that you would make for longer time scale things like when do chord changes happen, right? So like usually it's the other way around. You have these AI, you know, machine learning models can handle local texture, but you have to have to decide that. So, yeah, my point is if we get to models that can handle longer structure and nested

structure, we'll have a lot more ways in which we can decide what we want to work on versus what we have the machine help us with, right? And has this now, has it affected your creative work? Or do you still do? Like, creative, like, composition? Yeah. So I'm working here at Google. And, you know, this is like a coal mine of work to do this project magenta like every day. No, joking aside. Yeah, as we're good. Yeah, plus two kids. Plus two kids. Yeah, no, I basically, I've been using music as a, as more of like a catharsis relaxation thing. I don't, I don't feel like personally I've done anything recently that I would consider. like creative of a level that I want to share with someone else. It's been more like jamming with friends or like, you know, just like throwaway compositions, jamming. Like here's 10 chords that sound good. Let's jam over it for the evening and then like don't even remember it the next day.

And really trying hard to understand this creative coding thing. Like that's something else I've worked on. A lot of it's just like I'll start and then I'll get distracted. But yeah, so that's sort of the level of my creative output, I'm afraid. Well, the creative coding thing, it's seemingly, I don't know, so many people are looking for it in every venue, and it's so difficult to find people. There's just kind of like one-offs now. Yeah, I think that's right. It's so, it's so hard to have the right, I think maybe, you know, maybe we need the garage band of this. Like, you know, we need to have like something that's so well put together that it makes it easy for a whole generation of people to jump in and try this, even if they haven't had, you know, like four or five years of Python experience or something like that.

I didn't know if that's what you were alluding to when you were saying that, you know, command line, obviously not the way to do it where it dumps MIDI files. But now it's an API, right? And like what is the next step that's very obvious? Try to make it more usable and more expressive. Expressivity is hard in an API, right? You know, it's like so hard to get it right. And I think it's almost always multiple passes.

So we've got, I think the API, the core API that allows us to move music around in real time in MIDI and actually have a meaningful conversation between an AI model and multiple musicians. is there. And there's just a bunch more thinking that needs to happen right to get it right. Cool. So if someone wants to become a creative coder or wants to learn more about you guys, what would you advise them to check out? So I would say the call to action for us is to visit our site. The shortest URL is g.co slash magenta. It's also magenta.com. We can link it all up to you. And like have a look at, we have some open issues. We have we have a bunch of code that you can like install in your computer and I hope you can make work

and maybe you will be able to. And you know, we, you know, we want feedback. We have a pretty active and we certainly follow our discussion list closely and our game for philosophical discussions and our game for technical discussions. And, you know, beyond that, we're just, we're just kind of keeping rolling. Like we're just going to try to keep doing research and keep trying to build this community. Okay. Great. Thanks, man. Sure. It's fun. all right thanks for listening so if you want to get started using magenta you can check out magenta dot tensorflow.org and if you want to watch the video which we filmed in one of google's very swanky libraries you can check out blog.waccommodator.com. Okay see you next time.