Future of Coding - Elephant in the Room

Starting point is 00:00:00 Arr, this episode of the future of coding contains tastelessly self-referential humor, self-censorship, uncensored swearing, and it's like three hours long. Listener indiscretion is advised. Now, I wrote these summaries, including some of the notes for things like, Ivan is going to say this at this spot. And I wrote this so that it'd be like i'm reading this so i say like brett says and then quote marks so it's not a summary that's meant for you two to read it's just uh sure sure sure yeah but now we can predict what you're gonna say ahead of time and preempt you by saying the opposite.

Starting point is 00:00:47 This is fun because we can compare. Like I have a summary of Dead Fish. I mean, I guess it's kind of in the title, but I'm sure the title of this episode doesn't say enough of what we're going to do. And I'm saying we have to jump in right now. But yeah, because I still have to blow my nose. Oh, yeah. Yes, we are doing Brett Victor and we're doing three videos. We're doing future programming, inventing on principle, and stop drawing dead fish.

Starting point is 00:01:09 I think the title should be The Elephant in the Room. Because Brett Victor has been the elephant in the room since the beginning. Stop inventing dead elephants. Got it. Stop inventing 2,000 dead elephants. No, elephant 2,000 we definitely have to do at some point. Yeah. I have, I'm very, I know this is not on any topic ever,

Starting point is 00:01:36 but I've been, we wanted to do videos. I know Ivan wanted to do videos for a while. I guess it is on topic. Ivan wanted to do videos for a while or something that was not a text for a while. Oh. Yeah, yeah. Yeah. I thought you meant like release these episodes as video.

Starting point is 00:01:53 No, no, not release these episodes, but do something that was not text for a while. And I've been like hating papers for the last month. You what? Like every paper I read, I would just hate read them. No, that's not nice. You're supposed to be the one who likes papers. Yeah, I know. papers for the last month you what like every paper i read i would just hate read them but no that's not nice you're supposed to be the one who likes papers yeah i know and i but like luckily having to do these videos instead have made me just like love papers again you troll and so like i have so many papers i want to do now and it's it's been great i was also just in

Starting point is 00:02:25 like a hate mood i think that's why i got covid that's what i've learned from my extensive internet research you know you get covid by hating things they're gonna put a little banner under our podcast jimmy itunes is gonna say you know spreads misinformation oh no covid is caused by the hate in your heart yes yes wait are we uh have we started um no yeah see i that's the fun part right is if you just we're recording so no matter what you say here what is, you're playing a game to figure out where Ivan's gonna start the episode or not. Boop. All right, so we're doing three, three, count them, three Brett Victor talks in this episode. Yes.

Starting point is 00:03:18 What could possibly go wrong? And I think my summary takes about 30 minutes to read. I time myself reading about half of it. What? Yeah, your summary is quite a lot. Having looked at it, I'm conflicted on how we do this series of videos. Partially because if you haven't seen these talks, I do think you should go watch them not listen to us talk about them yes yeah at the same time it's like well if you've watched them what new are we gonna

Starting point is 00:03:51 add well okay hold on wait a minute hang on a second like i mean part of yes you could listen you could listen to these episodes and if you've the paper, and I'm sure some people do. Read the paper? I'm talking about in other episodes. Oh, okay. Other episodes. Okay, okay. Right? But part of what people are getting is, you know,

Starting point is 00:04:12 we're the cliff notes for them. Like, we're doing their homework that they didn't have to do. Whereas here, we're commenting on things that I'm betting a large portion of our audience have watched. And if you haven't watched these Brett Victor talks, you probably should. Yeah, go do that. Like, and haven't watched these Brett Victor talks, you probably should. Yeah, go do that. And if you're commuting right now,

Starting point is 00:04:28 you're not going to, and that's fine. I've done the same thing. And you can listen anyways. But the talks are a better medium for conveying what they want to convey than us commenting on them. So us trying to convey their information seems like the wrong answer here. Whereas when we're doing papers, that's what we're trying to convey their information seems like the wrong answer here whereas when

Starting point is 00:04:46 we're doing papers that's what we're trying to do is convey the information the paper's talking about while also commenting on it let me flip that around on you what did we all get out of watching these videos because i watched these videos for the first time in in some cases a decade and got a ton out of them since in that last decade you know i've i've very much changed my relationship with this field to the point where i'm now actually doing more of this kind of work and you know there's the benefit of a decade of the industry living sort of in the shadow of these talks, if that's not being too bold, and changing its ways in reaction to them. And there's the recent surge of AI. So there's like a ton of stuff

Starting point is 00:05:32 that is now newly existent and relevant, that changed how I view these talks and how I appreciate them. So even without somebody needing to rewatch the the talks which i mean if i would encourage you to do that even if you've already seen them i think that we will have no shortage of interesting things to say about them uh please don't unsubscribe there is another thing is that ivan's summaries are really long we could just do a shorter summary you know it's not out of the question maybe we could try it for one of them and and see how it goes you'll note that the um the the inventing on principle summary is literally half this document and then by the time we get to dead fish it's like actually sorry more than half dead fish and uh future programming are much shorter

Starting point is 00:06:25 so yeah i mean i think that's the correct thing although i don't know about the order the order question i think is like do you really want to backload the episode with good content or front load the episode with good content i mean i just did them in chronological order so it's sort of like like dead fish definitely builds on principle i think hey here's another thing right brett victor is the elephant in the room on this podcast right because that's what kicked off the podcast when steve krauss started yeah and you guys won't wouldn't shut up about him and it's time you told the world what you think, you know? Yeah, yeah. Me included. And the other thing is, obviously, Brett Victor is like this Jesus figure of the future of coding communities, you know, with a small F and a small C. And, you know, he's universally loved, but he's also, people have always have something to say about him, right? They're like, oh, I like Brett Victor, but I don't agree with him on this. And I've seen so many people and podcasts and chats and Blair give their own criticism of Brett

Starting point is 00:07:30 Victor. And in my opinion, they all do it wrong. So I think we should join this tradition and do our own criticisms that are entirely wrong as well. It's like a rite of passage. So what is, all right, hold on, hold on, hold on. What's the criticism that other people have given that if you can remember any, that is wrong? Spoilers. I don't want to jump ahead, but... Okay, well, we've got to do this though because now I don't care about us criticizing Brett Victor.

Starting point is 00:07:55 I care about us criticizing Brett Victor criticism. Yeah, that's why I'm here. All right, great. Cool. So you just want me to pull up Hacker News comments and we'll just read them for the next three hours? We just need to get it out. What do you think?

Starting point is 00:08:08 Do I think? Oh, this is going to take longer than I thought. Yeah, so I will say, I didn't get a whole lot re-listening to these episodes. Episodes. These talks. Re-watching these talks. Re-listening.

Starting point is 00:08:24 Because I mostly listened uh-huh because because we have to translate from a visual media to an auditory one and i thought it would be better for me to experience it the way i already know what what the visuals are i mostly listened i did watch but i mostly listened i listened like three or four times to each talk. Oh, okay. That's better than me. That's a lot of times. Yeah.

Starting point is 00:08:50 I did two-ish. Wow. Okay. I almost watched all of them. I just made a fake summary. I'm not kidding. I have a whisper app on my Mac that I can just invoke with a keyboard shortcut

Starting point is 00:09:07 so I can like talk to my computer and make it do stuff that's weird and it's good enough that it can hear my TV playing Brett Victor from across the room when I invoke that so that was very helpful for getting a lot of these quoted sections it's just like yeah Whisper

Starting point is 00:09:23 listen to that video over there and just are you kidding me i'm 100 serious there i was putting in my effort you know people have already made a transcript for inventing on principle and uh future programming it it is and i'm i'm actually okay jokes have left the room it is cool to be able to go multimodal and say when it comes to like describing this moment in the talk i want to type that but when it comes to doing a quote i just want to pull the words out of the air and put them into the text and i found i was not tempted to speak the summaries because i wanted to write them a little bit more carefully and i was listening to the talk while writing the summary.

Starting point is 00:10:05 So it's like being able to flip between these different modes was really empowering in a way that I'm not used to, because I'm not used to having a keyboard shortcut that just says, turn the words you hear into text. So what you're saying is you were able to have a direct connection to the things you were making oh no and get live feedback we don't need a segue jimmy we're already in we don't have opening banter this episode we're already in the main thing all right so uh should we start there should we start with inventing on

Starting point is 00:10:37 principle yeah go on i guess we can start with the best yeah let's yes well yes well start with the most important um the best yeah okay i watched this talk yes over a decade ago and i do think that this talk was very influential on me getting into this space and to to all of this and so i i'm betting that's a lot of people's origin story in this community who've been in it for a while was inventing on principle. I know that it's called Future of Coding, so Future of Programming, I think, was the talk that really inspired it. But inventing on principle to me was kind of this first talk that showed the world that programming could be a bit different than what we had made it yeah and i i think that these demos are really interesting but i do think a lot of the discourse on this talk has like lost the big picture and that's what i hope that we we focus on is what brett's real point of this talk was not the

Starting point is 00:11:37 programming stuff yeah like my summary starts this talk is about finding a guiding principle for your work and that's how he opens the talk and he says that repeatedly throughout the talk is like the point is you need to motivate your work with a principle i'm following a principle and so that's what lets me do all these cool demos but you need to go find a principle of your own well that's that's not what he says no that's not what he says he says you can find a principle and this is a way of living your life yeah yeah you don't have to it's the right way the best the cool way yeah but i don't know you do you i guess i don't care if you do the wrong thing so so so you all both took this even though you know brett victor doesn't say it that way as like this is this is the way yeah do

Starting point is 00:12:26 you have a principle that you live okay i guess we should state what he actually is saying right yeah the point is that there's an a different way to live your life if you're going to be a technologist uh you know programmer designer i designer. I don't know the right word here. Already wrong. An inventor, a creator. Getting closer, getting warmer. Any field, any person, if you're going to have a life's work, you should have a principle that motivates your life's work rather than something like, it's important to me to make sure this obscure JavaScript engine is not left to the dustbin of history.

Starting point is 00:13:04 I'm going to take a personal crusade to keep this javascript engine alive that's like a very narrow like specific thing whereas having a principle is a thing that does not dictate a particular course of action it just says this is a change that i want to see in the world. Damn, AI-empowered Ivan coming up with the quotes. You're ready. I don't like this. My point is that we often think of technology as something where what you decide to do is mastery, etc. This is what Brett talks about.

Starting point is 00:13:39 The way that you live your life if you're in technology is to become a master at something or solve some certain problems and he's saying there's a different way that you can create change the world through inventing that you can make the world a better place through inventing and it's not just an opportunity it's not that like oh you made it because you made this cool invention but you saw something wrong in the world and you're fixing that wrong through inventing rather than through advocating or organizing or some other social thing it's through creation that you're fixing the ills of the world yeah that's why i want to talk about as a technologist, of course, we already know that if you're

Starting point is 00:14:27 a social activist, you can make the world better through some principle, but we haven't talked about the idea that people who invent can also do the same thing. And so, Brett wants to say that this is the third way, or this is like

Starting point is 00:14:44 another way of living your life that you don't often hear about if you're in technology, is dedicating your life to fixing something. And he then adds on another thing of how do you really do this well? You find some principle that you hold on to, and that is what you focus on trying to fix. And so for Brett. The principle is that. Creators should be directly connected. With their creations. Yeah. And so this whole talk.

Starting point is 00:15:14 Is both Brett telling us. Another way you can live your life. But also him demonstrating. How he goes about. His principle. Which is that there should be a direct connection. To your creation. but also him demonstrating how he goes about his principle, which is that there should be a direct connection to your creation. Yeah, I can't disagree with that. Okay.

Starting point is 00:15:32 Can I introduce a framing? A framing? I don't know if that's the right word. Can I introduce a way that I look at Brett Victor, like the man on the whole and all his work? I use my eyeballs usually. Right, yeah. Like the man on the whole and all his work. I use my eyeballs usually. Right. So I think about it in three ways, right? I can admire Brett Victor's work from just how he presents stuff incredibly well.

Starting point is 00:16:01 Like he's this exceptional public speaker. His timing is weirdly delicate and good. And his demos are just like perfectly paced etc and i take a lot of inspiration from that just presenting full stop doesn't matter what it is and there's i think there's two other areas that i look at when i'm reflecting on this work there's what bre Brett Victor says about coding, inventions, technology, right? Like how to make that better. And then there's like this third stream, which is what Brett Victor says and states about art and creativity and creation and making things and exploring. And I think, I think it's really helpful to see what he's saying in just one of these three tracks at the same time, right? So like, I think inventing on principle, there's a little bit of a crossover, right?

Starting point is 00:17:00 Because he's sort of giving his opinion and advice about, hey, if you want to do invention better, if you want to do technology better, then take a principle. Right. That's like one statement that he's putting out. And then he also has another statement in this talk. It's kind of more like an example though, right? Maybe it's not the main point of the talk, but it is definitely a point of the talk that immediate connection to what you're creating is important. And I find myself agreeing with those two different things in different quantities. And so, Ivan, earlier you asked me about like what people get wrong when they're criticizing Brett Victor. It's that they can't separate these two things, right can you can agree with one part and disagree with another

Starting point is 00:17:49 part and and often people say ah well i don't like what he has to say about technology because i disagree with him on what he says about art or i disagree with with him on art because you know so like i don't know i think it's i think it's helpful i think it's especially helpful for dead fish which we'll get to later yeah that that is going to be i think we will have a lot of things to say about that one in particular yeah so funny i have nothing to say about that one i'm i i knew i i actually intentionally did not even make notes because I knew that was going to be a very Ivan and Lou-led discussion there. So inventing on principle starts with this point of, like, I'm going to tell you a way you can live your life. I will be very clear that Brett does not say this is the only way.

Starting point is 00:18:39 Yes. Brett does not say that you must do this, that if you're not doing it, you're wrong, or that it's not as good of a life or whatever. He just offers it as a way to live, a way that you might not have been told by your career counselor in college. Yeah.

Starting point is 00:18:54 Right? It's true. No one's going to tell you in technology to start your own personal crusade, is what he says, but he's going to. Ivan's considering whether he's going to interrupt all right sure you really think he's do you think he's making a stronger point by

Starting point is 00:19:10 like yeah he doesn't say it explicitly but that's what he means i just don't know why this distinction matters i just don't know why oh it really matters because i'm gonna i'm gonna disagree with oh okay great yeah if you're if you're not just being like well ivan presented it as though there was no wiggle room and and in fact, there's lots of wiggle room. No, we have to ask the question, is this actually the right answer? Cool, okay, ask that question. Okay, yeah, we have to get there. We can't sprinkle our own thoughts. We gotta talk about the talk. You have all this, like, I'm trying to get us back into the talk. This is the problem with talks versus text. I can't just be like go to paragraph two and let's read it i gotta i have time stamps all over the summer but i can't get us all on the same page and then start reading things all right i i have it right here all right

Starting point is 00:19:55 i'm gonna start taking us through the summary so he does a series of demos he has five demos each of them showing a different thing that he made uh in an instance of him kind of living by this principle this like he'll be doing something and he'll find oh here's a case where i'm you know not being given an immediate connection with my work how could i change this bit of software that i'm using or whatever so that i have a more immediate connection because i'm trying to do something and so the first demo is this sort of, and I'm going to use modern terminology to describe this. I'm not going to try and take us back in time, but it's a live coding editor that he made where there's a sort of a JavaScript script on the right half and on

Starting point is 00:20:41 the left half, there's a live updating canvas and you can type some code in the javascript and the canvas re-renders immediately um which which was pretty eye-popping at the time that was not a really common site you know 10 years ago or whatever when this talk came out and he shows that you know not only is this a live updating code editor where you can change the code and the and the drawing updates but the code has some nice little affordances some little augmentations added into it like if you have a number literal you can grab that with the mouse and scrub it back and forth to see how you know the drawing changes and you can just stay looking at the drawing and scrub the number and not care at all about what the concrete values are and just dial it in until it satisfies you creatively and you know compare that with the state of the art at the time and what is still

Starting point is 00:21:33 kind of the common practice of you change a number you hit compile and run you wait for x code to you know do its thing and copy symbols over and 30 seconds later you get to see that one change on your device or what have you in the simulator rinse and repeat um this sort of like immediate feedback is is so immediate that it's like it gives him an idea for an animation just playing with one of these values like oh i'm wiggling this value and it's updating so fast that makes me think of something that i would want to animate And it's a sort of idea that can only occur because this connection is so immediate. Because he's working in this new and different way,

Starting point is 00:22:14 it's enabling him to have thoughts and reactions with his work that just otherwise wouldn't be possible. This demo, I still think it is like one of the best demos for live coding ever. I don't know if there's something that's actually matched this. I'm going to put it out there and say, this is the best live coding demo so far. I think, you know, some things, like you said said some things we do have now like this live live updates right but some things we don't like being able to inspect the canvas and then see which bit of code is responsible like i i crave that i create i've been working with shaders this

Starting point is 00:22:59 week and i crave that so bad and sure i could like build it myself from scratch each time but like it it's not the norm you know it's not the norm and and i i would i would love just yeah more people to look at this again and rediscover it and see what i don't know debugging a 2d canvas could be like and playing with a 2d canvas yeah and i think that backwards going from the drawing to the code is one of the harder things to to make yeah right i think we do kind of uh you know we see a flashy demo and kind of think oh well why can't we just automatically do that if you i mean if you try to do this in like a production environment rather than just a toy it is very difficult to get that sort of live feedback

Starting point is 00:23:45 consistently not have infinite loops going on you know i think this is like an area where like we really do need to have you know more research more engineering go into it but going backwards is one of those things that i think like nobody nobody has focused on and i think that's one of the most that was one of the most interesting parts of the demo for me, is being able to... I mean, it's like dev tools in the browser, being able to highlight it or whatever. But that doesn't go all the way back

Starting point is 00:24:13 to your JavaScript that created it. It might go to your HTML. You can kind of do it with React, but it's not quite the same. Anyways, I think this canvas, being able to do that, is really neat. And I do think that this demo set the stage perfectly for the next demo, which to me at the time when I first watched this was just mind-blowing.

Starting point is 00:24:35 And one of the things that I think is really fun with the recording is you can hear the audience reactions. I'm pretty sure someone was sitting there and just adjusting the gain uh as he was talking because if you listen to the like the original recording brett will be talking and the mic quality is fine and then all of a sudden the audience is like reacting to things and like you can hear the room noise crank up yeah and there's a bunch of static and i don't know if they're turning on a second mic or they're turning up his mic or someone posts ivan shaking his head because he knows what they did so what did they do ivan it's probably just this way that the audio was run through a mild

Starting point is 00:25:15 compressor which automatically adjusts the gain based on changing input levels um yeah okay so why can i hear the audience better as they get louder uh because they aren't close to the mic okay so there's two things happening there's like oh the audience is there and they're bleeding through a little bit but it's quiet and since the overall level perceived by the mic is lower the compressor is going to boost that gain a little bit and so you're hearing the noise floor come up as the gain goes up but then you're also hearing the audience get louder because the audience is getting louder okay uh yeah so like it's it's iron sound engineering corner

Starting point is 00:25:53 regardless it was probably an accidental property but i actually think it makes the recording much more interesting i've been at live talks that i then hear the recording of and at the live talk you could hear the audience participation and it actually made the talk better like you could hear them you know oohing and aahing laughing whatever and then you listen to the recorded talk and they don't laugh at the joke yeah yeah even though they did yeah right i have have this all the time i've i've recently re-uploaded my uh uh my splash con talk right and because the official upload i knew i knew they'd do this they they uploaded it with like worse audio quality and it's like i put it onto the timeline and i was wondering do i do i just

Starting point is 00:26:40 crank up the laugh laughter but no i know they didn't i didn't i am so close it makes the talk more fun like hearing the everyone yes it does yes you know whatever right i just enjoy it so much that's why what was so good yes oh yeah yeah yeah you're right but then you look at the recording of the future of programming by burt victor and they didn't do that and there are just periods where you can hear yeah you can hear the silence but you know that the room is laughing yeah anyway so i wanted to just add before we get back to talking about the actual talks meta um i want to say one more thing about the audio quality for inventing on principle specifically um super relevant um so i think part of what you're

Starting point is 00:27:26 hearing jimmy is also that the mic that is being used to record this talk is a weird mic like it it's some i think it's like a little bit more omni than a a mic would normally be um i and i didn't get a good enough look like maybe he's wearing a lav or something like that and that's why we hear it but it's the sound quality is very poor and that meant that uh or that that that had the effect of causing somebody to take this talk and run it through some sort of ai powered like audio clarity improving tool and there's a version of the talk with that like ai boosted audio quality on youtube and if you listen to it those boosted crowd noises get turned into speech and so you'll get the occasional weird like some old woman's voice saying like pancake potatoes and it's like when the audience laughs you hear like little weird murmuring nonsense voices in

Starting point is 00:28:29 there and it's really tasty so i would recommend people listen to this this ai inflected version of the talk because it's uh yeah it's it it delivers the goods so i am curious like this is where i this is why i wasn't sure about talks in this format do we really want to describe each of these demos the five demos like yeah i don't but i do want to talk about the first one and the second one and maybe one more okay okay that's fine the second one is braid yes exactly yeah including um everyone switches off after the second one yeah yeah what's the what's oh what crap why can't i uh david hellman yeah it's like david hellman did artwork for this or he used david hellman's artwork from braid uh some of it and i think hellman might have done another graphic or two artwork for this, or he used David Hellman's artwork from Braid, some of it. And I think Hellman might have done another graphic or two specifically for this talk.

Starting point is 00:29:29 But it's, yeah, it's Braid. So unlike that first one, you know, which I agree, when I first saw that first one, I felt this tremendous shame for having been building art tools for many years at that point that weren't nearly that good. When I saw the second one, I was like, really? That's it? Like, we have that already. That's a pretty common thing. In game dev and in, you know, contexts where you're doing bespoke tools for artists, kind of like let me you know go across time and twiddle things and look at their effects across time like not rendered in exactly the way that

Starting point is 00:30:12 he rendered it but that that idea or like uh input replay like let me do a bunch of things save the inputs to that and then reapply those inputs continuously as i change parameters that that's like um much more common. Where the first one... Yes, I think that was the opposite for me. I thought the first one was really lame. The second one was really cool. I'm talking about my first reaction.

Starting point is 00:30:37 Because I had no interest in building and making art. So, I mean, okay, let's see. I was 20 when i first saw this so like you know i had not done i had not seen people make games that have these you know input playback and all of this stuff right uh so like the me that was just so cool and like the the time travel aspect and then being able to see that i'd also not played braid um so i didn't get the reference but uh you know being able to see in real time how this you know character can hit the the spot that's so much cooler than oh i drew a tree and the code's over here to me i was like i could build that demo tomorrow yeah that was you know obviously not true but that was my my reactions you know

Starting point is 00:31:26 like i think you're both wrong right i'm gonna come out and say it i think you're listen listen it's not about the technology i think i think this is about selling an idea i i've i'm gonna keep saying this it's like a fashion show you, he's like parading this new concept of imagine a world where this was the norm and this was seamless because I know the technology for this exists and I know I can, I can make some of this. I could, people could always make this. Maybe it did exist in tools, but to have it so seamless, you know, for it to feel easy and to be demoable, it, its goal is not to make you think that that's the right thing. I think the goal of this kind of demo is to make your head start turning, you know, the cogs in your brain start going, and think, wait, wait, what if I apply this to my field? Or like, what if I

Starting point is 00:32:18 apply this to this other thing? And I think, you know, like these, these two demos seem to have done that. Like, we don't even need to guess, like so many people have, have come up to me and say, Hey Lou, have you heard of this really, have you heard of this guy? He's called, he's called Brett Victor. Right. And, and he had this, it's like a plat, it's like Mario and he jumps and then you can put him back. It's amazing. Right. And like, I think it's near a hundred people that have told me that like you can see the effect it's had you know and and i think it's been a good effect you know people want to make more live tangible things with code that's a three stars for me three out of what

Starting point is 00:33:01 wait no three three out of three oh okay i do agree it's been a really good effect. I think a lot of people, this is where their criticisms come in, though, is that the demos were too polished. That they kind of pretended as if they had built a system that could do all of this, rather than really talking about the difficulties of doing this and the downsides. I've heard people

Starting point is 00:33:25 make this criticism like it's a flashy demo with no substance because actually making a system that gives you all these properties and doesn't just suck i mean i think about like early light table right the early light table came out in response to to this work by brett victor i mean some of the demos were it i mean it was it was It was very... It said, like, inspired by inventing on principle. You can't react to my facial expressions without actually describing them. Otherwise, the listener will have no idea, and then we'll go

Starting point is 00:33:54 back to not doing video. Yes, I do that too much. Yes, I know. My bad. I will not react to your... Well, you should not make such incredulous facial expressions all the time whenever I say things. You shouldn't talk shit. You shouldn't say things that...

Starting point is 00:34:08 How am I talking shit? Oh, there we go. Thank you. He got me to say, to make something he has to censor. That's good. Yeah, that's great. Oh, wait, is that a news resolution? Oh, no, prediction.

Starting point is 00:34:22 Prediction. Yeah, achievement unlocked. Look at that. Look at that come down. Achievement jimmy all right so so did that just for me so yeah chris granger made light table yeah in response to seeing inventing on principle and trying to make some of this stuff happen live and if you actually tried to use the live ripple it was awful like i don't think chris Granger would disagree with that, because that's why it moved on to Eve. Because he would go back to that well

Starting point is 00:34:51 and keep taking cracks at it. But it was not a fun experience to try to use the live REPL, and languages weren't designed to be run that way. And I've done some experiments trying to make things that are designed to be run that way. And've done some experiments trying to make things that are designed to be run that way and i know i think it's jamie brandon who's done some talk or uh written a blog post on some of these things and like it's it's hard so this like the third demo which we don't even need to talk about because it's just like boring typical it's like learnable programming it's the line the line yeah if you if you've if you've read learnable

Starting point is 00:35:29 programming this this third demo is just the same as that it's like let's show state as code executes um oh i mean we do need to talk about it for a second which is like this is the other major criticism is that like these live programming tools work really well when you're working on a visual program like like a game or art but the binary search sucked yeah so i was about to um just say what jimmy said but jimmy said it so i know i don't need to do you think you think that demo sucked no i okay so i'll say what i actually was gonna say which is these these demos are are interesting to compare and contrast even though that's not the point of the talk remember the point of the talk is not to get into the demos it's to focus on principles and and creators connections with

Starting point is 00:36:17 their work but that criticism that it's like these are toy examples this stuff would never work in a in a production system this stuff only works because it's in a visual domain. Like, I take that as a sort of a, like, I look at that and I say, so what you're actually telling me is that since this stuff won't work in a production system, we just shouldn't do production systems. And if this stuff won't work in domains that aren't inherently visual, then we just shouldn't work in domains that aren't inherently visual then we just shouldn't work in domains that aren't inherently visual like if we can come up with breakthrough incredible ways of visualizing things and creating an interactive experience by leaning on visual affordances and then when we try and do it

Starting point is 00:36:56 with textual code it's like kind of falls flat it's it's like you can see the memory but it's not very meaningful and it's not interactive and tangible in the same way then like simple we just need to change what programming fundamentally is to make it so that we can have more of that earlier demo experience rather than the the less interesting binary search demo experience i don't know i i really like this one as well i didn't expect to be defending brett so much so hard but or text coding text coding yeah no listen okay okay hang on okay there's this criticism right that like the demos are too flashy you know the demos are too flashy you know when you actually try to do them in a real situation, it's really hard.

Starting point is 00:37:46 And I don't think that's a criticism. I think that's just outside the remit. That's outside the scope of this talk. Too bad. Deal with it. Imagine taking along your friend, your colleague, to Paris Fashion Week, and someone walks down with this incredible like you know they got like banana peel dress with like I don't know like a model of the solar system around their head and then the person to your right leans over and says it will never work in practice it'll never

Starting point is 00:38:18 sell it never sell in the supermarket I'm like yeah that's not the point that's not the point but maybe but maybe people will start thinking about oh what what different materials can we in the supermarket. And I'm like, yeah, that's not the point. That's not the point. But maybe, but maybe people will start thinking about, oh, what different materials can we use for dresses? What crazy things can we do to get the zoomers excited about headpieces? I don't know. I don't know fashion, as you might tell, right? Yeah. Or maybe, maybe just like it's possible to present your body with the affordance or with the augmentation of clothing in a way that is exciting because of what you do with color and geometry and texture and material. And there's like there's things you can do with clothing that are more than just like, you know, what's it called? Like the Alberta tuxedo or whatever, like jean pants and a jean jacket. We just call it the Canadian tuxedo.

Starting point is 00:39:07 There we go. Yeah, yeah, yeah. So this binary search demo, you know, this binary search demo, I think it's just another example that Brett Victor is showing. Look, I'm trying to apply my principle and I'm trying to apply it in all areas.

Starting point is 00:39:23 This is what I've done so far. Maybe it has limitations. Maybe it sucks. Maybe it's not as good as the first demo, but I'm just trying to do one bit at a time. And what's the goal of this demo? Like he's not trying to, again, he's not trying to sell us that this approach that I'm doing is the one you should be doing. The goal of the demo is to try to sell you on this idea that you should have a principle, and it's trying to sell you on his principle, right? Like, he's not trying to sell you the idea that this demo is the way to do it. So, sure, we can say that this demo isn't the way to do it, but hey, that's like one other thing we've

Starting point is 00:40:02 tried. You know, in one of his other talks we're talking about he talks about we need to try everything we need to try a hundred different things i've tried one great who's going to try the next one and i think i think that this talks like does that so from sort of talk delivery point of view that's a three three stars as well for me three stars code three stars talk delivery and and goals fulfilled. One star, but, you know, well, never mind. I completely agree with Lou that, you know, the criticisms about the demos I do think are misplaced because they're not trying to sell you this is the way you should program.

Starting point is 00:40:36 But I have to pick on Ivan's. Yeah, go on. Well, then we shouldn't do anything that's not visualizable and we should just fundamentally change. That's just silly. Like, there are so many things for which trying to solve the problem of how do I visualize this thing is harder than solving the problem you had on hand.

Starting point is 00:40:57 So, for example, I downloaded 100 gigabytes of Lee Chess chess data. Okay? So, I needed to get... It was Tuesday. It's a normal Tuesday for Jimmy. I'm not surprised. I'm not phased. I downloaded, and that's a very tiny, it's like one month of Lee Chess games.

Starting point is 00:41:17 Love it. And I needed to make an openings book so that my bot would play the openings differently, because this bot always plays the same moves and responds to the same thing. So the way you cheat is you make it play different moves at the beginning so you get it in unique positions. So what I wanted to do, though, was not just pick all the best openings. I wanted to pick openings of players similar to my rating on how they would play and what the likelihood of them doing it is. I could not easily visualize all 100 gigabytes of this data

Starting point is 00:41:49 and figure out where things were. And even trying to figure out how to parse through this data and get the right bits and all of that, that was the hard part initially, was trying to even make sense of this data. And if I tried to start by visualizing the hundred gigabytes, I would have just never made any progress. And so like, we can't say that like, we just focus on the things of programming that are visualizable. I'm not

Starting point is 00:42:16 saying there isn't a visualization of that data. Of course there is, but trying to solve that problem before I could solve the problem of gathering up all of the examples to make an opening book is the wrong order i think you misunderstood my criticism okay or i presented it badly yeah let's let's put it that way i'll take the blame yeah i presented my criticism badly i wasn't saying that the thing that we ought to do is like what brett does in the first two demos where he finds a way to visualize something that was previously invisible or finds a way to make something interactive that was previously not highly interactive what i'm saying is the problem with the third demo is that it's still just textual coding and so visualizing the state space is like

Starting point is 00:43:02 it's nice but it's it's great. It's underwhelming. I was underwhelmed by the third demo. And it's not that we need to visualize your 100 gigabytes of chess data. We need to give you a programming environment where you're working with some kind of, I don't know what to call it, visual programming. Because...

Starting point is 00:43:21 But that wouldn't make any difference. I would still be programming with a blindfold on. It's just that I would not be using text. I'd be using your fun little pictures. I don't disagree. I'd still be programming with a blindfold on. I would still not know what that 100 gigabytes of data is. Fun little pictures.

Starting point is 00:43:39 My point is, and for certain kinds of things, like if you're working with text, it makes sense to use a textual language. Like that would be dumb for anybody like, say, me, repeatedly to argue against. If you're going to be working with something that is inherently textual, you want to be doing that in a textual representation. That makes sense. Same reason if you're working with like, you know, math in a symbolic algebra kind of way you want to be able to use symbolic algebra trying to put that into you know like doing math in textual programming sucks right javascript doing like math.min math.max math.this you know it's terrible compared to nice

Starting point is 00:44:17 concise notation you want your notation to suit the thing you're working with if you're working with 100 gigabytes of chess data and you could like for free have that chess data visualized on a chess board with pieces and with you know oh let me like scrub through all of these positions or whatever like if you for free like for magic had that would you still choose not to use it would you still choose to do it with a textual output of that data in a you know tty or whatever it is that you would have been using to work with that or would you take the visualization on a chessboard if it if you didn't have to build it yeah my point was about the fact that like trying to solve that problem of how do you visualize it the thing

Starting point is 00:45:03 you're saying i get for free yeah is a much harder problem than the problem i was trying to solve that problem of how do you visualize it, the thing you're saying I get for free, is a much harder problem than the problem I was trying to solve. Yes, but that's why I'm interested in figuring out if it was for free, would you have taken that? I don't know that it would actually be useful for these purposes. Like, it's cool. You make a four-year-old demo of being able to see all, I don't know how many millions it was 60 million games. But like at that point

Starting point is 00:45:27 and like I brought it down to a subset of 6 million I don't know that I would have gotten much use out of it for what my purpose is. Yeah, so then I just my problem is I don't really understand what your purpose was and whether or not it would be a good

Starting point is 00:45:43 fit for visualization. And I don't want to suggest that all things and whether or not it would be a good fit for visualization. And I don't want to suggest that all things always need to be visualized. That's my point of like symbolic algebra, you know, is a symbolic representation. It's still abstract. You still have to like do equational reasoning in your head. Point is, I really, especially in the context of this community and this podcast, I really take offense at the argument that it's like just because it's harder to build the better interface, we should stay with the interfaces that we've got. I think that we need to very seriously and emphatically attack the problem of why is it harder to get these richer interfaces? Why is it harder to get these richer interfaces why is it harder to get these better visualizations why is like like brett's demos are in the areas where he could make a compelling demo with minimal effort because those were the resources available to him but he

Starting point is 00:46:38 makes the point and i'm skipping ahead to the the end of the end in future programming, where it's like, if we settle for the sort of status quo version of programming as it kind of concretized in the 60s, and don't continue trying new things, we're going to get to this point where – and he says almost exactly this, that like the status quo version of things is going to get more and more effort put into it and it's going to get solved and it's going to get good enough that trying to do any new different kind of thing is going to have a tremendous uphill battle because it wouldn't have had any effort put into it. It wouldn't be very rich by comparison. It wouldn't have had all the quirks worked out and that's that's the thing that i'm saying is like yeah for you personally it's not worth it to put in the effort to try and um visualize your chest data but for us as a collective force of will in the world i think it's imperative that we all try and find not like here's a specific domain in which i can like make a little visualization of this one specific kind of data for this one project but like hey i came up with a clever way to visualize or do something with or

Starting point is 00:48:01 add an affordance to this entire category of thing. And it's a small affordance, or it's a small visualization, or it's a small increment. Or, for instance, hot module reloading, right? Like, that's something that does get you incrementally closer to having a real-time connection with a live running system in the context of status quo programming. And so that's the thing that i want to take away from from this is like yeah he he went super narrow and went super deep and super compelling but i think there's value in like in going broader than that and saying not just within a specific project but like across all domains like where's a thing you can find that's like this is a thing that i

Starting point is 00:48:45 can make incrementally better in this direction i completely agree with all of that i think we can get more into that in the future of programming but yeah i guess my the thing that i was just hopping on there was you saying we shouldn't be doing things that aren't visualizable in that way and that was just i think a little too strong of a statement well okay a hundred years from now we shouldn't still be doing that a hundred years from now all right i'm putting in my calendar yeah yeah yeah that's my we did our six year predictions i'm gonna go get some painkillers by the way if that's all right painkiller break you need them for this flora just discussion flora just snuck in under the camera oh freya does that too freya also

Starting point is 00:49:25 sneaks in under the camera sometimes yeah no way no way she never wants to be seen yeah so fair enough all right just give me a sec i'm just gonna get some uh lem sit lem sit how you doing uh not too bad. Sorry for the silly little pictures. It was just too fun of a turn of phrase to... Silly little picture? Yeah, like I said, like, instead of doing textual... I just want to make sure...

Starting point is 00:49:56 No, that's good. That's good. Good, okay. I figured you would be fine with it, but... Yeah, if you're not giving me a hard time, I want my money back. Okay, good. Yeah, that's what I'm here for. Good. I'm here for a hard time, I want my money back. Okay, good. Yeah, that's what I'm here for. Good.

Starting point is 00:50:07 I'm here for a hard time, not a long time. You obviously know I'm not against visual programming. Of course not, yeah. But I'm also going to use you as a stand-in for the people who are. Yeah, yeah, of course. Yeah. That's my job here. Well, yeah, we all surface each other's invisible biases.

Starting point is 00:50:32 Foregone conclusions. Ugh. Ugh. Yeah, I don't know about videos. I think it's harder to structure the conversation. I mean, if we had a good track record of highly structured conversations, I'd be more willing to... Our conversations are... The better the the text the more structured the conversation i still think there's like sampling we don't have enough samples to definitively claim that i mean there are three

Starting point is 00:51:15 of us i would agree we don't have enough samples yet but i think the better the text the easier it is to structure the conversation which is why i just want to do now and gabriel will just be the now and gabriel podcast we can i'm i'm not opposed to doing more now and gabriel i was skimming through pygmalion um yeah and i actually think like it's way too long yeah but i want to cover it there's so many like juicy silly bits in there yeah that are just like so not modern computing yeah the fact that it's like a phd thesis that like talks mostly about art for like most of it is just so fun and i do think there's some good stuff in there i just got to figure out how we do it without reading all 154 pages yeah that's a little much so i might go and spend my time like making a reader of it with the

Starting point is 00:52:12 good bits yeah i don't think we want to read 154 pages of if we didn't do a monthly episode i would say we could do a series on it but i think with doing one episode a month series are boring yeah we'd have to do like three episodes a month i think before i'd want to because like if you did a month of pygmalion and we did three parts yeah i think that'd be reasonable but like waiting three months to just get pygmalion every time yeah if somebody doesn't like the first one yeah they're not gonna listen to the other ones. Yeah. I still have to find a way to get the editing to be faster. Chop, chop, chop, chop, chop, chop, chop, chop, chop.

Starting point is 00:52:51 Thank you for contributing more stuff for me to cut. Well, sorry. If I could just take our stuff and ship it unedited and be okay with that. Something you could do to make it quicker, like do more of a scrappy edit. Would he be happy with that? That's the real thing. While still keeping the personality. That's not the slow part. The slow part is that I have to listen through it

Starting point is 00:53:09 and think about what we're saying and decide, do I want us to actually say that? I'm curious about your process. Oh, I'm glad you asked, Kermit. My process is pretty simple. I just delete all the parts and leave in all the parts. I picked up this technique from a fellow podcaster named Merlin Mann,

Starting point is 00:53:29 who has this saying, Merlin Mann. He has a saying, Merlin Mann. He has a saying where anything that's in the show is in the show. That's not actually right. It's everything that's in the show is in the show. Yeah, but that's part of the Merlin Manniness of the joke, is that he keeps misquoting himself. So in that spirit, I tried to do a really good job of editing these conversations.

Starting point is 00:53:50 That's what I like to do here on The Future of Coding, this podcast that you are listening to. Yeah, sorry, but I listened to the podcast, but the discussions were not well edited. I mean, like, I would say, I say i would so i would rather sacrifice other stuff like everything we we have a good chat you know just just no one needs to hear them there's yeah yeah i know is it dead fish time it's not dead fish time we have two more demos oh we are not even we're not okay we should not talk about the rest of these demos do we have anything i didn't think we should yeah i don't think we should i just want to say for the for the fifth one the animation one i actually tried

Starting point is 00:54:33 to build that at my previous job and got pretty far and made something that was really fun and cool and then had to stop working on it and go work on customer support and scaling and other bullshit. So I think that there's something. Okay, so I made the opposite point earlier. I want to make this point. I think it's also good to just pick a super narrow thing, even something Brett Victor himself did, and just solve that one problem in that one tiny little use case.

Starting point is 00:55:02 Clone something Brett made. I think there's tremendous value in that. You don't have to go and find like, oh, what's a better way to, you know, visualize numeric types and make them tangible and interactive at runtime or whatever. Just even just cloning Brett

Starting point is 00:55:16 is a good thing to do as somebody who did it to great effect. Yeah, I've mentioned before, I think a Brett Victor code jam where you go make a Bretttt victor clone or inspiration would be fun yeah that's actually a great idea you've mentioned that before how come i haven't internalized that i mentioned it on the podcast i know because somebody else messaged me and was like that's a good idea yeah that's actually a really good idea you know you know cuttle yeah

Starting point is 00:55:47 i know cuttle yeah yeah toby shockman and uh forest elephant yeah if you like what follow back the family tree of cuttle it starts from quite a brett victory place right from a from do you remember you know recursive drawing i do yes yeah so look clone well i don't know make a directly inspired brett victor thing and you could end up with cuttle but instead of making solid shapes that go kind of on top of other solid shapes you should make like thin rectangles that you can put inside other thin rectangles it's way better yeah make them colorful way better and have a dark background screen pond have a dream sequence so we've gotten through the demos to the actual meat of the talk oh no the meat no you've got 1.7 vegetarians on this show with you jimmy meat doesn't have to be

Starting point is 00:56:40 uh animal based i think that's what you really need to change. I love pea protein. Pea protein makes the best meat. Don't you call coconut, isn't there like the meat of the coconut or something? I think there's some fruit that you can say the meat. Tofu meat. I love tofu meat. Anything can be meat if you mean it. There's a meat of anything. So, there's some like he talks about tesla and tesla's no modes is an example of a principle um we could talk a ton about we should actually do some tesla on the show yeah yeah so tesla would be a good one to do the whole

Starting point is 00:57:17 point of all these demos was to show his principle of how you can go and apply his principle to all of these circumstances. And it's not in his mind solving some sort of problem. It's not a product opportunity. It is him trying to fix a wrong in the world. That ideas are precious to him. And when he sees ideas locked in people's heads and not being able to get out because they don't have this direct connection, he wants to fix it. And he wants to give us other examples of people in, I'm just going to call it technology, that have followed this same sort of pattern of seeing an ill in the world and wanting to fix it. And he wants to say that these principles need to be very particular. So he gives Larry Tesler as an example, who came up with this idea of modeless editing,

Starting point is 00:58:12 but that's not the way he would want to put it, right? He would want to say that he dedicated his life to the principle that no user should be trapped in a mode. Yeah, and just to be clear, the thing that Brett in this talk says is the wrong take on Tesla is that Tesla was the inventor of cut, copy, paste because inventing cut,

Starting point is 00:58:33 copy, paste is not the thing Tesla set out to do. Tesla set out to escape modal editing to get us away from modes, which were the prevailing interface paradigm at the time that you'd be in a mode and when you're in that mode your inputs are interpreted a certain way and if you go into a different mode those same inputs are interpreted differently and that causes problems for i think one of my favorite examples is if you sit down in a car and you know that it's in a certain gear and you push the gas pedal, unless you go and check which gear you're actually in, you don't even know if you're going to be going forward or backward.

Starting point is 00:59:11 And that can have dramatically different results. And that, like, series of inventions that Tesla did are all well and cool. But it's this, like, championing of no modes that is the thing that really defines what he was all about i'm curious before i think i think there's some i do have some criticism of this idea and the way he puts it the like uh that you should have a principle and that yeah that should be what drives your work that that idea i i think there's some inconsistency with what he was going to say but i'm curious to know if you all have a principle you know that you that you really do follow for your work sand yeah okay let's do this let's do this lou you should go first it sounds

Starting point is 00:59:55 like you super do uh well my joke answer is sand just make sand as much as possible i'm gonna tattoo it to me i'm gonna paint up my face i'm yeah. But the real answer is that I, my idea is that there should be no more ideas. And we have enough, we have enough of them, no more ideas and that you should merge them instead. I think we, I don't know when it was around like 2001. We had enough. And now people need to do the hard work of figuring out how to combine them together. And that's my non-joke joke answer. There you go. Actually, it's genuinely my principle. I write about it on my blog, which is a wiki slash blog slash garden,

Starting point is 01:00:42 because it's a combination of ideas. How about you, Ivan? I don't think I have a principle. Like I tried to think like, what is, what would a principle that informs my work be? And I don't think I have one. And I don't have any shortage of motivation to seek out and work on oh come on

Starting point is 01:01:07 what to me are interesting problems but i also don't have any desire to like change culture change the world or anything like that like i don't have a a need to dedicate myself to a life's work to find meaning i have like other ways that i find meaning sliding life everybody and your way to and i do end up making a lot of things and trying to make things that are of course you do you know at odds with convention or that are doing something interestingly different i just so i feel like i approach it more as why why do i care art practice it's art where is it yeah of course it's about expressing things that are subtle did you just say that i don't have a good handle on and that I don't fully understand.

Starting point is 01:01:47 And then I go through the process of making these things to try and understand them better. So it's something that I do. That doesn't mean anything. That's like utterly meaningless. Because I have some feeling or some impulse or something that I want to explore and understand. Intellectual curiosity. This is like a first-year art student artistic statement. Just for my own personal edification. Nothing to that's what motivates my work good word you see

Starting point is 01:02:10 this is this is what i find sometimes tricky about brett victor's talks and stuff is because there is this idea of like what's your principle for art and what's your principle for invention tangled up a little bit because my principle for making creative stuff is completely different from what I follow when I'm trying to invent stuff right and make new technology in a way and maybe for some people that's the same thing but I think we need the capacity to be able to see them differently. Totally. Yeah. Draw your own boundaries around your own activities and what motivates you to do them, by all means.

Starting point is 01:02:51 Like, Brett definitely has boundaries around why he does different things. And that's evident from the ways that he talks about creating technology versus creating banners know, banners on Twitter of windmills, right? He has these creative outlets that he gives himself that are, I think, scratching a different itch than the maybe higher calling he feels to reshape our relationship with technology and with the, know dynamic medium i don't draw a distinction i definitely definitely see everything that i do as being all part of the same impulse all answering the same questions do you have a principle jimmy see this is where like i find it

Starting point is 01:03:41 hard to answer this is what my like meta-criticism about the idea is. Brett kind of gives us a picture at the end about what a principle should be. And I'll actually quote here from at least the transcript I have about like he really wants to be specific about principles, he says. And finally, if you choose to follow a principle, a principle can't just be any old thing you believe in. You'll hear a lot of people say that they want to make software easier to use, or that they want to delight their users, or that they want to make things simple. That's a really big one right now. That has to be a dig at Rich Hickey, right? Did this come out after Rich Hickey or not? I don't know.

Starting point is 01:04:23 I think it's a dig at Apple. Okay, okay. I think it's very square's a dig at Apple. Okay, okay. I think it's very squarely a dig at Apple. Okay, yeah, yeah. I mean, that's a timeless dig though. Yeah. Everyone wants to make things simple. And those are nice thoughts and maybe kind of give you a direction to go in,

Starting point is 01:04:36 but they're too vague to be directly actionable. So he goes on to say, like, Leslie Terry, Terry, Larry Tesler. Terry Lesler. Terry Lesler terry lesler like simplicity uh and you know but this isn't enough they've got to be specific and so his principle your principle has to guide you uh and you have to be able to take your principle see some wrong and then apply your principle and so i took this part really to heart when I first heard this talk. And I really thought he says that, you know, finding a principle is self-discovery and you have to find something like actionable like that, that you can look at, see the wrong, see a way to fix it.

Starting point is 01:05:17 But when I, this is the one thing I really did get when I went back and re-listened to this talk. And it's, he kind of contradicts himself because yes, Tesler counts. Tesler is definitely somebody who has this very similar principle to Brett Victor, but he also mentions Engelbart. And Engelbart, what was his principle? What was his goal? His explicit goal from the beginning was to enable mankind to solve the world's urgent problems. And Alan Kay, his goal, and I quote, was to, quote, amplify human reach and bring new ways of thinking to a faltering civilization that desperately needed it. Which is very Alan Kay. Yeah. That aside, what do you, it sounds like you object to those as valid principles,

Starting point is 01:06:07 and I don't understand that. I don't see that yet. They definitely do not seem to have the quality that Brett says they need to have, where they make it very, they're not just any old idea. They're very specific, and you can see a violation of the principle.

Starting point is 01:06:23 The only things that I think for both engelbart and k are violations of these principles is everything like the world hasn't been able to solve these problems because they're not as they're not augmented or alan k kids aren't super smart in doing computational modeling and so like they're not principles in the way that Brett or Teslers are where they're here is an instance of a violation of this principle. They are just big lofty goals that you can't really point at exact problems of, oh, look, that's an example of not a new way of thinking that the faltering civilization needed or hey that's an example like you can say that's an example of an urgent problem that's not being solved but like it doesn't give you the next step with brett victor's hey here's somebody not being

Starting point is 01:07:20 directly connected to their creation here's how i can make them directly connected. I don't see the same criteria for Alan Kay or Engelbart. I think that Tesler's principles specificity is doing a little bit of a disservice to the overall argument because it's so tightly focused on a specific problem in a specific context with a specific solution that it is to me erring a little bit on the side of like talking about cut copy paste you know oh that's the wrong thing or talking about engelbart in terms of the mouths oh that's the wrong thing it's like no they were standing for something greater than that they just made those things in service of this greater thing and i think that when you look at k and when you look at engelbart and when

Starting point is 01:08:10 you look at brett victor like their principles are vague but they are they're like a vector they're like this is a direction that i want to push the world. And yeah, it's hard to say that you will ever hit an end if you go in that direction, right? Like the world is never going to run out of hard problems. Sorry, Engelbart. There's always going to be those problems. Or like kids are never going to be, you know, 100% smart, right? It's all relative. And the same thing with creators need an immediate connection

Starting point is 01:08:46 with their work like a great creators in the past have had tremendous success even with an incredible amount of remove from their work there are many many instances of somebody doing a great creative work with just tremendous amounts of intermediary between their will and the effect in the world and you can tighten that i think forever and never hit a limit where people are like 100 completely intimately connected with their work done like even even if you're in a medium like dance or something like that where it's like all i have to do is move my body it's like bodies are complicated right there's there's nobody out there there's no like jacob collier right i bet he hates the sound of his voice i bet for somebody who is so in love with the sound of his voice he loves it doesn't

Starting point is 01:09:39 he loves it yeah he loves the sound of his own voice but i know listening to him he's like i wish i were just a little bit less nasally in this range and a little bit less yeah i just wish i was just a little bit you know a little bit less you know mucusy sort of that's pretty good yeah yeah yeah um i'm just gonna do okay so no no no no yeah that's that's my thing it's like i actually don't think that those vague principles are bad i think that a principle that is a vector without an a discernible end or without even measurable increments or without even like a a sense of where or how to apply it is fine. Yeah, but Brett doesn't. He says that, talking about Larry Tesler, and that is a powerful principle because it gave him a new way

Starting point is 01:10:30 of seeing the world. It divided the world into right and wrong in a fairly objective way. He could look at somebody selecting text and ask, is this person in a mode? Yes or no. If yes, he had to do something about it. Yeah, those are good qualities for a principle to have, but there are many different kinds of principle.

Starting point is 01:10:46 And I think what he's saying is, like, here are some ways to get a more valuable principle, right? He only spends a couple of sentences talking about this, but I'm sure his views on it are enough to fill a book, right? Yeah, yeah. Mine are, and I'm no scholar of this shit. Taking him at his word on the end here, I think this segment is pretty – he's pretty adamant that your principle has to be very direct and very easy to recognize when it's being violated. And if we loosen that, that I could imagine a principle maybe for work. But when we're trying to make it so narrow, that's where I find this problem. I do think as time has gone on, I'm actually less enthralled by this idea of finding a principle. I think when I first heard this talk to me, this seemed like,

Starting point is 01:11:39 yeah, that's how I want to be. I find so many things wrong in the world. I mean, that's just like my everyday being is finding the world wrong. But like, I was thinking like, what is it that unites all of these feelings of wrongness? And I think the answer is like, nothing. Like, I just find lots of things wrong. Like, I'm frustrated by lots of different kinds of things and you don't have any like highfalutin way to wrap that up and put a bow on it and say look i have a principle i mean i could be like people should stop like i could do like the alan k thing of people should be smarter or something but that feels i think if you just replaced this idea that

Starting point is 01:12:22 brett says here which i understand he's trying to give you advice on how to come up with a principle. And maybe if I take it just that way, it's fine. But, like, sorry, I have a dog who just got a second treat for today. Lemon. So she's very excited. Janice just came home and gave her another treat. So she's super excited. And I got lemon medicine.

Starting point is 01:12:44 Oh, nice. Yeah. i have a lemon yes pen do you need to do anything jimmy you know i'll be fine i'm sure that there was enough clamoring in the background you could probably hear the the the dog running on the floor um oh yeah but yeah oh yeah yeah but i think that this like trying to have this overarching principle has become less attractive to me. Like, to me, I do think that the path I'm on is, I don't know that it's expression, like you mentioned, for art. It's definitely not stopping people from having ideas, to badly paraphrase Lou's principle.

Starting point is 01:13:22 Yeah, no more. Yeah, no more ideas. I think I just like making stuff. I don't care necessarily about mastery. I don't really, like, maybe I want to have mastery. I want to be better at what I'm doing and get better at my craft. Maybe that's how some people would say it. But really, I just enjoy the experience of building things.

Starting point is 01:13:44 And I enjoy fighting the wrongs in the world but i don't know that i uh i have a principle of why they're all wrong i just think they're all wrong so i have i have a another criticism like it's it's more of a society thing right i think there's a recurring theme in Western society to try to solve society problems with technology. Right. And that usually comes from rich white guys that work in tech. And I know that Brett Victor isn't necessarily just talking about tech inventions here, but I think it's pretty clear that that's the domain that this is in. So I just, I find it a little bit worrisome

Starting point is 01:14:33 that he connects Tesla with suffragettes advocating for women's votes right that doesn't seem like a good comparison to me and and i can see nowadays there are more and more communities of i would say i don't know disparaged young men looking for some guiding principle in their life and they're turning to very hateful movements some of that is not in tech like in cell movements but they're guided pretty clearly by a certain principle and within tech there are many things like e slash anything on twitter that is a tech yeah movement guided by a principle. So I guess, how do you know that your principle isn't shite, right? Like, and maybe, maybe it shouldn't be a tech principle that's going to solve the world's problems. I mean, the classic made up example is, hey, you know what? Like there's someone who's in a wheelchair and they can't get into our work office, right? Because there's some steps there. Like the obvious

Starting point is 01:15:53 thing is to put a ramp there. But the technologist's solution is to invent bionic legs, right? We have the solutions already. We have the ideas and it's a ramp right not bionic legs or as some e-ac people put it like technological supermen which is yeah just a little bit problematic i mean why because it's focused on men and it also has this weird crossover with some fascism in our past. I, listen, I think Brett Vector's a lovely guy. However, I think it is a little bit dangerous to say that technology combined with principles can solve the world's problems. It's probably a societal solution to many of these things.

Starting point is 01:16:42 I think that's a fantastic point. And it's helped me realize my real principle which is that everyone should be directly connected to a bitcoin wallet you know we could just like finish the episode there yeah we could so this is the end of part one beginning of the sponsor break you may ask ivan why do you ad lib sponsor break in this accent but you can't do that and i say i don't know what this accent is i think uh french transylian, a little bit of the guy who used to run the coffee shop where I got married, maybe. What the fuck? This episode of Future Coding Podcast brought to you by Patreons.

Starting point is 01:17:36 Patreons.com slash futurecoding. it's on patreon we publish every month a bonus episode so you can enjoy more of whatever this is except you can't because we don't do this this is not what we do we don't do silly accents on the show we do computers and feelings about computers and people about computers and videos about computers. And if you like computers, go futurecoding.com slash Patreon except the other way around. And $5 a month you get bonus

Starting point is 01:18:17 episodes. You make us have the finances that help us produce the show. Most recently, we upgraded audio interfaces and a microphone, a microphone stand. It helped make us make a podcast better. Such a perfectionist. You like podcasts.

Starting point is 01:18:37 You enjoy podcasts. You listen to podcasts. You listen to podcasts. Why do you listen to podcasts? Why do you listen to this podcast? Why do we make this podcast? Why do we make these podcasts like this? Why can't we just make normal podcast? Let's make a normal podcast. Future Recoding, Patreon, thank you very much. this is great because stop drawing dead is like i have so many stop drawing deads other people have stopped drawing deads i think stop drawing dead is just like a thing on its own as an idea or it's like what can we stop drawing dead i love that wait you mean like jack russia

Starting point is 01:19:17 stop writing dead programs yeah i've stopped drawing dead fish because i stopped drawing good for you i don't know if i ever started so i don't know if you could actually say stop but i was forced to draw when i was a kid so i'm so sorry do you know i reference dead fish in one of my videos yes uh is it the one with all the fish with the different ai or whatever where it's like oh it's not actually ai but it's yeah very clever it's it is the it with the different AI or whatever, where it's like, oh, it's not actually AI, but it's... Yeah, very clever. It's the fish one. Yeah, the one with the fish. Yes, I say something like,

Starting point is 01:19:51 sorry, these aren't actually fish or something because they're just drawings. I think in an older version of the script, it was more of a reference, but it ended up as just a little one. I immediately got the reference when i saw it that's good mission accomplished and of course brett is making a reference to magree the this is not a pipe oh yeah i i've done my reading today i have done i went so far back you know like we could do a

Starting point is 01:20:19 whole five hour art history thing from this teal drawer i've got in front of me alone. You have the floor. I have the floor. Um, okay. Once upon a time, there was an art movement called Surrealism. Surrealism was a reaction to the norm at the time. It was a revolutionary new idea and movement. Most of us know it as starting in Paris in the 20s, but that's only a very small picture, but that's the most well-known. And there was an edition, like a publication called La Révolution Surrealiste. The Surrealist Revolution. We're going to take on the intellectuals of the world with our Surrealist Manifesto. And in issue number 12, which was the

Starting point is 01:21:18 final issue, and just a heads up, that was a final issue because of disagreements, because none of them could agree on the right way to be revolutionary, right, because they couldn't agree on the new status quo. In that edition, there was a piece of writing called Words and Images. Les mots et les images. And I have it right here, yes, by... Is it Magritte By Magritte. Magritte.

Starting point is 01:21:47 How do you pronounce it? Put these foolish ambitions to rest. Rene Magritte. Yeah. And this. Okay. If you haven't seen Words and Images, then you should see it. Because this is really funny.

Starting point is 01:22:01 He's drawn these really hilarious images. The first one is just a leaf, and it's labelled as a cannon. And there's another one I really like. There's a horse, and then there's a painting of a horse, and then there's a man saying horse. Cheval! He's playing around with the idea of real objects and representations of objects so like a drawing is not a real object it's a representation of an object there's even one of the drawings is just like two identical drawings of a house one says real object and the other one says representation of an object

Starting point is 01:22:40 translation it sounds better in french right and um about the same time as this, he also did this treachery of images art piece, and well, no, called the treachery of images, and it says Ceci n'est pas une pipe. This is not a pipe directly underneath a pipe, and it's just playing around with the same idea. This is not actually a pipe, this is a representation of a pipe. And many years later, a charming young man called Brett Victor decided to use this in his tech talk. And he puts on the screen a drawing of a fish. And of course he says, this is not a fish. And as we all know, it's not a fish because it's a representation of a fish but that's not what Brett says that's not what Brett says Brett says this is not a fish because it's a dead fish

Starting point is 01:23:31 so for me someone I'm like a surrealism nerd right, I love surrealist art. Like, this Parisian surrealist art is just a tiny, tiny fragment of it. But I love it. And so that, for me, is this really funny, like, divergence between the two ideas. It's taking this old idea, which is very popular and, like, too popular for its own good, and using it, but, like, misusing it but i think we we know what he's saying brett victor is it says that this is just a representation it's not an accurate depiction of a fish is what he means all right so so then what he does is it okay if i just keep summarizing for the moment yeah go for it but i'm going to stop the music. It's getting a little tiresome. I don't want to listen to it anymore.

Starting point is 01:24:32 Then what he does is he draws a little line for this fish to draw around. Like, maybe you do it in Flash or Premiere Pro or After Effects or something. And he animates this fish. So he's moving around. Brett always says he for the fish. This is a male fish, right? Then of course, he writes again with the fancy font that McGree used way back when and said, this is still not a fish, right? And me, you know, surrealism nerd says, of course, no, you're absolutely right. This is a representation of a fish you know but but it's fun to play around with this idea isn't it you know like when we're drawing it's

Starting point is 01:25:09 just a representation that's what art is at the end of the day it's symbolism right but Brett Victor says no this this is this is not even a representation of a fish this is a dumb fish and he illustrates this by putting down an animation of a fishing rod and a shark. This fish doesn't react to them at all. So again, we've gone even further on our divergence from the original Magri. You know, maybe that's fine. Brett doesn't really care about that point. He's making a different point. But it's kind of of funny i think that that they're so far apart anyway then brett does uh the rest of the tech talk which i'll skip over and then he has his version of a fish it's a fish that he can basically puppeteer through this amazing visual

Starting point is 01:25:59 coding tool which he says is not a coding tool it's a drawing tool but he uses this amazing visual coding tool to make a fish that he can manipulate and move around directly like a puppet. Then, of course, I'm waiting for him. I'm waiting for him to say this is not a fish, right? Because it's just a representation of a fish, right? But he doesn't. He says this, this is actually a fish. And that is my facetious summary of Stop Drawing Dead Fish. Thank you. Brett doesn't know what fish are. That's what we've learned.

Starting point is 01:26:37 Listen, listen, listen. It's a great, it's a great talk. I'm being facetious. It's a great talk. It's a great talk. But'm being facetious. It's a great talk. It's a great talk. But I want to make this really clear. The first talk was about inventing with a principle, right? And it was about Brett's principle. This talk, and Brett is very clear about this. This is a talk about art. He says time and time again, this visual art has always been made with direct manipulation,

Starting point is 01:27:07 citation needed. He says something about great art at the end. Great art has always been created by thinking visually. He says that this kind of responsiveness that he has in his fish art is something we have never had before. We've never had it. We've never had it in a visual art medium before. Now, I just want you to, and a listener, to think about this. Do you think that's right? Right, because I made a list, and I say, I'm not really an artist, but my partner Flora, she's an animator. She's a frame-by-frame animator. She's done art degree stuff. Well, she's done an art degree and a foundation degree.

Starting point is 01:27:50 And I know you've done some arty stuff too, Ivan, right? And we went through a list of things that there's some visual arts that have been made for a long time that do not involve direct manipulation, like printmaking. You directly manipulate something, but the resultant artwork is very different. Glazing in pottery, you're guessing what will happen to this chemical that you put over your pottery. Frame-by-frame animation, you don't directly manipulate a character's movement or motion. That's something that you

Starting point is 01:28:22 learn to get good at predicting and extrapolating. Just any loose, unpredictable media, mixed media, setting up some fruit in a bowl and letting it decay and calling it an art piece, that's not direct manipulation. Photography, cooking, fine dining, composing music, writing great literature, costume design, choreography. I think Brett Victor is talking about like a subset of art. So I just want to be clear that this is not about all art. Yeah, I think so. The two things. One, the term direct manipulation.

Starting point is 01:28:57 A lot of people are using the term direct manipulation these days. And a lot of people. I try to avoid that term term at least right now in this moment i'm gonna say that i try to avoid that term i don't know that that's true but um i think it's a loaded term i think it's tricky because for a long time people said the mouse was a direct manipulation interface that you're like putting the cursor on things and directly manipulating them they say oh the touch screen is an even more direct manipulation interface because you're putting your finger right

Starting point is 01:29:29 on it and they say oh you know the the new vision pro where you look at the thing and pinch your fingers together is not as direct manipulation because you know you're using your eyes as an output device or what so this this term is like hotly debated and i think it's largely a useless term because it implies a kind of binary like oh this is or is not a direct manipulation and i think that that's not effective i think that there's lots of dimensionality to the way that you interact with something and just saying is it or is it not direct is a kind of a hard way to look at it. It's a limiting way that there's richer and more interesting ways we can look at it. For example, how do the context of art, like a wonderful thing to do because then you can say, ah, what if I just change the way that that thing is allowed to respond to me? I still express my intent in the same I think, 60s, where the idea was the audience and the performers would be in the room together like a traditional live musical performance. But there'd be a screen put between them so that the audience could not see what the performers were doing. thing to do because it's coming in the context of recorded music where more and more the performance

Starting point is 01:31:06 of the music and the audience listening to the music are separated by this you know intermediating technology that lets them time shift the performance happens before and the listening happens after and so akusmatic music is like let's recreate that experience of being detached in some way from the creation of the music, but it's still happening in real time. So you've changed one of the dimensions while leaving one of the other dimensions unchanged. And there's lots of those kinds of things that birth different art movements. fascinating to me about this talk and about i think the point that brett is trying to or one of the points that he's trying to make is that the computer is a is a new medium in that it is sitting in this sort of in between space between being like a live animate entity like a person who can hear what you say or see what you do and react and an inanimate

Starting point is 01:32:07 you know inert entity like a rock or maybe something that is animate but not in a way that we can have a conversation with like a tree or maybe an animal there's a there's a new spot in that space that the computer occupies and because it occupies this new spot in that space that the computer occupies. And because it occupies this new spot in the space of ways that you can express yourself to it and ways that it can express explore them and that there's wonderful things that are going to happen as we do so that we shouldn't just see the computer as an opportunity to upgrade existing art forms like oh now we can distribute film more effectively or the film making technology can be um you know, what's the term for it? I got you, fam.

Starting point is 01:33:06 The term you're looking for, democratized. Anybody can be, you know, a musician, recording artist. Now you don't need access to a studio. Anybody can be a filmmaker. You know, the tools of creation are now more widely distributed and there's less gatekeeping around what were formerly much harder art forms to get into if you didn't have access to those means and brett actually he specifies what requires well what the requirement is for something to be considered native computer art right he says they i think referring to like computer art creations.

Starting point is 01:33:46 Check me on that. They have to behave and respond in order for it to be native art. Right. So he's saying that this fish example, it has to behave and respond. And he talks about two different ways that it can happen. One is through this like manipulation, direct manipulation. And the other one is through um this like manipulation direct manipulation and the other way is through simulation basically you being able to guide something directly and also something being able to i guess behave by itself right and that's what he states is the deciding factor of whether something is native to computer art and I would disagree with that because I think that is one advantage that you can get out of a computer but I can think of many others like simply just

Starting point is 01:34:34 drawing on a virtual whiteboard being able to move what I've just drawn to somewhere else like I would say that that's not what Brett's talking about. I would say being able to undo something, being able to redo something, that's even less what Brett's talking about. But I would still say that they're only possible on a computer currently compared to other things. I'm sure you can think of loads of ways as well. So I think that's one thing that I just can't of loads of ways as well so i think that's that's like one thing that i just

Starting point is 01:35:08 can't get out of my head when i'm watching this talk um the other thing is i would just ask why right like why do you want to fully use the medium of computers when you're using computers just think about it you're You're making something creative. Let's say you've got a box of paints out, right? And it's got like red, green, blue, etc. Why do you need to fully use that box of paints for your artwork? In fact, it's a common thing for people who are starting out in art. You find that you you think you have to do more whereas actually a big part of it is actually cutting back and only using the bits of expression that that medium allows you to do that you want right so with this paint box say you've used all the paints and then you're like no no no i could get more out i'm gonna i'm gonna

Starting point is 01:36:00 crush the metal of this paint box and i'm gonna stick it on top of my painting and now it's native to this painting okay that's silly silly example but like why do you want to fully use and so of course there are many benefits right to use it making the most of your medium but I think it's wrong to say that you have to do that it just might be the benefits that you might want. And it's clear that Brett Victor loves puppets, right? Like I love puppets, right? He loves being able to puppeteer a fish, right? And I'm all for it. But I think it would be wrong to say that this is what all computer art has to go towards. If you sat down with him and asked him, I bet he would say that what he shows in this talk is not what he is getting at in a very specific way.

Starting point is 01:36:53 That what he shows in this talk is in the context of a live presentation in front of a group of people that's being recorded as a video so that people can watch that video. He is inherently making something that is indistinguishable from the audience perspective when compared to pre-computer media, right? He's making a little animation. He could have hand keyframed all that stuff. He could have built that, you know, frame by frame drawing it. There's traditional ways to make the work product

Starting point is 01:37:26 that he ends up showing and it's interesting that he happens to do it as a live performance but that's incidental that's not the point and i don't think that's something that he would have claimed i don't think he would have claimed like yeah there's something fundamentally different about this thing that i made what he tries to do though is encourage people to look at the computer and see that there are these new affordances there's this simulation there's this interaction this you know you can get an immediate response out of the thing or you can get the computer to do a whole bunch of work for you. And those things would let you have an experience with something that's kind of like the experience Brett himself is having when he plays with these quote unquoteunquote live fish and you can go a step further and say you know building those

Starting point is 01:38:27 interactive or simulated systems and sharing those interactive simulated systems with other people that's a kind of a new thing right like you don't have to use it to make a video that other people watch you can just give somebody the system and let them play with it and they can have an interactive experience and that's kind of new. to encode ideas that can't be encoded in perfectly repeatable sequential time-based media like traditional animation or recorded music. Like there's ideas that are uniquely well-suited to being expressed through recorded music or ideas that are uniquely well-suited to being expressed through film. There's probably a category of ideas that are uniquely well suited to being expressed through interaction and through simulation. And that if all we ever do with computers is try to emulate previous forms of media in a, you know, slightly new context, we're not going to get to

Starting point is 01:39:39 engage meaningfully with this new idea space that has opened up with this new way of communicating these ideas to people i think that's the the point he's trying to make even if he does it in a kind of an inelegant way with this puppet show yeah no i i agree with like what what you're saying you know that different different media are good for different things or you might reach for different approaches when you're making different things like i do this all the time with my stupid videos like when i have this this little bird flying around crossing stuff out that's me puppeteering it with my mouse like that's very similar to this dead fish demo that brett does but there are other times where it's completely frame by frame animation and there are some times where it's

Starting point is 01:40:25 tweened through like a hand-drawn um tween and there are some times where it's just i don't know some weird vfx done in after effects or something like that so yeah there there are different reasons to use these different things but i just worry that that that, I don't think, I don't think that's what I get from this talk. Like what I get from this talk is he wants you to feel dissatisfied with certain types of computer art. And he says this, he says, the main thing, now I quote, the main thing I want you to get out of this talk is dissatisfaction. And then he continues, and I find this quote hilarious right he says to your face he says everything you draw is flat and static and lifeless and it's just so insulting you know

Starting point is 01:41:16 like like i don't know i don't know maybe that's just your taste man man. Maybe it's... I find this really fascinating because, like, okay, so I'll also quote here. I believe that behavior and responsiveness is the essence of the computer as an art medium. And what that means is that any time we create art that doesn't have behavior, we're not living up to the potential of the medium. It's not native art but lou yeah your whole thing is making making computer art that has behavior yeah sand is about making computer art that isn't static yeah that has behavior and like cell pond is about letting people explore the behavior of these systems so i find it so interesting to hear you being so against this message when like all of the things i see you

Starting point is 01:42:13 making it's exploring that behavior a roost is exploring those behaviors like it's all encoding this behavior and letting people play with it so it is you see, like, I'm just curious, you know, you're kind of against some of what Brett's saying here. Do you see it as kind of like this gatekeeping or like, is that what you're reacting to? Or, yeah, I'm just curious. Yeah, okay, I think there is an element of gatekeeping here, you know, saying that if you've made some computer art and it doesn't satisfy these criteria,

Starting point is 01:42:49 then why did you make it? You know, it's not native art. And I think there are many reasons, like a lot of the work I've done is, I guess, poking fun at this sort of thing as well. You know, like, why did you bother making this when it's not X, Y, or Z? Like, it's not new, it's not novel, it's not native. It's really fun to play with these sorts of things, but it doesn't mean that they're the only things. Like, I put out this content basically in the form of something completely uninteractable, a video, you know, at the end of the day. And like I say, I love puppets. You know, have you seen this guy?

Starting point is 01:43:29 Have you seen this, what's he called? Ivan, you know. Barnaby Dixon? Barnaby Dixon. Barnaby Dixon. And I first came to know Barnaby Dixon because the person that ran our local pub quiz said, oh yeah, I do some puppet shows with my friend called Barnaby.

Starting point is 01:43:44 And we looked him up and it's insane. He makes these extremely intricate mechanical machines that have aspects of direct manipulation and simulation, you know, with the way they're constructed. And it doesn't use a computer at all. So I really question this idea that those are the things that make something computer computer native because i know that you don't need a computer for them i i look at like what is the computer native thing because i don't think i don't think it's what brett fechter says yeah so i've got i've got a really good one on this so so there's this question of like what's the native form of art to a particular technological medium? And how can you claim that something is native to some technological medium when you could like – you could still do that same thing without that new technology right like recorded music enabled a whole bunch of new kinds of music to

Starting point is 01:44:46 be created you could still make that music without recorded you know without recording technology like there's ways to do a lot of the new things without the new technology that enabled those new things to be discovered but there's a there's an importance here where like a large enough difference in degree starts to feel and look more like a difference in kind and so for the computer like i'll give an example of a piece that i'm working on right now i'm working on a new i'm calling it a metronome uh spoiler it's not actually a metronome um but what it lets me do is it's a it's like what a metronome does where it's just gonna like make a a sort of a rhythmic sound at a certain interval but i'm gonna do a bunch of weird things with those rhythms and with those intervals not because

Starting point is 01:45:38 it's going to be a piece of music on its own but because it's going to let me play music along with that metronome in a way that I'm not currently physically able to play music. Like there's music that I want to make that I intellectually know is possible that I don't have the physical ability to play. And most people, and I've been looking for examples of this for many years – but most people don't have the physical ability to play this music unaugmented. And I'm going to write a computer program that if I put a certain kind of rhythm in my ear, it's going to let me play this music. So the computer is going to enable me to do this thing that I physically have the capacity to do but I don't in my brain have the ability to do. And I'm going to change what's going on in my brain with the aid of a computer in order to do this.

Starting point is 01:46:32 And that like does not require the computer. If I were just like superhumanly good at stuff in my brain, I could just play this music unaugmented. I could also physically build a metronome out of gears and wood and springs that would make these rhythms exist in a way that I could listen to them and play music along with them and arrive at this artistic goal that I have. But what the computer does, the difference in degree that it has that is so great that makes it a difference in kind is that it is way more malleable and way more readily malleable and easily malleable than

Starting point is 01:47:11 physical materials like to build this kind of a metronome out of wood and springs and physical materials would not just require mastery over woodworking and mechanical engineering but it would require probably years worth of of concerted effort like if you look at you know winter gardens marble machine music contraption that has gone viral over and over again like building something like that takes a tremendous amount of effort but coding something like that is like an afternoon project, right? much more pliable in a way that allows you to explore all of these sort of, you know, explore the possibility space of certain kinds of artistic expression. So that like that,

Starting point is 01:48:13 those, you know, Barnaby Dixon's puppets are a great example, right? I think it is not just easier because I'm a, you know, relatively capable programmer, but I think it's, it's easier it's easier just as a fact of physicality versus the intangible thought stuff in the computer.

Starting point is 01:48:34 I know Jimmy loves it when I say thought stuff. That like building Brett Victor style digital puppets that react to you in a playful way with a little bit of simulation interaction is way easier than building a barnaby dixon style puppet entirely separate from how you then go on to play with that puppet and emote through it and use it just the construction of the thing is harder in the physical world than it is in the digital world. I guess like when hearing that, it makes me think, well, maybe it's the malleability more, you know, that you talk about it's easier because it's more malleable. Like you can change things, you can bend things more, right? You can undo what you've done, you can tweak. So me that's that's something that i would want to like

Starting point is 01:49:28 investigate more maybe the thing that makes computer art native to computers is the fact that you can have have behavior and response or maybe it's something else maybe it's like malleability but i'm not i'm not sold personally on the one that Brett Victor puts forwards. I think Brett's intended audience for this talk was not artists. Yeah, certainly. I think his intended audience was people who make tools for artists. Yes. Welcome to the future of arting. Well, no, wait a minute, wait a minute.

Starting point is 01:50:06 There's a lot of people in our community in our programming ass community who make tools for artists that's not an uncommon occurrence yeah and so i think this you know we could we could have debates on like whether behavior is the essence of computer art or or any of that i you know he makes statements like that i think that you could defend it. You could disagree with it. But I think when he talks about being dissatisfied with these tools, it's not because you shouldn't be happy that you made some art that's a drawing. It's that the people who are building these tools for you

Starting point is 01:50:40 stopped you from being able to create some things that you, with your skills as an artist you would have been able to make maybe you chose not to maybe that's not what you're trying to express right now but the thing holding you back is not the medium of the computer it's the people who built the tools in that medium for you and so it would be the same way as if you were a painter and all you ever got was a big, thick brush. Like that's all, all brush makers are just making big, huge, thick brushes

Starting point is 01:51:15 and you can never get that fine detail. And of course you could. You can go and sit and take the big, thick brush and really painstakingly make that sort of thing. But it's not the way you would want to express these ideas. You want brush makers who are making these fine-tipped, nice brushes so you can really do this well. And so I think this is what Brett is getting at,

Starting point is 01:51:40 is that this dissatisfaction with our tools is the fact that there's all sorts of creations we could have made with the computer. There's all sorts of these behavioral things we could have made and no one's building the tools in a way that lets us express that directly. Instead, they make us express that through textual media like code instead of something that might be a little bit more in the medium that we're used to of a geometric expression. And that's why he kind of shows the system. And I think this is really interesting because like what he did,

Starting point is 01:52:14 if you ignore the art angle for a second, because I don't have, I don't have a lot of art that I'm making. So I found it more interesting from like the thing is what he's done is he's kind of made a small talk like system, but where the code are these geometric constructions, right? He talks about, oh, this is self for historical reasons. He refers to the object to sell for this or one of the two, I can't remember which one. But anyways, it's this little small talk system with these methods and all of these different things.

Starting point is 01:52:45 And what he's showing is that there's this other way of doing programming, of having behavior in a system that doesn't rely on textualness. if they want to create these behaviors have to go to code is not a failing of the artist or the fact that they create these non-behavioral objects is not that that's their bad art it's that no one's given them the tools to create this if they wanted to yeah no i i i'm i'm with you there i'm with you and and for the record i give this talk three stars out of three for code for technology is three out of three and um and and also three out of three for like the talk and effectiveness on the whole so with your example of you know being stuck with a thick brush i think that's a good analogy and i and i would agree with bre Brett that you know it does sometimes feel like you know we should be able to do more with computers right maybe we want a thin brush yeah as well to let

Starting point is 01:53:51 someone have the choice to do the kind of art that they want to do but from my perspective currently me and Brett agree perhaps that we we only have red we only have red paint and that's all we can paint with. You know, with the same analogy, Brett is saying, hey, well, we could, in computer art, we could have blue paint as well. And that's what makes computer art native. You just need blue paint. And so I'm sitting over here saying, what the fuck about green, right? Like, it's not just blue paint that we need. So just to make it very clear, like, me and Brett Victor both agree, we could do more with computer art. Brett Victor says, yes, the way to do that is you need to have this one very specific thing that I

Starting point is 01:54:39 want, which is puppeteering, and geometric puppeteering, which I'm calling it. And I would say that, hang on, hang on. That is one really helpful thing we can have, but we can have other things too. And so when you talked about Aroost earlier, I really, really, really didn't want Aroost to be a thing that you can directly manipulate. The way that i wanted to use a computer to like help you is by having more sloppiness more accidents that come about from things being more indirect if that makes sense so like there are multiple ways of trying to make the most out of computers it doesn't have to be dragging a fish's corpse around yeah i think the the puppeteering part of this to

Starting point is 01:55:26 me was always actually like a a red herring if you would it was brett not fully living up to his vision in my mind because the the point that i see these things is that it's not the puppeteering it's not the direct manipulation it's not any of that it's not the puppeteering, it's not the direct manipulation, it's not any of that, it's the behavior. Well, and like how you imbue the behavior. Yes, it's... Well, but the medium itself is about behavior. Yes, he has

Starting point is 01:55:56 opinions on how you should make that behavior happen. But it's about the behavior in his mind, is that computers are about encoding these behaviors and being able to interact with them, explore them in some way as part of the art. So I think of the Arctic exhibition where there's rain and you can walk around through the rain and you don't get wet. What? Yeah, I don't know this one.

Starting point is 01:56:19 Describe this. No, never heard of it. Okay, so there's a big rain room and it's gone to a bunch of different places, London, LA, etc. And it's just like pouring down rain constantly in the room. But you walk through it, and the rain where you are completely stops. Oh, that's cool. And you do not get a drop on you. You can even like put your hand out, and like the rain does not fall on your hand um so it's these sensors and you know all of this engineering work to make

Starting point is 01:56:46 sure that like you get the experience of walking through the rain while remaining perfectly dry yeah which i and this this is going to sound like um the podcast equivalent of a shit post but i'm 100 serious is the exact invert of what they do in video games where when it's raining in a video game the rain is only falling around your character because to simulate rain anywhere else would be wasting computing resources so they just put a hyper localized rain effect immediately over your character and it follows you wherever your character goes why is that relevant because video games we haven't yet talked at all about video games yet for this segment. I want to put a pin in that. We have to talk about video games a little bit.

Starting point is 01:57:27 Yeah, I agree. So this, to me, is an example of that kind of behavior that computers enable, right? You trying to do this system, yes, it might be possible to make a rain room like that that had no computers involved whatsoever. But this is the kind of behavior that you should be able to do with computers. And I think a roost is another example where like it is behavior that you're encoding in the system. Is it about direct manipulations about puppetry?

Starting point is 01:57:57 Nah, not about that. But it is about putting, instilling that behavior into the art so that it becomes this living artifact. So it's not a this living artifact so it's not a dead fish so it's not dead music it is it reacts to its environment it reacts to the manipulations that go on can i talk about a roost for a minute yeah all right cool so i'm gonna put some words in your mouth lou uh-huh so i'm So I'm going to upfront disavow you, Lou, of any agreement or disagreement with what I'm about to say. I'm about to say. I agree.

Starting point is 01:58:32 Okay, great. Here's the art of a roost. Here's what the art of a roost is. And this is important. We'll get back to this in a minute. So a roost, for anybody who hasn't seen it, is a tool you load it up in a web browser. And it lets you do a bunch of things but the main thing that you can do is make these little circles that to me look like frog eggs or that look like the little circular recording button like the record

Starting point is 01:58:58 button where it's usually red you click it and you make some sound your computer microphone is active you make some sound and then you click it again. And it recorded whatever little sound you made in between the times you clicked it. So it's like for making these little tiny snippets of recording. And you can make as many of them as you want. And then after they've recorded something, you can click on them and it'll play back that little bit you recorded. And so if you want to, you could make 10 of them and have each one, you know, record yourself saying an individual word. And then you can like fridge magnet poetry, like tapping them and whatever and remix the words you said. Or you could play a little note on an instrument for each one of these and make yourself a little, you know lets you play with sound. But it is not at all built with an interface that makes you have a lot of control over the sounds. You can't really edit the sounds.

Starting point is 01:59:54 You don't get super precise timing for when the sounds start and stop. It's kind of sloppy a little bit. And then you can string these sounds together so that if you click to activate one, it will, with some rules that you input through the system, activate other sounds in a kind of a sequence. And so you could build little constructions that play a melody or that say a bunch of words in a certain order. And you can play with that. And the art here, and what is interesting to this audience, the Future of Coding audience, is that Lou put some art into the very design of this thing as a system. Like they put some creative intent into it. There's an intentional bit of friction that they talked about a minute ago. And I want to compare that to the design of guitars. Guitars normally have frets on them, which are little bars that go across the string so that

Starting point is 02:00:53 when you put your finger in a certain spot, it plays a certain note. And you can go up and down the fretboard to play different notes. And these notes are evenly spaced apart so that they match the notes you would get if you sat down at a piano keyboard, you know, C, D, E, F, G, et cetera. There are other kinds of guitars that don't have those frets. They're called fretless guitars. And they are like a violin or a cello

Starting point is 02:01:17 where you can put your finger anywhere and you get exactly whatever pitch it would be from putting your finger at a certain spot. And that won't often line up with a specific piano note unless you're, you know, very skilled and have practiced this a lot and can put your finger in exactly the right spot. These are two different things. It's not like one of them is better than the other. They're like better contextually right like a guitar is an easier instrument to play if all you care about is playing a perfect g instead of like oh it's a g but it's a little bit too sharp because i didn't put my finger in the right spot but there's an in-between thing

Starting point is 02:01:54 that people can do where they can make guitars where the frets are spaced differently where they don't match up with the keys of a piano keyboard, where you can put frets that are, you can have many, many more frets. You can have 31 frets per octave and get all these interesting in-between notes, all these microtones and make microtonal music. And it's a very different kind of music. And it comes from choosing to like, not even like create friction right because lou talked about the the

Starting point is 02:02:28 work they did on a roost as almost like intentionally creating friction or intentionally restricting direct manipulation and i don't think that that's quite it it's like you can have something that feels like an affordance whether that's the presence of frets on a guitar or the absence of frets on a violin those are two different kinds of affordances you can have something that feels like an affordance and you can like dial that up or down or change the shape of it a little bit and each time you do that with this affordance you end up with like a new possibility space where you can make new kinds of expression and get new sorts of ideas out of you or get ideas from somebody else through

Starting point is 02:03:12 that interface and that is why i am such a crank about visual programming it's not that i think it's like here's the the way that all programming ought to be. It's like, it's really easy for us to find things to make graphical on a computer because we have screens and we have rendering routines and we have, you know, color and animation and all these kind of things. And you can do anything you want with those combination of elements. And it's going to change your relationship with the things that you make through that interface in interesting ways that are worth exploring. And so just as Lou made a roost with some artistic goals around wanting to restrict people's ability to directly manipulate things because they had this artistic goal of encouraging scrappiness.

Starting point is 02:04:08 Normalize sharing scrappy fiddles. That mantra. And it has the second order effect of like it's an artistic tool, right? A roost is an art project that is itself a tool that other people use to make art. For instance, Lou used it to make music that then went into a video and it's like many levels of snakes eating other snakes, eating other snakes. You can do that with your programming tools as well.

Starting point is 02:04:34 You can use a crappy React app to make a little coding sandbox environment that lets people build things that they can use for something else there's no right or wrong here and so to the extent that there's some kind of prescribed notion of like what's native to the medium of the computer or not it's it's i think that's the wrong framing i think you need to look at what the computer gives you and try all sorts of different ideas at different amounts of like taking advantage or choosing not to take advantage of what the computer lets you do nice yeah yeah obviously

Starting point is 02:05:15 yeah you that's how i think about a roost is by restricting some of this direct connection that you have to what you create like it kind of frees you when it doesn't have your name on it you're like i didn't make that music arus made that music right that it kind of frees you to just make it there's a blog post by shane crowley about talking about how he got stuck on making music then he used a bit of arus and now he's back making music again like that that's the goal of it you know i could ivan you talked very nicely about it there's this other thing called blank.page it's this website and it has the same sort of idea about trying to free you with

Starting point is 02:05:56 writing it makes you type invisibly if you choose that setting that's like the literal opposite of what brett says, right? You lose this direct connection, but it's something you can only do on the computer. It's actually better. Look, I think direct manipulation is really important and I couldn't have done a roost without it, but there's just more, right?

Starting point is 02:06:22 There's just more. I think the text, to me the the point that brett victor was making the text disappearing is exactly the kind of thing he is talking about it's behavior right right it's not about direct manipulation i think that's the problem is his demos were him directly manipulating things but yeah if he let's imagine the demo again, but he direct manipulates. He has just a puppet that he's doing, and then the puppet appears on the screen. So everything he does is just by taking these physical things, and he's moving them in the exact way that they end up appearing on the screen.

Starting point is 02:06:58 That's direct manipulation, but it's not encoding behaviors into the program. The interesting bit for him was about how the fish can do more than just what his hands are doing. His hands are moving the fish up and down, but the fish is looking in the right place. The fish is darting. The fish is having these behaviors based on the environment. And so being able to make the text disappear is a great example of a behavior that is encoded into the program. That to me, I think, is the key here. And I think this direct manipulation bit is not as interesting. And I know you want to talk about games, Ivan.

Starting point is 02:07:38 I want to talk about game development. Yeah, I think that video games are a great example of this where what you are doing is encoding behavior that people are exploring and i think the games that i enjoy the the most at least are kind of setting up these behaviors that you have to now either become an expert in the behaviors you have to understand them more deeply you learn i think of like super meat boy all right it's a very simple game very simple behaviors but like i learned that physics system and now i can kind of feel the physics systems and it's not that i'm directly manipulating the physics itself it's that i'm exploring those behaviors and being

Starting point is 02:08:16 put in different situations where i have to understand them and i find that very satisfying i think this talk and all of his talks maybe would have been better served by not being delivered as a talk, but by being delivered as a system that you can play with. Except that the key idea here is, hey, people, pay more attention to how you build systems that people can play with. And that idea is not something that you can easily express through a system. So he's faced with this conundrum. I find myself faced with the same conundrum because this is something I care very much about. That I want to get the idea out there.

Starting point is 02:09:01 And I think he also wanted to get the idea out there that like there are things you can do when you are making a system that people can play with, and the choices you make when you're building such a system are a channel through which you can express ideas. You can express a worldview. Like, Super Meat Boy, what it gave you, Jimmy, is it gave you this, like, feeling for a sense of physics that is now like when you or like celeste or any of these like very tight platforming games when you play them you get this this feeling

Starting point is 02:09:33 in your bones you get this like sense of rhythm you get this sense of space you get all these sensations and you get this tacit knowledge or like body knowledge, but it's not for your body. It's for like within this simulated world from the perspective of this character in the world. You get this whole range of ideas that only come from interacting with the system. Yeah, that's great if you want to make a platformer, but you could also do that if you want to make a change

Starting point is 02:10:02 to like the way that people view economic policy or if you talk or writing a book or going around and going door to door and shaking a lot of hands and talking to people or any of these other things but you can also express that idea through building a system within which somebody has an experience and that experience is what puts the idea into their person so the like that quote that we keep coming back to the main thing I want you to get out of this talk is dissatisfaction. We omitted it earlier, but he says, I want you to go home and open up Photoshop and feel like you're drawing dead fish. That's a critique of the people who made Photoshop, because they did not build a system that lets the users of that system express ideas that they couldn't express through other

Starting point is 02:11:08 forms of visual art. Photoshop is just an emulation of painting and drawing because the ideas you can express through Photoshop are by and large the same sort of ideas you can express through painting and drawing for better and worse. For better because the drawing skills that you have from real world drawing translate kind of nicely into Photoshop. And if you develop a sense of color theory or composition or whatever in Photoshop, that can aid you if you do photography or that can aid you if you do painting or whatever. But what they didn't do, what the Photoshop makers didn't do, what John Knoll and whoever else didn't do, is they didn't make a system in which you're expressing

Starting point is 02:11:45 ideas that you can't express through painting or drawing what what lou didn't do with a roost is make an environment where people can express ideas through making environments that you know you use those environments to do whatever you want like there's another meta level where it's like what would it look like for somebody to make a tool that lou could use to make a roost where a roost is an environment where somebody can have a different relationship with music than they otherwise would have or a different relationship with their own creative process what would the like next step back from lou look like what would the the the like meta medium the medium for making mediums i'm not joking yeah the pre-loo yeah that's what i think this future of coding community is is kind of often dancing around doing is it's like we're over here looking at you know oh make this

Starting point is 02:12:40 environment for making generative music or make this environment for building web apps or make this environment for that yeah and and the community itself is a little bit like an environment for making environments yeah that's the the thing that i am interested in is like what is the native form of ideas for those meta mediums i mean the thing is maybe there is none maybe maybe the environment for making environments is not an environment maybe it's like a mindset or or a community right yeah a culture yeah or a surrealist uh art movement no but you know that that point about photoshop like going into photoshop and feeling dissatisfied i think that's just where i i get lost from this talk though like i just don't see that because i think you know we can i hate adobe i hate adobe whatever but like photoshop is this incredible tool and and it ticks none of the boxes that brett victor wants it to tick but it ticks ticks so many boxes for making the most of computers.

Starting point is 02:13:46 Right? Like layers, like history, like blah, blah, blah, like selection. And I just, I just, I'm just not on board. I just can't quite go along with this. You know what I mean? Maybe you don't see the problem because over the last 40 years, people have made this tool so good and have explored this paradigm so well that, yes, it is a very effective medium for doing all of these things.

Starting point is 02:14:16 But maybe, just maybe, if people had explored different avenues or thought differently, we could have had a future of Photoshop, future of programming, even. I mean, it just feels to me like the kind of computer art that Brett Victor wants us to enable is just not the same as what I want to enable. And that's, and that's okay. Like, and, and I'm, I'm completely okay with that. I'm fine. But, but like, I just don't agree with the framing of this as being the one thing we have to do. That's all I, that's all I disagree with. Like everything else in terms of this being a really good way of using computers,'m fully on board with but I just don't get this dissatisfaction from going to tools like photoshop and and again I'm I'm wondering

Starting point is 02:15:12 are we looking at this the wrong way like if we want people to be more creative and be able to make more adventurous computer art that they wouldn't make in any other media because it's impossible or it's really hard how do we help them do we need to give them more behavior or do we need to give them more i don't know emotional support or creative freedom obviously you know where i sit on this yeah but like i just want that to be a consideration yeah there's the like you made the point that um and and jimmy also made the point that there's like, this is sort of about we are technologists. What can we as technologists do to, you know, invent the future that we want to see? How do we make a change in the world through invention as our form of interacting with the world as opposed to social advocacy or other kinds of change?

Starting point is 02:16:03 While still very carefully walking the line to not try and solve social issues with technology, right? There's that tension there that we need to be cognizant of. I also think that, like, you know, we're moving on to the future of programming, a point of which is, hey, don't be set in your ways don't just take the common practice and the status quo as the only way that things can be or even as the best way or even as a good way it's probably a good way it's probably got its flaws and its warts and whatever but it's probably also got its strengths and something like photoshop is good it's great it's different but it is also not the only thing and as an example that i think you might agree with lou like it's kind of good that the company that you know offers

Starting point is 02:16:55 photoshop also offers after effects and also offers flash with you know rip and also offers like other tools that try to do what photoshop did in one area but either applied to a different kind of media so in the case of after effects it's like photoshop but for time-based media instead of static images and some could argue that those two things ought to be combined i think i would argue that they ought to be even split up smaller than that but flash is a is a great one because it's like what if photoshop but you can also code against it and the drawing affordances are designed to go really nicely with the apis that you use for coding that there's meant to be this synergy between the activity of drawing and the activity of coding.

Starting point is 02:17:48 And how that birthed a whole revolution. We look at Super Meat Boy and all those kind of games. Super Meat Boy was made in Flash originally, or Meat Boy was made in Flash. Flash and the Newgrounds and E-Bombs World and all of that, that you know early indie game scene and experimental interactive art pieces on the computer and homestar runner and all of this culture that came out of that like came out of somebody not just trying to emulate the traditional thing but trying new things trying different ideas trying lots and lots of things so that that like i don't think brett says hate photoshop i think he's meaning like just don't stop at photoshop keep going

Starting point is 02:18:33 make another one make a different one make one that has these qualities of of simulation and puppetry and whatever but don't just stop at puppetry and simulation and interaction. Do something else. Maybe he should have called this Don't Stop At Drawing Dead Fish. Yes, yes. Right? Yeah, yeah. It's like a wise man once told me. You take all of the metal

Starting point is 02:18:58 and you put it back in the ground. This is an in-j in joke you don't get it because you have to inside the joke to get the joke and today you are not inside the joke you're not inside the joke and i will not take you inside the joke try as i write i cannot take you inside the joke if we're gonna get through future programming we gotta transition at this point all right i'm lou now no it's yeah okay i have less less to say about this one. Yeah, so this talk's fun simply because Brett Victor time-traveled, which is great. I mean, he invented time-traveling just to do this.

Starting point is 02:19:54 Three stars. He goes back to... What year is he supposed to be in? 1968? 1973. 1973. All right. Yeah, yeah, you're right, because on the thing it says 1973. Yeah, yeah. So 1973. 1973. All right. Yeah, yeah. You're right because on the thing it says 1973. Yeah, yeah.

Starting point is 02:20:06 So 1973. And he's kind of given a – the title is kind of a joke, The Future of Programming, because he's talking about the future that never happened, right? Which is what I kind of take the future of coding to kind of be as well, right? It's about the future that could have been. It's about the counterfactuals, the way things could have gone but didn't. That's what I think is interesting to explore here rather than what will actually be the future, because that will probably be boring. And so he starts with 1973, and he's kind of giving us an

Starting point is 02:20:40 overview of a bunch of work that's happening in and around that time period that is going to change programming in the future. And of course it doesn't. And he focuses on four things that we've already been shown the way of how we can not do these things, how we could go the other way. And he kind of makes the joke that in 40 years of course we'll never be doing these things so one is writing programs in code two writing programs as procedures which really he means like imperative kind of style programming as opposed to like goals or constraints where you say what the computer what you want the computer to achieve rather than how it does it. The third one, I'll do it because it's me,

Starting point is 02:21:31 that programs are currently written using lines of text, but there's this remarkable new thing happening where people are hooking up video displays to computers, which enables us to work with spatial representations. I think they should stop that. Yeah. Stop drawing live programs. And then the fourth one is that programs are currently written as sequential statements and if you look forward a little bit it seems like hardware is increasingly moving towards a massively parallel architecture and that we're

Starting point is 02:21:59 going to need a parallel programming model to take advantage of it yeah like the t2 tile and robust first computing and coin in the jar reading from ivan's uh description here that he sent us and the first slide he puts ivan o brotherlands work on hestpad what pad what nice job jimmy delivering my joke three stars perfectly done wait i don't get it i don't get it there's somebody named ivan sutherland who made a thing called sketch pad oh oh that's that's really funny and so the joke was gonna be, brother lens, because then you go, oh, brother, Ivan. Yeah, that's why I... Test pad, really? The joke was so bad that I had to ruin it. Thanks.

Starting point is 02:22:55 Couldn't let Ivan have his fun. Oh, because you normally do. Uh-huh. I let Ivan have so much fun on this podcast. Yeah, well, I get to censor you. So even if you let that one slip without meaning to do it, I let Ivan have so much fun on this podcast. Well, I get to censor you. So even if you let that one slip without meaning to do it, I'm still going to claim my token.

Starting point is 02:23:15 So, I mean, this whole thing is a survey of papers, some of which I'm sure we'll cover. I'm sure we'll talk about Sketchpad at some point. I plan on talking about Planner. There's a bunch of papers here that are talking about all these different ways of programming. You know, mentioned a small talk. Okay, wait a minute. What? No.

Starting point is 02:23:30 What? This is not a survey of papers. What? This is a survey of important works. Papers out of the room. We don't care about papers. He's not presenting papers. He's presenting these works.

Starting point is 02:23:40 He literally puts papers up on the- No, he's using an overhead projector because it's a period specific joke and he shows papers he shows scans from the documentations of these projects because he's like he's cosplaying it's not a dirty word ivan you're allowed to have papers no because the there's an important distinction here which is that we're not interested in talking about these works as though they are like historical artifacts that are documented for us to reflect on it's like uh we need to treat yeah we shouldn't learn from them what we need to treat these things as if they are real things that

Starting point is 02:24:18 are going to be the direction that things go like the things that we would be talking about post hoc looking back as papers are like fortran and c and okay yeah let me rephrase yeah he talks about works yes that have recently been done yes that are like here's the promising future direction we're going to go we're going to actually be using sketchpad like systems but better and some of these works have been cataloged in the form of papers that we will probably talk about at some point got him in more detail like sketchpad's invention of chickens and hens as their data structures very key technical insight that we still use today it's funny that just to get under my skin you're sacrificing the quality and enjoyability of the podcast as a thing that we're putting out

Starting point is 02:25:15 in the world jimmy doesn't care about podcasts and neither do i the experience of the dozens of listeners to this sacrifice just so that you can you can get a little bit more of a dig in on me uh-huh you you had to dig up my papers so good reason i i want to just say like i think i i think you know we don't have enough time to dive into each of these works well less i we this is what happens every time we get to in these podcasts, okay? I just got to go meta here for a second. Ivan's like, let's make this one short. And then every time I try to skip over sections, Ivan's like, hold on, I have a lot in this section.

Starting point is 02:25:53 I'm like, okay, then we need to have skipped over other sections in order to get our time constraint. And I have a time constraint today, so we're not going to go into each of these things. People can watch the talk i reckon that if if it was just me and jimmy we wouldn't have this problem i reckon we'd get it done really quickly we'd stay on topic and then the editing would be much easier too that's why i reckon i'm just putting it out there all right i will sit here and have no opinions and not offer anything.

Starting point is 02:26:26 And I will let the two of you take this talk that I actually don't care all that much about and don't have much to say on. It's like the only thing I have to say about this talk is like, yeah, duh. See, let's do it. Let's go. I think that's such a silly take. Yeah, it is. It's such a silly take. Okay, the big point of this talk is not really about these four ideas.

Starting point is 02:26:47 It's summarized at the end that it would be a shame if in 40 years we're still coding in procedures and text files and sequential models, blah, blah, blah. But the bigger problem is it would be a shame if we didn't realize there were another way. That's the goal, is that there's other ways of programming. If we thought we had everything figured out and we never explored new models of programming, then that would be the problem. And really what we have to do is we have to get to this conclusion, quote, we don't know what programming is. We don't know what computing is. We don't know what computing is. We don't even know what a computer is. And once you truly understand that,

Starting point is 02:27:29 once you truly believe that, then you're free. And you can think anything. That's how the talk ends. But saying like all four of these ideas are clearly right is just, they're clearly not. And the fact that everyone tries

Starting point is 02:27:42 to always tackle, like so many projects try to tackle all four of these ideas at once. And that's why they end up being like utter failures. Because these things of having code, having text files, having procedures, and what's the fourth one now? Parallelism. Oh, parallelism oh parallelism like there's a reason we do all of these things and to just try to say like oh clearly in 40 years we shouldn't be doing any of these things it's not as if people didn't

Starting point is 02:28:14 know about these ideas still don't know about these ideas and don't pursue them because they're just straight boring people who won't explore cool things it's because they're really hard to to get away from these are not just good because they've been explored they're good ideas because they have a lot of nice properties about them and so if you're going to go away from them you have to really find some good benefit and i don't think we have found that benefit yet in most domains for most problems is that like is that like how you view these things and like the reason they haven't happened is that they're too hard is that is that the reason or is that is that what brett victor is saying i think the talk kind of gives you this idea that they just

Starting point is 02:29:06 weren't explored because people didn't explore them, more or less. It's not anything about the answers themselves. They're totally feasible. Look, we were doing them in the 1960s and 70s and so the only reason we're not doing them today is lack of imagination or something.

Starting point is 02:29:23 But yeah, I think a lot of these things, it's not that they're impossible. It's just that they're not doing them today is like lack of imagination or something right but yeah i think that i think a lot of these things it's not that they're impossible it's just that they're not obviously better than the alternatives and that trying to make them obviously better than the alternatives is a lot of work and it was a lot more work especially given computing hardware at the time etc then it was to follow the path we did follow. Yeah, like one example, the parallel processor array, you know, that he shows. Like, it's such an exciting slide, this.

Starting point is 02:29:55 It's just like a, instead of having one computer, you know, connected to some one bit of memory, you have lots of computers in a big grid, like a T2 tile, and they're all working together somehow. And it's really, it's a really appealing idea. But I've seen Dave actually struggle over the past how many years it's been.

Starting point is 02:30:17 And it is clearly really hard. There are really tricky things to figure out with it. So yeah, I see that. It's not just that people haven't tried them have you built a system using ivan have you built a system using goals instead of procedures yes and how was that uh it was great in the domains where it was appropriate uh-huh like i think you made a very good point which is that you can think of this as almost like two columns there's like you know things that we already have that we know very well so textual code that is

Starting point is 02:30:53 you know written as procedures sequentially or whatever and executed sequentially and then there's the other column which is like oh the new experimental interesting things that point to you know radical dimensions that we should definitely keep pursuing which is yeah absolutely you know direct manipulation of data structures goals and constraints spatial representations and parallelism right like i i agree with your point jimmy that it shouldn't be an either or it should be like a yes and should be like i want all of this stuff i i want like there are some circumstances where a sequential recipe of steps is awesome right like he even does a little joke where he references like he's talking about the actor model as a potential way to leverage massively parallel hardware

Starting point is 02:31:36 and he makes this throwaway joke about like oh it'd be great if like a swedish telephony company were to get really interested in this, referencing Erlang, which is famously, you know, the actor model. Inside each of those actors, you're still executing sequential statements, right? You're still working with a sequential thing, but it's broken up into like millions of tiny little sequential pieces that together are massively parallel. So I don't want to say, hey, we need to be dispensing with the old thing as we go to the new thing i i would never sincerely make that argument and if you ever hear me sincerely making that argument come wrap me on the knee um i also don't want to settle for hey that

Starting point is 02:32:19 those four things that he talks about are hard and so it makes sense that we don't really have them because we do have direct manipulation of data structures in a spatial context using goals and constraints massively parallel we have that all that those four exact things in highly effective synergy with each other all over the place in game dev that so many parts of game dev take those four things and put them together jimmy is shaking his head yeah well jimmy is wrong so the ps3 put in a co-processor which was a no jimmy parallel gpu gpu a gpu is massively parallel yeah i, I understand. I'm making a different point.

Starting point is 02:33:06 Okay. All right. The PS3, I'm pretty sure, was that generation. Yeah, it had a cell processor co-developed by IBM, and it was a disaster, and it was really hard to program. Absolute disaster, right? Like, nobody ended up using it. People tried. People really wanted to use it, and there was just no good use for it. Why? people really wanted to use that and there was just no good use for it why because trying to do

Starting point is 02:33:25 this sort of parallelism for general compute wasn't good it wasn't helpful it didn't make it easier to build these sorts of things yes of course graphics are massively parallel because they're and we explored that area and have done, really well at it because it is the kind of domain that it works for. I wasn't talking about graphics. I was talking about GPUs. Yeah, but GPUs for the kinds of problems they solve. Yeah. You don't write all of your game logic in the GPU for a good reason.

Starting point is 02:34:00 Yeah, and we can get into concretely what those reasons are, but I don't think that's the interesting thing. The interesting thing here is to look at the yes and version of this that you proposed and find that there are lots of instances of problems that like uh physics in games is a great example because for the longest time people thought oh there's no good way to do that with data parallelism so you have to do it with sequential execution you have to do that you know with with a with a traditional like cpu style single thread of execution mindset and then there were breakthroughs that moved it on to a GPU-like device going massively parallel and finding great things you could do there. And then it moved back to the CPU for reasons that are interesting to get into here. And it is moving back to the GPU again, that as we learn more and more about these different problem domains and come up with new algorithms and come up with new techniques and in the other categories like spatial representation direct manipulation those

Starting point is 02:35:08 move back and forth between being things where there are great artist focused tools with great visual affordances and you use them with you know a mouse or a space mouse or a pen or whatever and often those go back to being more effective to use through code, through writing in that, you know, indirect representation and move back to being artistic again, when there's another change in practice or another breakthrough in technology, these things are not fixed. They are fluid and every new kind of way that we have of tackling these different problems or these different dimensions of computation, right? Because there's things in here about behavior, right? Code, execution.

Starting point is 02:35:51 There's things in here about data and data representation. There's like all of these different dimensions along which our work with the computer is mediated. The way that we can express ourselves keeps changing and keeps, you know, we keep finding new ways, or at least we ought to keep finding new ways to do it. So we do, though, and that's the opposite of what Brett Victor is saying. He's saying these are all things we didn't explore. And if your examples are all true, we have explored them. We continue to explore them. In what, but what you said is right. In narrow domains in specific circumstances. There's like specific cases where we do that.

Starting point is 02:36:29 Brett's idea is that they shouldn't be confined to these specific circumstances in narrow domains. This should be the mainstream of how we do all programming. This should at least be available for us to move into and move out of. I think he makes stronger statements like it would be all of his talks make strong statements i don't i don't want to yeah yeah he says like we should just pack up if we're still using locks yeah because he's a rhetorical person what does that mean it means he's a he's he's a rhetorical i mean of course he's making of course he's exaggerating a little bit there but i think the point of this talk is that these four things of course not just these four things but these

Starting point is 02:37:13 four things are examples of paths unexplored that we know how to do them that but also that they're unimaginable to us today because of how unexplored they are. And that the problem is we've had this kind of hegemonic way of programming that's become the mainstream and no one can imagine new systems. And I guess what I'm saying is like, this one never sat well with me because I think all of these things, of course there's better work to do with them.

Starting point is 02:37:47 Of course we can explore them further. But it's not that we haven't explored them. It's not that we haven't tried. It's not that we don't continue to try. It's that the reason they haven't won out as mainstream programming is because they just don't work that well for mainstream programming. And they might one day, and hopefully some of them will one day, because they would be great. But we haven't solved these problems. See, I find this talk interesting

Starting point is 02:38:18 because it ends with this idea that the most dangerous thought you can have as a creative person is to think that you know what you're doing. Like that's that's quite a statement at the end. And it really resonates with me. Like he's calling for people to be open minded, which is funny because like I didn't like the dead fish talk because it felt more close-minded. But like, you know, seeing this makes me realise that no, yeah, Brett Victor is calling for people to try stuff. And maybe, yeah, maybe the reason that these ideas haven't happened is because they were bad, or maybe it's because they weren't explored enough. I would actually suggest we don't know, right?

Starting point is 02:39:05 Like maybe Jimmy can see these ideas for what they are, the piece of shit that they are, right? He said, I know that's not going to work, right? And do you know what? Like, I trust him. I trust him. He knows more than me on this stuff, that's for sure. But like, I feel like Brett is almost inviting you.

Starting point is 02:39:22 Well, prove to me that it sucks then, Jimmy. You know, like make one of these things and show me what. So listen, listen, listen, listen. I have a, when I was doing some spatial programming work, right? When I was building Cellpond, this spatial programming tool, I looked back into like papers from 20 something years ago, and there was a point where they stopped investigating a certain line of inquiry, right? They stopped looking down this symmetries route for making more expressive spatial rules,

Starting point is 02:39:57 right? It's not a very important thing, but for one reason or another, they stopped it. And it didn't make any sense to me. I'm looking at these papers. They were so close. Why did they stop? And there's a really clear reason why they stopped. It's because they came across block-based programming, like scratch-style programming. And then everyone stopped bothering with the spatial programming, and they moved to that.

Starting point is 02:40:19 So that's kind of like an unsatisfying reason to stop in a way you know and and i wonder what what are the actual reasons because you know we can come up with overarching reasons you know they oh it's just no one was inspired enough or it didn't work out but what's the like specific reason each one of those cases i bet that would be an interesting story in each of those cases and i and i kind of like to know to know, go and find out someone, please. Yeah, I do think, of course, there are these unexplored areas and the reasons we don't explore them

Starting point is 02:40:53 are often on a whim or because of social factors or because of funding got cut or because AI winter happened or da-da-da-da-da-da, right? Like you can have things that have nothing to do with AI that get associated with AI,

Starting point is 02:41:06 Lisp, that people don't end up pursuing it. And it's not that Lisp was bad. It's that, you know, these confluence of factors happened. At the same time, I can recognize Lisp was probably never going to become the dominant thing because of the properties it has. There's just certain things. I mean, I think this is going back to the worse is better series of things, right? There are certain properties that things have to that make them more successful or not. And I think all of the

Starting point is 02:41:35 technologies that of these four things that became popular here that became the mainstream have this worse is better dimension to them and all the things that brett wants in this talk to become the mainstream are the the right thing and i i think that ignoring that is what i find so i i understand why for this talk this is again also why i don't i i'm skeptical about talks for this format is because they lack the nuance that a paper can have yeah there's so little content in a talk and i love talks but when we're trying to get through this they just they are not made for that nuance they're made for getting a point across for getting people to think and then we're dissecting them and they they don't hold up to

Starting point is 02:42:26 that dissection and that's pretty normal for a talk i see what you mean like i felt i kind of felt that with the dead fish one like the criticisms i had like you were defending the position and i could see that yeah that's probably what he was getting at but that's not what he said you know exactly you know but yeah like maybe it's the same here there's a thing that we're all doing that i want to acknowledge which is we are all when we're talking about like do these ideas work for programming why why are these worse is better ideas the more conventional ones why are they so well suited and why are these worse is better ideas the more conventional ones why are they so well suited and why are these more radical ideas less well suited and there's a thing going on here which is that when we say programming or program or code we're thinking of a very specific

Starting point is 02:43:20 kind of activity to make a certain kind of thing to make a certain kind of artifact and i want to just flag that like i'm i'm gonna play the you know like html is programming photoshop is programming well maybe maybe not but on a good day i will say photoshop is programming right now i want to do a more specific thing and And I want to say that, like, he flags Sketchpad as being programming because what Ivan Sutherland can do in Sketchpad is draw some lines and then express some relationships between those lines that the computer will maintain. And it's now the computer's job to say these two lines need to be perpendicular. So if I grab one of the lines and start turning it, you, computer, have to turn the other line to maintain that perpendicularity. So that's, you know, constraint expression. And that, I would strongly argue that that is programming,

Starting point is 02:44:19 because you've told the computer to do some work to maintain an invariant, to maintain some condition, and the computer is going to now change the behavior that it exhibits in order to do that thing that you asked. I think that's a minimally controversial statement of what programming is. You've made the computer now do some work for you, and it's going to keep doing that work to satisfy a requirement that you set up. And you could have written that imperatively but you just happen to do it with a constraint the place where we find these more radical ideas are thriving are places outside the things that we usually think of as programming like if i say game development right like where is where jimmy you asked hey ivan have you ever worked with goals and constraints yeah of course i have like building ik chains in maya so that when i make my character walk their knees do the right thing right like that is in a purely artistic domain

Starting point is 02:45:12 i'm making a visual character that can move around and making it so that their knees bend the right way and i don't have to manually do that i can just lift their foot up and the knee bends forward and i can just animate the foot that's it was a rhetorical question because i know you have yeah but that to me is every bit as much a programming activity as me writing some javascript to power my website or me you know building a swift ui you know interface or me building writing an algorithm to sort a bunch of numbers or whatever. Like, the fact that I was in an art focus tool, making a character's knee band does not in any way diminish the fact that I was programming.

Starting point is 02:45:55 I don't, to be honest, I don't think that was implicit in any of the things we were saying, though. Well, there, I just want to make sure that when we say that, oh, yeah, these ideas, these more radical ideas are only relevant in these specific circumstances, right? What we may do or what somebody may do when they hear that, when they say, oh, yeah, in these specific circumstances, then these ideas are worthwhile. But otherwise, it's hard to make these ideas universally useful what we're kind of doing is carving out exceptions or carving out things that are like outside the core activity that we um unconsciously think of when we think of programming like when we think of what is programming right like you know we think of text editors and terminals and you know an maybe you have a browser window or something where you can see your result.

Starting point is 02:46:50 And we carve out all these other things like working in Maya or writing a shader or working in AutoCAD only because of history, right? history right at one time those things in the sort of activity that you would do to do character animation or to build a cad drawing you would do those things with the same activity that you would use to write a binary search or to build a an interactive system for for solving flight trajectories or whatever right all of that was done by writing code in Fortran or COBOL or whatever, right? It was all done through that same interface. And we made richer interfaces for those things. And now we carve them out as an exception. And I would say that all the stuff that we do that hasn't been carved out is just begging to be carved out and pushed out of that core conception. I just don't think we're carving anything out.

Starting point is 02:47:45 I think what you said there is exactly right. Like we initially didn't have good interfaces for those kinds of things. And we had to rely on the same stuff we would do for a binary search. And we figured out a better way of doing it. Completely agree with you. And I think that better way is still definitively programming. And anyone who suggests it's not... They're not listening to this podcast. Right. Yeah. Still definitively programming. But I think it reinforces the point. It took us a long time to figure out

Starting point is 02:48:14 that the problem itself was hard of how do you do character animation, right? And when we started, we didn't have great answers for it. We didn't have great techniques. And we had to rely on bad tools like text in order to code up everything because it was general purpose. And as we understood the problem better, we were able to make these better interfaces for it. And I think that's the pattern for all of these things is there's a reason like small talk didn't take off as like the small talk image system didn't take off not because like small talk images are a bad idea but because we don't understand them well enough to have a bunch to solve a bunch of problems that we could have solved with them yeah i mean this is what like jonathan edwards is looking to his whole

Starting point is 02:49:02 research project is looking to do is understand those kinds of things. We're missing some primitives there. Right, exactly. And it's the same thing for these domains where we can have these better interfaces as they become domains that we understand better in order to get them into this place. So why can we do things massively parallel in certain cases? Because we understand the problem and the problem itself lends itself to massively parallel why can we make non-text interfaces because we figured out good interfaces for talking about these things and we understand how to manipulate those things in a general way etc like that's my point is not that these things aren't programming or whatever it's that the things that we are able to make these better interfaces for, it's because we've understood them in a deeper way.

Starting point is 02:49:47 And we're able to get to that point. And of course, it's an end goal. But I don't think it's that we've ignored these ways of going about it. It's just that we don't know good answers for so many of them. That's my point, is when you are doing conventional programming when you when you you know don't think of a pink elephant or whatever when you're like okay what is programming in that conventional model of programming that we all just like jump to quickly when we're not challenged to think of it with a broader sense like i see all of that stuff as just waiting to be carved out

Starting point is 02:50:28 and put into its own world with its own you know richer approach and that the the fact that we do all of this programming in a way that like we call it general purpose i think that's a really loaded term right like i don't think programming is general purpose i think that that is a that is a misnomer but we call it general purpose as sort of a grab bag of saying like this is all the stuff that we need to do in this way because there's something like essential about that stuff that requires some generality that comes from using text and using you know von neumann architecture and using um you know this kind of indirection and i think that that's flawed i think that like the point of this talk that i really take to heart is if we keep exploring

Starting point is 02:51:21 and if we keep finding new things and we don't treat the status quo as a golden goose or like a sacred cow or whatever, and we aggressively try to find parts of it that we can excise and move into a new way of working, that eventually it's all going to go away. Does this talk make you feel like you're being told off by Brett Victor? It's like, guys, everyone, you had all this time. You had all this time and we have everything all figured out. I think that is – when I'm picking on this talk, like I said, I'm picking on it from a medium where we're being all picky and discussing all of these little bits. If I just take a step back and just take the talk as a whole as it's intended to be taken instead of picking apart all the little bits it's a great talk i'm sorry brett i'm sorry yeah like it to to channel alan k these talks are like the the only talks good enough to be criticized like i don't i don't i'm not saying that sincerely but like k said that about the mac right the mac

Starting point is 02:52:45 is the first computer that's good enough or the first like popular computer that's good enough to be criticized and it's like we're criticizing this not because it's bad but because it's so good and we want more good and it's like how do we get even more good from this i think that's the thing that i like about all brett victor talks is that even if you come away from them thinking I completely disagree what was he talking about they're so good at making you think you know like they they capture your attention and interest like nothing else I think about the dead fish talk and the inventing on principle talk, like Steve Jobs introducing the iPhone, right? There's something about them that just gets you thinking.

Starting point is 02:53:32 And look, here we are all these years later arguing with each other about it. And I think it's great. I think it's great. I will just say that, like, I think these talks are great. I'm very happy that we talked about them. I'm happy that they exist. I really liked Invincing on Principle,

Starting point is 02:53:51 even if I maybe don't find the principle idea compelling anymore for myself. I think Stop Drawing Dead Fish is a thought-provoking talk. It wasn't my favorite. This one I do really enjoy but i i hate the ending i know lou said that they love the ending i hate the ending saying that you don't know or saying that you know is now the dangerous thing i shock that i would be not happy with this i know i know it's so out of character that I would be unhappy with this, but I, I do think we have to come from a place of knowledge in order to make

Starting point is 02:54:31 progress on these systems. I think saying, I don't think, I think conflating knowledge saying, oh, I know these things and conflating that with like an unwillingness to change and learn new things. I think we have to know in order to learn more. And I think that's the point that we have to say is that we are explore. If we, if we think our exploration is done, that's the dangerous bit.

Starting point is 02:54:55 Yeah. I think that's what he means. I know, but I hate the rhetoric of it because that's what everyone says is that knowing things is bad and that you shouldn't know anything. I hate that. I'm sorry. I, I'm not trying to, know lou you liked it so don't i don't think but like this i think people too easily conflate open-mindedness and ignorance and i think obviously we don't want ignorance all right we we've got to get his response on this, right? Because he did write a little response to people who made that criticism. Okay, I did not know that.

Starting point is 02:55:29 He says, in the references for the talk on his website, Brett Victor says, I want to be clear. I am not advocating ignorance. Instead, I'm suggesting a kind of informed skepticism, a kind of humility. Okay, then I agree with that. Yeah. Absolutely. of informed skepticism a kind of humility okay then i agree with that yeah absolutely and that's why we will never do a non-paper again because the thing that brought us all together was some writing

Future of Coding - Elephant in the Room

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.