Future of Coding - Compassion & Programming: Glen Chiacchieri

Episode Date: July 3, 2018

Glen Chiacchieri has worked at the MIT Media Lab on Scratch, at Dynamicland with Bret Victor, and is now becoming a psychotherapist. He's known for his Legible Mathematics essay, his Flowsheets programming prototypes, and the Laser Socks game, among many other projects. In this conversation, we discuss: how he grounds his research in compassion, the tradeoffs between working on the "model vs UI" of programming, his software-company-in-the-making, based on Flowsheets, our shared dream for the future of open-source READMEs, and how Dynamicland does and does not point towards the future. The notes for this conversation can be found at futureofcoding.org/episodes/26.Support us on Patreon: https://www.patreon.com/futureofcodingSee omnystudio.com/listener for privacy information.

Welcome to the Future of Coding. Today we have Glenn Chiaki on the podcast. Glenn has a resume that many listeners will be very familiar with. He has worked at the MIT Media Lab on Scratch and at DynamicLand with Brett Victor. He's known in these programming research circles for his legible mathematics essay, his flow sheets, programming prototypes, and the LaserSox game, among many other projects. So welcome, Glenn. Thanks, Steve. It's good to be here. Yeah, it's awesome to have you. So I like to start these conversations off by talking about your kind of origin story,

how you got into programming and specifically this subfield of improving programming. Yeah, I got very lucky. Feels important to acknowledge that first. So in college, I went to Northeastern University for a degree in what eventually became computer engineering, although I kind of made my own software engineering degree out of that. And in that program, there's a kind of co-op program where you go to work for a company for six months, then go back to classes, then go work for possibly another company, and then go back to classes. And there's usually three of those. I only did two, and I did them at the same company, which was a startup with a kind of like, it was a freight logistics startup called Open Mile. And I actually learned to program on the job there, which was the lucky part.

I was not qualified for that job, but I learned it all very quickly. And that was my first real foray into programming. And then over the years, I got more and more interested in kind of the act of programming itself. got more and more interested in kind of the act of programming itself so how i was doing this thing that i was doing and getting more and more like wait how how is this what is actually going on here why why is why am i making mistakes like what's what is all this that makes sense so what um who was who or what was like your first entry point into this way of thinking? Was it like Alan Kay or Seymour Papert or Brett Victor? Because I know, you know, you're one of the people who's interested in all of those folks. saw from him was Magic Ink, his essay on design. While I was working at that startup, someone showed it to me and my mind was blown, as tends to happen when you read or watch things by Brett

Victor. And then I got more into his work. And I think shortly after that, he came out with Inventing on Principle, which had a lot of the, I think, core tenets of what I saw or what I currently see as important in programming environments. It's interesting. You're like an OG Brett Victor fan. You're pre-inventing on principle. Oh, is that what OG is? Well, I think like a lot like Chris Granger and I was reading an article that said that John Rezig, a lot of,

a lot of the people who were really inspired by Brett Victor, I think were inspired by inventing a principle. So you predate those guys. I still think magic ink is an amazing statement. And to me it was important about how to communicate well. And I think that's kind of an underlying thing in Brett's body of work that maybe isn't totally appreciated is that he is probably one of the better communicators

using media that I've ever come across in my entire life. And so that essay for me is like, captures the very important thing about how to communicate well, and like, okay, examining, you know, what a software experience is actually trying to provide for someone, and then saying, why not just do that, like kind of directly instead of this, like like interactive thing that at the time, I think was more prevalent than it is today. There was a lot more kind of interactive, like like widgets and boxes and pops up pop ups and things like that.

But today I think some of this, his principles in that have come more into software actually. Yeah. I remember when I first looked at Magic Inc. and one of the headers of that essay is like interaction considered harmful, something to that effect. And I reacted very negatively to it. I felt like I almost wanted to stop reading the essay

because it felt like almost anti-technology. What do you mean? Like, of course interaction is a good thing. But then when you read it like oh wow like like interaction means like dialogue boxes and like clicking like a lot of clicking and nobody likes lots of clicking to your point that a lot of the principles that he talks about in that essay are um have been incorporated into mainstream interfaces is that uh he redesigned amazon's search interface and it just is such an amazing example of the power of his principles.

And I immediately after I saw that went, you know, recently to search on Amazon and see what it looks like today. And it looks so similar to the exam to like his redesigned version. So I don't know if they got it from him, but somehow his ideas have found their way into the mainstream. Yeah. And I think part of that essay is quite well received, if I recall correctly. And so I assume it kind of it actually has sort of permeated out into the people who are designing these apps now. You know, it's like it's over 10 years later at this point. But I think one thing that Brett does well in that essay is seeing underneath what the kind of core thing that a person wants to get out of using these different interfaces. So for Amazon, it wasn't like the person wasn't trying to click a link or something and see more information. They just kind of want to know, is this a good book? Or is this a good product?

And I feel like Brett kind of took that as like, here's the actual thing they're trying to do. Let's design the perfect experience for that. Or whatever perfect means. Yeah, I'm totally with you. I remember whenever I read that essay or reread it, immediately after I find myself looking at all my interfaces, like the apps to find movies,

the apps to figure out which train to take, they're all woefully inadequate. It's not helping me to do the thing I'm trying to do. And I want to redesign it in a Magic Inky kind of way. And after I read that, I started actually doing what you just described. I started making doing what you just described I I started making like all these prototypes I like remember seeing I uh later when I visited Brett and met him in person I saw that he had designed a like BART schedule which was like very Brett victory it's like on

piece of paper uh just like a static it's like oh yeah i could like design my own one of these what is the perfect like magic ink style uh bart uh subway train schedule uh like layout that i can quickly get the information that i need if i'm just like running out the door or about to run out the door yes so what'd you come up with oh well it's a little hard to describe through audio got it does it look similar because i know in magic ink or maybe it's a little hard to describe through audio. Got it. Does it look similar? Because I know in Magic Inc., or maybe it was a different essay, he has a BART scheduler thing. Oh, yeah. Yeah, that was the software. He won the Apple Design Award for that, I believe.

So did yours look similar to that, or was it kind of a different take? It was a different take. It was mostly just like, here's the schedule for the different days and here's how like a format that you could see at a glance like when the next train is coming it was just like a thing to stick on the wall not like a widget to have on your computer screen got it and it was useful i got to use it every day oh cool when cool. When I lived in California. Yeah. So, um, I know that you're also a big fan of Alan Kay, um, to the point where you've started a wiki for a lot, collecting a lot of his work. So I'd be interested to hear kind of, um, how you fell into Alan Kay's work and how he's similar

and different from Brad and what you find interesting about him? Oh yeah. There's a whole world in Alan's, I guess in Alan's mind is one way of putting it. Um, the way I got introduced to Alan's work was through Brett. Um, so my, at one point I started working with Brett Victor, um, directly in his lab, just working with him. And at that point that, that lab was started by Alan Kay. Um, that was kind of the thing before dynamic land, which wasn't extremely public. It's called CDGG the communications design group and so in the process of working with Rhett I he just recommended and I found a bunch of different links to Alan's works and I got I kind of went

down the rabbit hole so to speak like whenever I find someone who has really interesting work I kind of tend to consume it all and like kind of download that person's published work into my mind and then like start churning through it which I think is a great way to learn so at that point I started reading all of his different works which I really enjoyed because a lot for a long time I saw my role as a professional not as a person redesigning programming it more a person helping people to understand things and programming was one of those and

Alan had a lot to say both about programming about how the human mind works. And he has a lot of really good ideas and work behind software artifacts supporting human understanding. Yeah, I've heard Brett say that people always give Alan Kay credit for being this technologist who invented small talk or object-oriented programming, but that drastically undersells him. Brett says that he's one of the greatest philosophers of our time. So could you maybe give us a kernel of Alan Kane philosophical wisdom to whet our appetites?

Oh, man. That is a tremendous challenge. I'll try and give something though. I would say Alan is not maybe the best philosopher, but one of the best systems thinkers of our time. He has a really deep sense of how important it is to be systemic thinkers. And these systems are like our political system, technology systems, education systems, systems in lots of different areas. And his vision, I would say, is something like a kind of really well-informed democracy.

He views democracy, especially like the American democracy, as one of the shining examples of something that was really hard to invent. And then once we have it, it's really important to hold on to and nourish and give rise to more of these beautiful kind of democratic ideals. And specifically, he would like that to happen through education, through people learning what he calls powerful ideas. These ideas that are not obvious to someone

just kind of growing up in a natural environment. Things like scientific thinking or democracy or just kind of basic human rights. That's a really weird idea in the span of history. Most people have not experienced human rights. And so he wants to nourish those through education and having really good educational experiences, especially for children. Yeah, I think that's a, I think you did. I gave you a hard prompt to like something,

you know, give me some kernel of Alan wisdom, but I think that was a great summary. And it reminded me of something that Brett said to me when I spoke to him in January. I was talking about how great Scratch is for kids and how, because it allows them to do the things that they want to do anyways, make games and make things dance and music. It allows them to do the things they want to do anyways but learn to code at the same time and um brett's response was that i don't i don't remember it exactly but it's something to do with he referenced the shit crayons essay that um that just because kids do creative things in scratch or minecraft doesn't mean that those things are are good tools um and it doesn't mean like kids are learning some form of programming,

but it doesn't, it's not that interesting because kids would make things move and play games anyways. They'd be doing those things anyways. Instead, we want to build tools and environments for kids to do things they wouldn't

be doing anyways, such as democracy or scientific thinking, the sorts of non-human universals. He said that Alan had that, this list of like the human universal things that people do anyways, like singing and dancing. And then of non-human universals. He said that Alan had this list of the human universal things that people do anyways, like singing and dancing, and then the non-human universals, the kind of enlightenment values

that we cherish. Anyways, kind of a little rant, but I thought this could be a good transition point to talking about Scratch, which I don't think is shit crayons, but I get Brett's point there. Yeah, I think that's a subtle, maybe not exactly subtle, but it's a non-obvious point that I'm glad that Brett has, although I also don't think Scratch is shit crayons. I want to acknowledge the amazing artifact that Scratch is

and what kind of concrete human good has come out of it agreed 100 so yeah i worked on scratch um as a contractor for it was only about seven months but it was a very long seven months in a good sense like i got really um friendly with the creators of scratch and i still talk to some of them to this day and I go back and visit occasionally and I kind of just downloaded the ethos of the space and got a little bit more sense of the design values of Scratch and I started learning a little bit about the history and the backstory of how Scratch came to be, which I think is very interesting. And from like, it, to me is one of the,

one of the like, top ideals, or the kind of best examples we have so far of what a good programming environment can do, what a good programming environment is like. And I think there's a lot to learn from both how Scratch is designed and how Scratch was made in the first place. Yeah, awesome. Let's get into that. So tell us the story. How did it come to be? This is all through my understanding.

So if I get any details wrong, I'm sorry. Sorry, Scratch team. understanding so if I get any details wrong I'm sorry sorry scratch team but so it I think it starts with Alan Kay actually so Alan had produced e toys in the think in the 90s and e toys was kind of a small talk specifically for children in some ways. It was this kind of all-encompassing software system that allowed you to do very powerful things within it and do this kind of simulations and other kind of programming tasks that Alan wants to help children be able to do. And one of the main programmers from that was John Maloney. And John came over to the Scratch team,

to the Lifelong Kindergarten group at the Media Lab, I think in the early 2000s. And Mitch Resnick was also talking with Alan and they had a bunch of different retreats. So there's a lot of cross-pollination between what was kind of Alan's group and was the not yet scratch, scratch group. And it started with this kind of like, you know, Mitch Resnick and i want

to say natalie uh rusk natalie was this amazing is this amazing designer who i think is a big part of why scratch is so important um or why scratch is so good because she's she is one of the people who started the Computer Clubhouse, which is a bunch of after-school programs for teaching kids how to use or having a creative environment for kids to use, like Photoshop and play music and try all these new digital tools that are coming out. And Mitch and Natalie saw this happening in their computer clubhouses which are now kind of nationwide or even worldwide at this point and we're like kids aren't programming

like how can we get them to program and that's when they started seeing e-toys and we're like oh i think there's something here with e-toys so they kind of brought john maloney in and started making their own thing inspired by e-toys and it feels important to say natalie sat down with children every week for two years and just like watch them use different iterations of scratch until it was like kind of airtight, this like, especially the first interaction with Scratch. Fascinating. Yeah. I had heard parts of that story before,

like the retreats that the labs kind of went on together. I think I spoke briefly with Mitch Resnick a few months ago and he, he almost made it like a bit of a competition between Scratch and eToys. Like Alan was saying, I want to teach kids scientific thinking, and eToys was a bit more serious, and Scratch kind of took a more playful video game-ish approach. And then I don't really know what's going on with eToys, but it seems like Scratch has certainly won the adoption game. Yeah. It's easy to set up that kind of... That's actually a quite common dichotomy, I would say,

is that I've struggled with quite a bit, which is the kind of Scratch approach versus the eToys and trying to understand how they're similar and how they're different and kind of what values each works on. And, you know, is it a good thing that scratch is so popular? I think in most, a lot of cases for me, yes. But I also hear Brett's criticism

and I know this is Alan's criticism, that there's kind of a sense of wanting to do more or that there's more potential to do that Scratch doesn't quite hit in some way. Yeah, so I guess maybe I want to ask kind of more explicitly, in what ways do you feel like Scratch points towards the future of programming? And then what are some of the places it leaves room for improvement? Yeah. So I'll tell a brief story. One thing I've done is teach programming to high schoolers as part of what's it's called it was called mission bit this was for it was a free after-school program for public high school students in san francisco and i mentored a few classes and taught my own class and i started with scratch because i like

scratch and there's this one particular moment i remember where I kind of gave like a three minute introduction to Scratch. It was like here's kind of generally how it is. There's the screen on the left. There's these blocks on the right. You put up the blocks together to do stuff and then I just told them okay go do the introduction. It's like kind of a sidebar thing that just pops out. Just kind of go through the step by step and then once you're done we'll come back and like figure out what to do next and they all completed it within like 30 seconds to a minute or something like that but then something interesting started happening which is they all started just spontaneously messing around themselves with no direction for

me i didn't tell them to do that. I didn't even suggest it. They just started making things on their own. Like I would say, I would call it productively messing around. And to me, that's amazing. Cause I was like, I had this insight right after that, which was like, wait, if this was JavaScript or whatever other programming language or environment, it would have taken them over six weeks to get to even close to the same space. So to me, something important about Scratch is that it took what could be a six-week or two-month or some really long amount of time interaction

and smushes parts of it down to like a minute or a couple minutes. And to me, that's a really a function of its UI. Like it really pushed down whatever that beginner learn learning curve is. So what's probably the bare minimum or close to it. And so, and what, how would like, how did it accomplish this? Would you like give the credit to the like no syntax errors, the shapes? What, what gives Scratch the magic? That is something I may write an entire essay on someday.

I think there's lots of really important parts, not just in UI, but also in the community aspects of the Scratch programming experience and the emphasis on doing things that children authentically find interesting in the realm of Seymour Papert or Piaget and these people who are focused on what children find meaningful. But just from a UI perspective for the moment, I think having a toolbox of options is really important. I think having the blocks that you can just drag out and are just kind of objects instead of having this blank text screen where you can put anything and having a kind of good starting point

of like here's a character and here's things you can do with the character give it like a pretty low on ramp I would say or low floor as Mitch likes to say kind of give it like a pretty low on-ramp, I would say, or low floor, as Mitch likes to say, kind of easily accessible to beginners experience. Yep, I agree. I don't think it's paralleled. It's the best in terms of the low floor. So I think now is a good time to switch gears a little bit and talk about your time at CDG and then I guess, which turned into dynamic land.

But first I'd be curious to focus a bit on CDG because as you said, it is kind of a secret organization that now doesn't exist anymore. So I'd be curious and probably listeners too about kind of what it was like there, what kind of things you and others worked on. Yeah. So it was a lab in San Francisco from I think late 2013 to sometime in 2017. I don't know exactly when it officially ended. It was the San Francisco lab was a building in south of market Soma in San Francisco. It's this beautiful building that we found and

it housed several of these research groups of the communications design group, which was basically, in my understanding, Alan kind of rounded up all the resources to make a research lab in the style of a park back from the 60s and 70s and 80s. And so there are several groups within that. That was CDG as kind of Alan kind of head of the organization,

but not actually in power, just kind of maybe spiritual leader or something like that. And he... It was a little bit cult-like um and especially because it was very radical in the sense of all the there's there's several different groups um led by different pis there was my group that was headed by brett victor there There was John Maloney's group who worked on a kind of successor to Scratch. There was Vi Hart's group, which was working with VR. They were called Ella VR.

And there was Alex Worth's group in LA. And he was working on a lot of different kind of future programming ideas. And I think Yoshiki was another group, although he was in LA and often worked with or around Alex Worth's group. But there's these several different groups and they're pretty much autonomous, but able to communicate. Um, yeah. Cool. Um, and so what kind of stuff did you work on while you were there? Uh, so the kind of dictum was, um, Brett was really interested in taking, he did the Media Lab talk, which was, do you remember the name off the top of your head? Oh, that was such a good one.

Was that Drawing Dynamic Visualization? I don't know. No, it was Media for Thinking the Unthinkable. Yes, of course, of course. that was Brett as I view it that was kind of Brett looking at his past you know five years of work or more and saying okay what is all this stuff he just made and what is it pointing to and it was pointing to him to this kind of really new idea which is now what became dynamic land but this like really big set of ideas around what dynamic media could mean, which is not a thing on a computer screen,

which is not a thing that is kind of inheriting old ideas of how media works, like static media, like paper-based media, things like that. And so Brett established this group to try and work in this new vein of like really wild, really new radical ways of working with dynamic content, dynamic media. And the group was called the Dynamic Medium Group. We also, we didn't know what that meant. We had Brett's work as a kind of template and he had published, I don't know if you've seen this document,

a floor plan and research agenda. Yep. It's combined. Yeah. Yeah, very long horizontally. Yeah, it was this huge poster uh basically it started as a pdf and then he needed a poster because it wanted to be a poster is is how he would say it i think but yeah it's this it's this giant document trying to lay out like the these different aspects of human experience and human media and trying to

capture things about how media today did not work well and how it could work well in the future yeah it's a beautiful document i find it very inspiring yeah and that was the kind of agenda of our group is to try and figure out what that actually means or to start making examples like little steps or big steps towards whatever this dynamic future could be maybe a future that's not so isolated to rectangles that became one of the the big like selling points i think of the division which is yeah why is all the computer stuff in this rectangle it kind of makes sense but like we have this whole world to work with we have these bodies we have such keen social senses we have all this like stuff that is not

supported at all by the dynamic medium as it exists today. It's true. Flat rectangles are flat and rectangular. Right. And Brett had his essay about the future of human interaction design. Yes. Yeah, something like that. That's a good one.

Where it's like, wow, we have all these hands, and these hands are amazing. Why aren't we using them? Sometimes I think he's at his best when he's complaining about stuff or like ranting. Like his future or his learnable programming essay is also one of my favorites. When he like takes something in particular and is like, this thing is so the opposite of what I think the future is.

Yeah, I like that kind of style of, Brad has made several things that like in that i've seen in person that are like uh yes more of this and no not this and often they're the form of pictures so when it's like a bunch of people hunched over their laptops at not looking or talking with each other he goes no not this that goes like the no column and then it's like a bunch of people cooking together and all like sharing you know a common space and working together and like looking at things together and collaborating in a more human way humane way that goes in like the yes column and i've seen like several of these kinds of yes this no not this kind of format i think it's just a helpful way of having like a bunch

of deeply felt principles but no like no concrete way to express them besides no no no not that and oh yeah yeah something like that so hearing you describe kind of the agenda of this group i'm a little bit surprised because i didn't realize how uh like dynamic landy the his group at cdg was i i didn't think you guys got 3d physical reality stuff until dynamic land so it like you guys is is my new understanding more accurate that you were like building precursors to dynamic land at cdg uh i think yes and no so that is the dream and that was the vision. But a lot of, especially the early things that came out of our group and CDG in particular were things that were more in line with Brett's work

pre-2013, I would say, or 2013 and before. So those were his main set of essays and videos around future of programming and new expressive artistic things and drawing dynamic graphics and explorable explanations and things like that. Got it. And so a lot of the things that I and many of the other people did were those kinds of projects. So that's where I came out with legible mathematics. And that's when I came out with a interactive multi-representation viewing of a civil rights documentary called Eyes on the prize and those were kind of in the spirit of explorable explanation things on the screen totally but eventually we started getting

you know brett got more and more into these ideas about dynamic physical media, so media outside the screen. And he was kind of like wanting to like pull us along with him is how I think of it sometimes. Awesome. And so then we started making, it really kind of, it started gelling together when Brett came, Brett and Robert Oxhorn created the kind of precursors to Dynamic Land, which was called HyperCard in the World.

And that was a system like Real Talk, which is what powers Dynamic Land, that allowed us to hook up lots of different inputs and outputs like cameras and projectors and processors and scatter those throughout our entire lab space and then create demos or experiences that could kind of talk with each other. So you could like,

we used a lot of laser pointers at the time. So like you could laser point on one wall a thing on one wall and it could affect something in a different part of the room because it was all one operating system and that's when the dynamic land like ideas were really starting to take off cool um before we continue on that timeline and get into like the dynamic land stuff, I want to talk a bit about your legible mathematics essay, because it really is one of my top favorite essays ever of all time. It's just amazing. I put it up there with like, you know, learnable programming. It's like I in my mind there, it's like almost an extension of that essay. Um, so, um, I kind of want to first acknowledge you for it because it's so great. And then also get, yeah, yeah. Um, get a sense of, uh, like, you know, from like me wanting to produce content, that's as good as it, like, you know, I want to kind of hear about how the ideas kind of came together and how long it took to work on it

and advice you'd have for writing really amazing content like that that points towards the future of programming. Well, that's complicated now. It was not as complicated when I wrote that essay, but my ideas have shifted and evolved as I've kept kind of working on this stuff. So for me, there was a bunch of things that I kind of felt awkward about in programming that I suspected did not have to be as awkward as they felt. And legible mathematics to me was the first kind of forays into imagining in this very, like

you said, learnable programming. It's pretty much modeled off of learnable programming, the essay. Kind of foray into different ideas talked about in a lot of it easier to understand how something is working if the symbols are augmented with like a more concrete thing, I would say a more concrete representation. So like you see mathematical symbols and it's just like, that looks like a bunch of symbols. What does this mean?

But then maybe you see examples of like the step by step execution of that mathematical formula and it's like, oh, okay. That's probably a little easier to understand because you can kind of go back and forth between the symbols and the concrete representation. And I ran into this many times, both while programming and in math class itself, where I was like, this feels weird. Those symbols could mean anything.

They mean a very specific thing often, but I don't really have a good sense of just coming into that, what that is, what it's actually doing, or what it actually means. So basically I brainstormed a bunch of ideas which most most of which went into legible mathematics about different ways of representing arithmetic such that you actually see what the data is instead of just having like the arithmetic um and i kind of got into the like you know what can we get just from looking like how can we design a mathematical expression so that just by looking at it we can get a whole bunch of understanding of how the structure of this thing is working so that that was really directly inspired by not just learnable

programming but also um magic ink of course yes to me a lot of what magic ink is just about you know the eye is so much faster than the kind of logical um thoughtful brain totally so being able to like put the put the kind of understanding in the representation itself i thought was very powerful and then once you had that there's a bunch of interactive things that one could do too put the kind of understanding in the representation itself, I thought was very powerful. And then once you have that, there's a bunch of interactive things that one could do too, like what if scenarios or seeing inside what the each sub part of the equation. And so that was just, to me,

that was a kind of early exploration of those different ideas, many of which I'm still kind of hammering on today. Yeah, and I want to get to that in a second, but I just want to acknowledge that I think you do a really good job in that essay of exemplifying the thing we talked about earlier in this conversation from Brett Victor's Magic Inc., the interaction considered harmful. You show like you have one visualization where you can expand a term to see how sub-expressions are evaluated. So that's kind of like an interaction based approach,

but then you show like another way to view sub-expression evaluation is just in line right underneath in the kind of a smaller subtle view. It's kind of hard to describe over audio, but I think you do a really wonderful job of applying Brett's principle that like eyes are very fast and efficient. You don't need to like add any interaction to make it better.

It's actually better if you do away with that. Yeah. And maybe instead of better, I would modify that to be like it has certain advantages depending on what you're trying to do of course yeah um so i guess one question uh is uh about legible mathematics is like how much of it do you see as being about the like drag and dropy nature of of it as opposed like versus like

textual syntax or or is it like you're really not about that at of it as opposed to like, versus like textual syntax or, or is it like, you're really not about that at all. It's more about like, what, what do these symbols mean? Definitely the second part. What do the symbols mean? Yeah. The, to me, the, I sometimes get confused. I thought it was about both. Yeah. For me, it's a lot more about the, the kind of understanding of what the symbols are doing and that that has been a theme that has followed me throughout my programming work which is

it feels kind of unreasonable to me on a deep level that there's you know these symbols are doing something and it's totally understandable what they're doing but we don't really get any help from the programming environment trying to like help us tease out like know exactly what he's doing or giving some some feedback especially in real time because it's you know quite often I have the experience programming where something happened, you know, 10 lines past where a bug has happened, but don't realize it for a long time. And then I have to go back and like I lose all the context.

And a lot of these bugs could have just been caught immediately instead of having to wait a long time to do it, to catch it. I guess just to play a devil's advocate, I was recently rereading No Silver Bullet by Fred Brooks, which is a great pastime for people like us to reread Fred Brooks. And he argues that software is inherently unvisualizable because it's so abstract and, um, sure you could take simple symbols and show concrete values that they represent. But once you get complicated data structures and abstract data structures, uh, it's hard to

pull this trick. Uh, so how would you answer, like, how would you respond to that? I think that saying software is inherently unvisualizable is a kind of self-fulfilling prophecy, where it may in fact be that lots of software or parts of software, especially in programming system, have alternate representations, which give us better information about how a system works than the very symbolic word-based or string-based or graph-based, maybe is another way of saying that, complexity of software systems.

And I think it's partly true that software systems are just enormously complex, especially big ones like many, like Microsoft Word is probably like, what, 20 million lines at this point? Like some huge number of just straight lines of code that, as Alan likes to say, if anyone tried to read all of that code,

it would take them a lifetime. Like, no one can actually read that many lines of code, which, if not one person can keep a thing in their head, you know, there might be some problems there. Or maybe not just one person keeping their head but like it's just a huge huge huge amount of information um and trying to cut back on that complexity i think is a really important task and one way we have doing that is by working with representations and representations that give us certain information

that might help us understand something in an easier or more complex way. So you're saying that if we had better representations, they'd lend themselves better toward to visualization i would say i would say it's a research project that i think is quite interesting and quite worth doing to try and to make software systems less complex or easier to hold in the human mind maybe it's another way of saying that i think representations are going to be the big one that helps us do that. And those could be visualizations, but I don't exactly mean visualizations.

Like text is a visualization, for example, or representation. But code is a particularly not great representation in a lot of ways, especially for human understanding. Like a lot of programming languages were designed for kind of the machine first to run and the human to understand second. And a lot of what my work is feels like it's about is kind of trying to do the opposite. Starting with, okay, what is a good human way of understanding this stuff?

And then let's try and make a programming system out of that. Is that possible? It may not be. So in like the Brett Victor, inventing a principle sense, how I kind of want to give you an opportunity to like rephrase your principle. It. Because it sounds like that's kind of what

you're saying. Something in what you just said is your principle. Or maybe I'm putting words in your mouth. Oh, I see kind of what you're saying. Let me think about that for one second. I'm actually not super principled about this. I would say that my heart breaks when people aren't given the potential or when people's potential is shut down. And I see that happen a lot with programming in particular because of bad environments, bad design decisions, bad languages, things that aren't meant for humans

or not like, you know, make it easy for humans who don't already understand programming or already have a super logical mind to kind of wrap their minds around so for me it's a lot more about the compassion of not wanting to see people suffer like that not wanting to like close people down to programming just because you know they need to install some python environment and then somehow that doesn't work and you know beginnings are so fragile and so wanting to like really kind of hold that kind of getting the sense of like a baby bird like really like hold and cherish and like

give like the most possible support to um that is more where i'm coming from rather than a principled approach yeah i i could see that i have a clearer picture it almost uh to me like the uh installing of. And there was another phrase you used, something about like, like having to know programming or having to like deal with random programming artifacts. To me, like the phrase that encompasses a lot of those things is incidental complexity or like machine complexity versus like the things humans care about. So I like the way you kind of put this common phrase of incidental complexity in terms of very like human compassion and like nurturing yeah i don't i don't know yeah for me that's the most important thing it's compassion and like all the kind of creativity kind of runs from there.

Because otherwise, you know, how am I going to know what I'm working on? Like for me, I struggled a lot with how do I figure out what is a good thing to work on? And if I just come back to compassion and like really what do I want for human beings? Then I have a kind of a huge amount of energy to work towards human benefit. If I just kind of, if I just connect with that compassion. Cool. I feel like this is an interesting transition point to talk about flow sheets, because I vaguely remember you talking about some problems that you were working on or saw others working on, and you thought that you could build a better tool to allow people to work on those sorts of problems and that's kind of how flow sheets came about um in particular i remember you mentioned that it was like watching jonathan bloom programming that got you inspired to do flow sheets um i know that your obsession with watching jonathan bloom program is also like an interesting interesting thing to talk about so i thought it's not there oh jonathan blow Blow. Yeah. Blow, sorry. Yeah, so FlowSheets, I worked on it quite a bit at CDG.

That was another one of the projects. And I made a couple different prototypes. The first one was in collaboration with Alex Worth, most notably, who helped build a lot of the first thing. And more or less, this was me understanding that I had a bunch of different kinds of similar programming tasks that were a lot about data manipulation that felt really not easy for me to do in Python,

which is the language I use quite a bit, or JavaScript, or something like that. And I imagined it would be a lot simpler if I just kind of got direct feedback, like I type a line and then do a thing, and it does a thing, and I see if it works, and then if it does work, I can go on to the next thing, otherwise I've got to modify the next thing. Otherwise, I got to modify

the thing I just wrote. Or I can like try things out. Like I could just imagine doing those kinds of tasks in a much more fluid way. And I took a bunch, like a pile of these programs that I had already written in Python, and redesign them in mockups at first, and then in a separate programming environment to see what it would be like if, you know, what it would look like and what it would be like to program if I was always looking at data as I programmed. And I never actually didn't have the option of not looking at data. Like I always had to be looking at concrete, something did something, rather than normal programs which are tend to be very abstract, or can be very abstract.

And that's where flow sheets came from. And I made the first prototype at CDG and the second prototype later when I left in I think late 2016 or early 2017 I think is when I made flowsheets version 2 and actually I would like to say one thing before I get to the Johnson Blow thing which is one important part felt like I really over-designed the first system. You know, Fred Brooks has like the second system effect. I would say actually I did that the opposite way, where I really over-designed the first one.

It was like imagining how every little thing was going to work. And then in the second prototype, I was like, you know what? I couldn't actually make any programs. It wasn't fun to make programs in that environment. What would it be like if I just threw away a bunch of my ideas that I thought I cared about and just tried to see what is it like to program with data?

Even if I have to type in little bits of Python, that's fine. Just I want to see what it's like to have data and it turns out it feels really good especially when you're doing like uh kind of data oriented like data transformation tasks and i made like a couple dozen programs in flowsheets version 2 and it feels great it's become my preferred programming environment to this point so So if I have a task that's very data transfer me, then I prefer to use that because I can see what's going on. It's like very, very clear. Yeah, and that's it's quite a claim. I'll just put that out there. But I will also put out there that it's a very easy claim to believe in it once you watch the FlowSheets video.

So if anyone out there who's listening to this hasn't already seen the FlowSheets videos, I highly recommend it. And you will believe Glenn's claim that it is a preferred programming environment for data-intensive tasks. I endorse it. Thanks. I don't actually have a claim about other people wanting to use this environment. I just have my very pure claim is that I really like using it.

Yeah. What I'm trying to say is that I don't believe you're deluding yourself and you just like something because you made it. I think that most people will. It would be a preferred environment for most people. Yeah. Or at least for me, I can say that. Yeah.

It feels good to use. And the like getting direct feedback on the data as you create the program, it, it feels kind of like a no brainer at this point to me. This is like, Oh yeah, of course we need something like this. This feels really good.

Like why would you not have this? Yep. Instead of like a lot of times I have to like guess what my program is going to do. And then it's like hard to get the information if it doesn't do what I expected to do. It's even hard to tell if it did do the thing I expected to do. So just getting the direct feedback, it feels great. And I think there's a lot of potential there too. I'm actually starting a software business based on flow sheets version two, which tries to make a more, I'll say consumer ready, consumer ready, like more accessible. So you're not having to type in Python, for example, but you could do more like drag and drop kind of things oh interesting um i feel i like not not uh like you know i'm sure you know this better than i do but i i liked flush's version two you type in python for the

reasons that you said you know like you can still do a lot kind of like in excel you type the equal sign you type your little formula you know like you don't need it doesn't need to be much more polished than that yeah the one of the big problems or the maybe parts that are incomplete is that um a lot of times you want to like have the program affect something in the world like make a network request or write a file to a disk or save some kind of thing. There's a lot of things that you'd want to do or run on a server every five minutes, things like that. And FlowSheets was not capable,

or the prototype I made was not capable of doing any of those things. There are probably ways to do that, but I didn't develop the prototype to that point. Got it. Well, so that's actually something that leads me to something that I've been working on for a while. So I wanted to get your thoughts on it because it's very related to flow sheets. So flow sheets makes a lot of sense for like one-way data transformations. You like import, like in your examples, you import a dictionary, you filter out some words and you map some words that all works beautifully. And that's a lot of what programmers do. That makes sense.

But there are some things that programmers do like reactive systems where they're like cycles, you know, data is update, like shows you some view and then people interact with the view and it updates some data that interaction thing, but vector doesn't like so much always. Uh, so, or, or just like network requests, like there are things that are more real timey that, that aren't like one way mapping. So, um, yeah. Do you have thoughts on how you would keep the concreteness with things that are kind of flow through time. Yeah. I think this gets back to me for the Jonathan Blow video.

So Jonathan Blow is the programmer who made Braid and The Witness, these two amazing games. And he's also the designer of the games. He sometimes does programming streams, and he shows him making the game engine for his games. I got very interested in watching him program for like a few weeks of last year. I was kind of watching him program obsessively because I think he is probably one of the best programmers that I've seen work. He has this like very sharp mind.

He knows his code really well. He has a really good eye for subtle details, like how things are mapped in memory and how that's going to affect performance. And he has pretty good senses of when there's a bug, what kinds of things will be the roots of those bugs. He still has a lot of issues tracking those things down, which I think, to me, is an opportunity as a designer, saying, okay, here's something, even one of the best programmers that I've ever seen has issues with. What can we do to help this person? And I don't know how to do that.

I have one prototype in that's a, I haven't, that's like, you know, like 20% finished or something like that, that I may try and submit to a live this November. Oh, sweet.

I'll, I'm planning to be there. So I will see you there. Yeah, I'd love to see you. Yeah, it's kind of a mix between a REPL and a debugger that lives alongside your code editor, and this is basically trying to do the opposite of flowshe in some ways where or bring the best parts of flow sheets into a larger code base because if you visualized every piece every the data in every line of code in you know a 40,000 line code base like Jonathan Blow is working on then you're just not going to be able to understand what's happening. It'll just be too much information.

It'll be a lot of irrelevant information. And so that, to me, I began to wonder, how do you bring some of these really nice feelings of flow sheets, like the kind of direct feedback and this good insight into data with live being able to try a lot of stuff out very quickly? sheets, like the kind of direct feedback and this good insight into data with like live being able to try a lot of stuff out very quickly. How do you get some of those benefits into a really large code base? And so that's this idea for kind of REPL or debugger, or I'm calling it replugger.

We'll see if the name sticks. Replugger. That is an elegant name. Thanks. But I think it's important because here's this expert programmer who still has lots of problems programming. He is extraordinarily smart and gifted and works really hard to do all this yet even he's having problems so for me it's not just making programming better and more accessible for beginners which i

think is really important like really really important but i'm also somewhat interested in of how do we make experts better? Yeah, I totally agree. And I am also someone who like oscillates on both ends of this problem. Like how do we lower the floor and also like raise the ceiling? I don't know if raising the ceiling is quite the right metaphor because we don't want to increase what people can do.

We want to kind of like increase like their ability to do the things i don't i don't know um so yeah i i agree with you they're both interesting problems and i think you and i probably are in the same boat and that we hope that like there's a kind of that you could use the same solution for lowering the floor and raising the ceiling if that makes sense like you could build something that jonathan blow would like and it would also help you know like you know new programmers at the same time that's quite possible i i worry about the kind of competing design considerations in that sense so for me personally i want to work on maybe one or the other or chunks

one at a time of each but yeah i i share that vision like a artist uses a pencil and a four-year-old uses a pencil totally i guess the the opposite side of that argument is like a lot of professionals have professional tools while beginners have beginner tools in other industries. Like kids play with shovels, but like adults use forklifts or whatever. Right. There's like an increasing capability that can come. Yeah. And like complexity, like kids use iMovie, adults use Final Cut Pro or Premiere or whatever. But I guess, you know, to criticize my own argument, iMovie and Premiere and Photoshop 10, or not Photoshop, Final Cut 10,

all look very similar. Like you just add a few knobs and you get the more complex interface so that could be kind of the the solution like they look similar but the adult one has like a little bit different design criteria than the kid one yeah and i'm a little wary of using final cut as they'll kind of end all of movie production software because there's a huge amount of problems with it. I used to know someone on the Apple design team who was designing the next versions of Final Cut.

This is one of the friends of the lab of CDG. And hearing some of the stories about how people use it and Apple's internal design process, it is far from perfect is what I'll say. Totally, totally. Yeah, well, Final Cut X was kind of like a disaster. The first version of Final Cut X, everyone left and went to Premiere. Because I was a filmmaker at the time.

I was in high school, but I was doing film stuff um and so we all switched to Premiere um and then but then Final Cut X kind of like you know like improved enough that I I think a lot of us switched back but but I'm not really a filmmaker I was just a high school filmmaker so I don't really know what I'm talking about. But still there. And to maybe, you know, I have like little versions of Alan Kay and Brett Victor in my head now after studying their work for so long. But I feel like an important part is to study the heights of human expression. you know they're not just exclusively like accessibility is still important but also trying to understand like what are the kind of greatest things that humans have made and is it possible to support humans in making those great things um and like what what kind of things would they need uh totally like expert tools like how do we empower scientists like solve global warming i know brett's

talked about building tools for that kind of thing yeah exactly and i don't have too many opinions about that but i think it is worth looking at the kind of amazing expressive expressive qualities that humans have and trying to support that in all its all its glory from its beginning stages to its you know completed stages yeah i'm with you i i feel like uh brent and or alan would might say that the extremes are the best places to work like allowing kids like glowing the floor for kids that's great uh raising the ceiling for experts that's great the middle the middle chunk the like the people who are working on snapchat like we don't need to make their lives easier well yeah because they're going to make snapchat they already made snapchat you know they already made that's true like from

a research perspective anyway uh less than not a business perspective those Those are kind of already done and people are going to continue making things like that. Whether or not we have them. And so there's a precious few of us working on pushing, it feels a little grandiose, but really kind of pushing the boundaries of what uis can do and who can program and how they program i think that's important for you know

it's hard to see computers getting any less uh like pervasive than they are now and we kind of understand that there's an inherent power that comes in programming whether that's through expression in scratch or through like simulating climate models in python or c++ or something and you know i've actually gone and talked with a scientist who was working on who was studying meteorites. He was like one of the foremost meteorite scientists that happened to be studying at UCLA. And I asked him about his kind of software tool chain,

like how he did his thing. And, you know, if you ever want to be inspired, go talk to a scientist who does any amount of programming or data, data munging. It is, it is quite, it is incredible the kind of horribleness that, that you'll see like the kind of spreadsheets that are chained together with

people manually updating things. And it's kind of amazing that the science is actually able to get done given the kind of their bad tooling, their bad tools. Yeah, that, that makes a lot of sense. I have a friend who's a scientist. And yeah, he just uses Python or maybe I Python. But, you know, unsurprisingly he's able to make it work because he's a scientist. And he just uses Python or maybe I Python. But you know, unsurprisingly, he's able to make it work because he's a genius. So like, it doesn't matter that the tools are so bad. Like, you know, or maybe I flip it. Because like the tools for doing science are so bad. In order to be a good scientist, you have to be a genius. Yeah, I think that's a big part of it yeah so it'd be nice if more people

could be scientists um i one thing i wanted to touch on that we kind of skipped over is um when i so i asked the question of of how we make things that are less obviously visualizable visualizable and so you talked about your, uh, what was it? Replugger. A repl plus debugger. Replugger.

Replugger. Got it. I missed the P. Uh, replugger. Um, it's almost as hard to pronounce as your last name. I'm just going to keep making you say it.

Replugger. Replugger. Okay. So, um, uh, so that's, I So I think it makes a lot of sense why you'd want to build something like a replugger. But the tact that I usually take when trying to visualize the hard things in programming, or the tact I'm taking now in my work, is I'm just trying to restrict the model of what's allowed in programming or the tact i'm taking out in my work is i'm just trying to restrict the model um like uh of what's allowed in programming like get rid of mutable state get

rid of side effects like kind of make it haskally almost um because my contention is that when you really restrict things like when you restrict the dimensionality of programming the sorts of things you can do uh it increases it like uh it allows it to be mapped onto a 2d plane better so um i just thought this would be a good time to see if you had any thoughts about that approach oh yeah feedback one thing i've thought about lately is that in modifying programming there's like a lot there's there, at least from my perspective right now, there's only two tacks you can take. And one of those is change the model of programming in various ways,

like you're doing with mutable state, or when someone invents a new language, they're at least slightly tweaking the model of how a programming language works. And you can also change the UI or the way someone interacts with that model. And that's the place that I've been working mostly, which is trying to change the UI. So right now the UI is mostly like a text editor or something like that.

And the representation, there's another part of the UI is a bunch of code, a bunch of text or a bunch of words and symbols. And so the thing that I'm interested in doing is how can we, it kind of doesn't matter which model to me, So the thing that I'm interested in doing is how can we, you know, it kind of doesn't matter which model to me, it doesn't matter which model of programming someone uses,

but if it's well represented or if like it's, it might just be obvious what the model is doing if given the right representation. And the one example I have of this is a friend was telling me that in Perl, when you iterate through like a hash table or something like this, I forget exactly what data structure was, it does it by mutating a pointer

to the thing that it's iterating over. And so if you break out of a loop, then that pointer is still in the same place. So it didn'tating over. And so if you break out of a loop, then that pointer is still in the same place. So it didn't get reset. And so if you start looping over it again, it will start from that place. Oh, boy. Yeah, that's the that's what a lot of programmers say when they hear that. And to me, that might be a reasonable design decision like i think at this point we've learned that that's kind of you know that's not great like that's kind of a bad model but i could imagine that actually not being a terrible model if somehow it showed you as you were doing this like

hey it is mutating this thing and it might get stuck in a place and if it could show you that that my contention is that'd be fine. Like as long as the programmer is aware that's what the model is doing, then hey, we're good. Like they know it does that and then they can work around that or use that to their advantage

or whatever they need to do. Interesting. I'm with you where my disagreement is that when you have something like you were saying, like Microsoft Word, the 20 million lines of code, and you're stuck with a model where, sure, you can understand it line by line, and I can tell you what each line is doing visually, or I can give you the values of each of the symbols, but you still have to go through the 20 million lines, which is going to take more, more than the years you have on this earth. So, um, constructing visualizations that allow for like greater comprehensibility, I think is important for like regular people to be able to understand complex software. What do you, what do you say to that? I am glad you said that. Um, I, so to be clear, I agree that your tack of working with the model, I think that's great and important because this is like the, the argument about Roman numerals versus Arabic numerals for multiplication

where, you know, this is a little bit of a, I can't tell if the Roman numerals versus decimals is the model or the UI part. I can't tell which way you're going to take this. That's a good point. You know, yeah, it's kind of both.

But I was thinking of the model. Like, oh, I can see how that's confusing, actually. But yeah, I think changing the underlying model is quite important because the model is like the main way we have of understanding a thing. And more powerful models just like simplify things vastly or can do that. Give you leverage. But, yeah. So, okay. but yeah so it's okay so given that you you're with me there um you would the way you the reason

you're working more on the ui is is why um because almost everyone is working on the model is one part of it sorry i i think you cut, uh, saying I think almost everyone is working on the model. Oh, okay. Uh, like that's why every new programming language gets created. That's why people get really stuck in compilers and type systems. And there's a lot of talk recently about like react and non mutable state and

all this stuff. And I think a lot of that's good, but there's a lot less work in trying to evoke what a model does as opposed to the model itself. And so to me, that's an important just kind of... It's actually quite easy to come into this field and start working on something like that and discover something completely new because no one's ever done it it's just a very

um not really it's not really like a field yet um but back to your point about Microsoft Word and 20 million lines. Yeah. So that, to me, that's part of this research is, you know, yeah, I wouldn't argue that line by line showing data is a good way to understand what Microsoft Word does. My apologies for struggling. Oh, no, that's fine. Just being clear. But I would say that, and this is partly what Alan's Steps project was about, which is different ways of organizing that software may make it a lot easier to understand in the same way that like this is again from

Alan like the pyramids of Egypt were a bunch of rocks stacked on top of each other in the kind of the most naive way and it took a lot of suffering and slave use and human power and human lives to create. Whereas something like the Chrysler building or the Empire State Building, you know, went up in several months and it's, you know, like a fraction of the weight. It took a fraction of the time.

It's almost all usable space as opposed to the pyramids, which had like a tiny amount of the time. It's almost all usable space, as opposed to the pyramids, which had a tiny amount of usable space. And something happened between the pyramids and the building of the Empire State Building that we kind of somehow got a way of organizing materials and having new materials that made it way more

efficient way easier to do way more doable and there might be something like that for software and this is to me I don't have any good like pointers yet because this is so new like I can't really argue that this is definitely the future, but I think trying to make these software systems more legible, more... I think to me that's a big part of it is understandable because if you can understand something, it's a lot, you can change it basically.

Yeah, of course. I'm totally with you there. So I'm glad you brought up the Steps Project. But yeah, to dig in more to me the steps project is kind of more on the model side um like it looked like a lot of what they were doing was creating new programming languages new programming languages models um like that um that vector graphics library that they created like nile and that was like a tiny amount of code, but it was super expressive, like other abstractions,

as opposed to like legible mathematics type of visuals or Brett Victory visualizations. So I don't know. Yeah, I think that is almost all in the model. But then, yeah, Brett came in and made the representation, almost all in the model. But then, yeah, Brett came in and made the representation, the UI for the model. Because as he pointed out in Media for Thinking the Unthinkable,

that if you read that code, it's just a bunch of symbols. But there's actually a beautiful underlying model to it, apparently. Yeah, I've tried to use Brett's tools to get at it but I wasn't able to do it yeah but to evoke the model there is some like UI that isn't necessary to like show here is what the model is doing here's why this thing is happening is this is fascinating because I really now I could see that I can contextualize my work as really being right on the center.

Like I'm trying to create models that lend themselves well to being visualized. So like I kind of, I'm constantly going back and forth. Yeah. Trying to create a new model, see if it's visualized. And I'm realizing how unique that is. Like people usually work on one side or the other, but I guess there's some people like you and me and Brett

and Jonathan Edwards who jump back and forth yeah and for me i would say i mostly work on the uh yeah representation of ui side like any to me any programming model is probably fine like it could be fine depending on what you're trying to do but yeah trying to understand and evoke what it's actually what the symbols are doing or how the model works or that there's a model at all because most beginners don't even get that when they come into a programming language yep yep um cool yeah this was a great great great little tangent discussion um but i think i want to move on um just in the interest of time to talk a bit about dynamic land and and and laser socks and um your adventure is there sure cool so um let's see i don't remember where we left off the conversation in terms of

how cdg turned into dynamic land i feel maybe Laser Socks is a good place to start. Oh, yeah. And I realized I didn't really describe CDG super well. So the one sentence summary is that me and the handful of other people in the group all made dozens of prototypes from little know from little things like taped together to kind of more software things trying to get at what this dynamic medium thing that brett was describing might be and so we had hypercard in the worlds which was set up throughout the room as a system that we could kind of play with and add things to.

And one of the researchers, Haim Gingold's decided he wanted to have a game jam, which is like, you know, like a one day kind of thing where we all just use this new system to make games in it. And that was really fun i should just say like we all like i think um about 10 people in total all created a bunch of different games in the system all kinds of different games

and it was interesting they were all multiplayer because they're all kind of out in the world we didn't like say that at the beginning but they all just kind of became multiplayer because that's what you do when you're in the physical world is you interact with other humans generally uh so we had all these multiplayer games and that's where laser socks came from where we had a lot of the system used laser pointers to do interaction and so we could like point to something across the room to make something happen with the laser pointer and i was like oh you can't there should be like a dueling game where you like shoot lasers at each other. But that's probably unsafe.

You'll probably hit people in the eye. And these were like pretty high power laser pointers. So I was like, you can't shoot at them. Oh, what if you just shot it at their feet? And so that's where the idea for Laser Socks came from, which is a game where you stand on these two game pads and shoot lasers at the other person's feet and they try and shoot a laser at your feet and once they hit it a camera detects that they hit it and it fills up a bar and once your bar is full then you've lost so whoever whoever whoever has their bar fill up first loses interesting i that makes a lot of sense that you thought like laser pointers and eyes are bad

and then like feet are kind of far from eyes. Yeah. Yeah. And so go ahead. I was going to say, so it's detecting that the laser pointer is not on this. How is it detecting that the laser pointer is on the socks? Oh, this is actually kind of one of the magical things.

I had a conversation with Haim about how do you get the laser, how do you get the system to detect when a laser hits a sock? And somehow, we both don't claim credit for this idea, but somehow this idea happened, where because the cameras had their exposure really low, and we're basically just looking for bright dots, we could use that fact to have a black gamepad, and so when you shoot a laser at the black gamepad,

it doesn't reflect very well because it's black, and when you hit, we had to buy a bunch of white socks and when it hits the white sock it lights up really bright and the camera sees it and so once that happens it once it reflects off the white sock and it reflects bright enough for the camera to see it, then it detects a hit and increases your, you know, your score. Oh, so easy. Yeah. It doesn't have to like know where the socks are in space and where the laser is in space. It's just like, is there a circle in this square or not? Exactly. Is there a bright dot in this particular square? And to make this system, it was incredibly easy.

It took me an afternoon to build it. And I basically taped some trash bags to the ground. And then you use the laser pointer to draw a rectangle around the trash bag or the gamepad. And then you say that that basically tells the system, this is an object of interest. And then it detects that object. And you can do things with that object. Like if there's a laser within this object, do something.

And so I had to write a line of code, or what would be a line of code. It ended up being like a dozen or a couple dozen lines, just because the system was imperfect. But you could imagine writing one line of code, which is when a dot is detected in this square, or when a dot is detected on this object, because it's an object-oriented system, increase the score. And then the score, you know, keeper thing just draws a rectangle, which is like draw a rectangle when the score increases.

And to me, that was like an interesting, from the future of programming perspective or the future of coding perspective, that was like a really interesting uh i'll call it a discovery it doesn't exactly feel like i made it but like i could program this thing that's a pretty new and fun game in an afternoon with trash bags and cardboard and laser pointers and socks and i could do that with like two lines of programming and it may like point out to me that there's like there could be something real here um or

something really interesting in bringing a system like that to a wider set of people like i wonder about the kind of creativity that would be unleashed if such a system like Dynamic Land had made it out and had a good corpus of information of like, here's the kinds of things you could make. Go nuts. Yeah, I agree with you that LaserS socks really does kind of point towards something like uh the ability to create multiplayer fun creative things very simply um there was an analogy someone used that i was looking for in preparation for this interview i couldn't find they compared

laser socks to something like it was like the killer app of dynamic land that kind of proved it do you remember what i'm talking about right yeah um brett said that laser socks was our space war and he was referring to um space war the like classic one of the first computer games that all the uh DARPA or sorry the ARPA researchers were playing and modifying and sending around it was like one of these like amazing this is like a really simple game about like shooting each other's spaceships on this really early prototype screens and stuff but Stuart Brand wrote an article about space war, about how like incredibly addictive it was and how people were like staying up all night to do it. And it kind of, it was like a Harbinger, um, of like,

here's something really new that computers could do that no one's ever really seen before. And that's the analogy Brett was making of like here is like a kind of game that's really fun at dynamic land like uh in this new medium that is pretty simple but it has this like interesting quality yeah I don't know if I quite agree with that analogy, but I'm flattered. Yeah, I can imagine. Brett's giving you quite the compliment there. So yeah, I'd be curious to hear your thoughts, pros and cons,

criticisms about dynamic land. Maybe if you want to go into why you left, that'd be interesting too. What do you think about the future of what, what dynamic land tells us about the future of programming? Yeah. So dynamic land was, so CDG started, it became clear that our funding was running out. And

I want to say in a really quick aside that the funding was pretty innovative in that it was just given to us. It was just money given to us with nothing expected in return. That's my understanding anyway. And that is the research model that Alan Kay would like to see more in this world of researchers who, you know, have some interesting ideas to just kind of let them go and follow their vision. And at the worst case, you know, you've paid their salary for a decade or something. In the best case, you get things like the internet or, you know, interactive computing or laser printers or all this stuff that happened back in the ARPA days. So it's a pretty small bet in comparison to like the huge amount of stuff that can come out of it. And I share some of that.

So that's just something I want to mention. But back to CDG. So the funding was running out and the whole set of all the researchers could no longer be supported by that funding source and so the dynamic medium group Brett Victor's group started trying to spin out and find different funding and kind of continue their research and that's how dynamically got created which is a nonprofit research group at this point and they have a building in Oakland and I was there for kind of the very last parts right before they they kind of had an office across the street from the current building that was under

construction and I left around kind of when it like the it was being designed and planned um and the reason i left was uh had really nothing it was the group was switching um roles a little bit to more the design engineering of real talk and to me that wasn't so interesting it felt a little bit more like kind of nitty-gritty flipping bits than i cared about yeah a little bit more engineering than i cared about and also my interests were shifting from computing at that point anyway and i was getting a lot more interested in maybe i'll say human happiness um yeah you dropped the uh incidental complexity stuff and you just focused on compassion and it's more pure sense yeah um like i personally i examined my work really closely and i thought

what would happen if this work succeeded wildly like if it like really took off and like the work that i personally was doing what would happen as results and like what kind how would humans be affected and really i asked would they be happier and i realized the answer to that was not really like i would like it if programming is more accessible and better for pro for expert programmers i would like it if people are able to make things like laser socks. I still think those are good and important. But for me, anyway, they didn't really have the like, grab like we all we have, you know, this, this is in some ways the age of wonder the we have these incredible things around us, like, we have supermarkets full of food, we're like, really, you know, pretty cheap food, all things considered. We have like amazing access to resources and plumbing.

And we have like this incredible entertainment and these amazing computers. And yet it doesn't seem like we've made a real dent in human happiness. So I wanted to work more directly on that. So i left to explore a bunch of things and decided i want to be a therapist and that's actually one of the main things i'm doing now to be a psych psychotherapist yeah fat fascinating pivot uh my girlfriend and i like to say uh like hard pivot whenever we like come up with a drastically different idea than we were thinking about um so one phrase so you use the phrase that like make a dent in the universe in terms of human happiness and so that makes that like would lead me to believe that you don't or while you may want

to do psychotherapy like on a one-on-one basis there's like maybe some sort of like a vague plan to scaling whatever you learn to achieve like large amounts of human happiness or creating a methodology or something. Or am I putting words in your mouth and trying to make a startup out of what you're doing? You're just trying to be a therapist for people one at a time. I have some thoughts about that I realize this isn't exactly about the future of programming so I'd be happy to go back soon but just to comment on that I think it's

it's worth not underestimating how important it is that one person has found some happiness. Like to them in this like one lifetime is actually, you know, it's the most important thing to that person to be happier. I think that's what, you know,

we're all going for. And so I try and frame it instead of thinking in this very like industrial, like, you know, you've inherited a kind of scientific industrial idea that like, we have to scale all of our efforts to many different people. And I think that's important and true in a lot of ways. But I also, and I'd like to do that. I don't know how to do that exactly for something so individual as therapy, yet I'm planning on working on it.

But I also wanna say that one person's happiness is actually quite important. And not incidentally, that one person being happy affects everyone around them in very very concrete ways they're not even that subtle a lot of the times and there's like a kind of wave that happens so that is my answer to, or part of my answer to, um, totally question. And I'm totally with you. Um, I just, just to kind of follow up, I just want to check, do you have any vague sense of scaling?

Like, I don't mean to say like, to, to like, as if I didn't hear what you said, I understand that helping one person do being, being a teacher or, or a psychologist has an enormous impact in the world. Um, just knowing, um, you, um, I was wondering if you hadn't just, if you had any hopes or vague sense of some sort of kickover. I don't know. Yeah, I do. Okay. Yeah, I want to bring happiness to everyone that I can

or help people find happiness. maybe it's another way of saying that and I would like to do that you know I I can't do that just by talking one-on-one with people so I'd like to put an eye towards or at least like some of my effort towards figuring out ways to help more people. One idea just to give you is to work as a kind of in-house therapist for organizations, maybe as a consultant or something like that. Maybe organizational therapist or something like that, which is a person helping the organization to achieve better results just like a therapist does for an individual

and also working kind of individually with the people in organizations so i think organizations are like the place of adult learning in our society the kind of most commonplace just like schools are for children i think the workplace environment can be a space of the primary space of adult learning. And so I'd like to help cultivate a culture of, you know, a growth culture or a personal growth culture within organizations. Cool. Yeah, that's fascinating. I watched the first episode episode of the tv show billions and i think there was a um organizational psychologist in that show it seemed like a

fascinating job yeah it's pretty interesting i have not yet done much with it yeah totally okay so to bring it back to programming um I wanted to give you some space to say what you liked and didn't like about Dynamic Land and how it points to the dynamic land exists. I think it's great work and it's really worth doing. I'm really happy that Brett and the team have found the space to do that. Personally, my heart goes out to them and I wish that I wish that the funding was easier I've heard that it's hard to get funding first you know it's like such a unusual kind of space that I my understanding is that they're looking for funding a lot of the times so I would really like that organization to keep continuing to do their thing because I think Brett is brilliant.

The team is brilliant. And I think it has a lot to say. I don't know if it has a lot to say about the future of programming necessarily. I think a lot of interesting programming things will come out of it. Like they, the team has been quite interested for a while now in making languages that are more like physical languages that sit on a table as opposed to code that is on a screen.

And they also invented like a kind of operating system that is pretty a screen. And they also invented a kind of operating system that is pretty amazing, Realtalk, that is kind of like an operating system kind of protocol, maybe programming language-ish

kind of thing. So I can imagine lots of interesting programming innovations coming out of just the team iterating and working on creating the next versions of Real Talk and Dynamic Land. And this feels a little... I'll say that something I worry about or wonder about is the technology at dynamic land being matched up with the ideals of dynamic land like i think brett in particular has had an

emphasis on um artists and inventors and scientists. And I think in Dynamic Land that shifted a bit in order to be more about everyday people, accessibility, making sure that the community can be involved. I think that's really important for programming in particular. But I do wonder about the kind of like, what exactly is the technology at Dynamic Land good for in some sense? Like what kinds of human experiences does it facilitate?

Is it just things like laser socks? Is it things like, you know, spreadsheets or business plans? Is it things like community organizing? Is it things like political intervention? Things like teaching, like lecture style? Like, I have all these questions about kind of how it's going to spread into the human world. But I'm more just excited to see where that goes. I don't know where that's going, and I hope it goes someplace good.

Yeah, totally. Me too. Cool. So the last thing I want to talk about is that your current project you mentioned that you're working on an interactive visualization of a library that you built yeah that's just the most current thing that i'm working on i started while working at the recurse center for one of their week-long batches in New York City, which I highly recommend. That place is awesome. Everyone there is really cool, and it was a nice space to work, kind of head down and just work on a thing for a week.

So yeah, I'm working on this representation of a JavaScript library that I ported to JavaScript from Python like five years or something ago. And I've never quite understood how it worked, even though I ported it. And yet it still kind of works. But every time I have to try and fix it, like fix a bug that someone reports, I keep running into, wait, what is this doing? How is this laid out?

Why is this code doing it this way? What will happen if I do this kind of thing different? And that's me, the maintainer of the library and so i can only assume that anyone who tries to like understand how this code works is running into even more issues i think this is a big problem is that for a lot of open source software it's open source in the kind of you can see the code but you don't really have insight into how the designers of the library were thinking about it, unless they have stellar documentation,

but that's kind of its own separate thing. And it's basically just hard to read code and understand what a thing is doing and why it's doing it the way it's doing it. And so... Totally agree. I feel like there's... I'm kind of working on an essay about this exact thing like code is open source but who cares nobody can understand it yeah and it's

useful like people do stuff with it but like as soon as you run into problems with the library it's just like looking into the code is a mess uh yeah there's just a huge number of problems with it so i'm trying to make a representation of this library that's like a little bit of an interactive explanation, but kind of shows the internal workings of the library, kind of like a, instead of like a black box, I think of like a clear box or something. Like the library is its own kind of artifact,

like its own separate thing. But within that, it has its own internal workings. And so I'm trying to just evoke those internal workings and really allow someone who doesn't know anything about how this library works not only to understand what the library does, but how it does it and how the different parameters work. Because this library has some parameters that you can tweak that changes the behavior of the library. You kind of can't really understand what they're doing unless you see the internals, because it's based on a mathematical model of cosine similarity.

So basically it just explains a little bit of the mathematical model and then just shows in both in these kinds of tables and interactive views so you can change things around and see how the behavior changes. And there's lots of like pre-built examples that you can play around with to understand what's doing. And it also looks like a fake, it looks like a GitHub page. I basically copied, saved a copy of a GitHub page and then just modified the documentation part of the page because GitHub doesn't allow you to like embed javascript in the documentation so i had to like fork off a page and it's it's kind of like a cheeky like okay i'm pretending this is github even though it's not but like this could be on github

like what if every library that you used had, like, some interactive documentation or interactive representation that allowed you to, like, play around and see what data was flowing through the program? And the author could maybe, like, add annotations being like, oh, and this is what I was thinking here and this kind of thing. The dream. being like oh and this is what I was thinking here and this kind of thing the dream well I think the final dream that this is pointing at is like I want to start with a representation that is like me trying the best I can to explain how this thing works in the best possible way that I know how and so it's very handcrafted. It's not automatically generated or something like that. But starting with at least one good example of a well-explained system, is it possible to get any lessons from that to be able to explain other systems with the hope that these things may just be automatically generated in the future or may become the way that the programs are authored in the first place working with a really good representation of that system that's really human understandable

instead of you know symbols and code i stand corrected that you're correct that what you just said that is definitely a dream yeah it's that's my dream, or at least something I'm trying to point out with this library. Totally. No, I have the same dream. We're co-dreaming here. Awesome. So in terms of this and future work from FlowSheets and the FlowSheets SaaS startup and et cetera. How should people follow you? What sort of ways are you interested

in interacting with listeners of the podcast, et cetera? Oh, I'm on Twitter. My Twitter is at Glench, G-L-E-N-C-H. I'll post about random programming things that I do, including this representation I just showed. I also have a newsletter about the future of programming on TinyLetter, and that's tinyletter.com slash flowsheets.

And I'll post things about the different... That's the main kind of venue. I'll talk about the different prototypes and things I'm coming out with. And my website is glench.com, G-L-A-N-C-H.com. And that's the kind of my repository of everything I've ever made. Yeah. And I don't know when my SaaS startup, my programming language startup is maybe another way of saying it.

I don't know when that will be coming out with a thing, but hopefully before the end of the year. Is there anything you're looking for, like investors, collaborators, employees, I don't know, early customers? Anything you're looking for? Is it too early? That's a great question. Thank you for asking. No, I'm not really looking for anything at the moment. I'm in the middle of doing a lot of customer interviews and really trying to understand what people are, what kind of data manipulation tasks people are doing,

especially in an automated way or wanting to be in an automated way. So things like I want this data manipulation task to run every 15 minutes and email me if some certain conditions happen. So if anyone has any tasks like that they'd be interested in telling me about, I'd be happy to hear them. Cool. And people should get in touch with you via Twitter, email? Yeah, Twitter works and email. My email is on my site. Cool. Great.

Well, Glenn, it was a pleasure. Thanks so much for giving me so much of your time. Yeah, this was awesome. Thanks for having me, Steve.