Future of Coding - Ella Hoeppner: Vlojure

Episode Date: January 5, 2022

Today's guest is Ella Hoeppner, who first came onto the radar of our community back in the fall when she released a web-based visual Clojure editor called Vlojure, with a captivating introduction video. I was immediately interested in the project because of the visual style on display — source code represented as nested circles; an earthy brown instead of the usual dark/light theme. But as the video progressed, Ella showed off a scattering of little ideas that each seemed strikingly clever and obvious in hindsight. You'd drag one of the circle "forms" to the bottom right to evaluate it, or to the bottom left to delete it. The sides of the screen are flanked by "formbars" that hold, well, whatever you want. You can reconfigure these formbars to your exact liking. Everything is manipulated with drag. The interface exudes a sense that it was designed with wholly different goals and tastes than what we usually see in visual programming projects — perfect subject matter for our show. This episode was sponsored by Glide, and the transcript was sponsored by Replit — thanks to them both for making this possible. The show notes (with copious links) and transcript are available here: https://futureofcoding.org/episodes/054Support us on Patreon: https://www.patreon.com/futureofcodingSee omnystudio.com/listener for privacy information.

Welcome to the Future of Coding. I'm Ivan Rees, and I'm going to just pretend for this episode that this is your first time listening to the show, because, you know, it could be. So what is this show? This show is from the Future of Coding community, which is a group of people who gather together on the internet around the singular vision of wanting to completely change what it means to program a computer. But of course, that singular vision is not at all singular. Every single person in the community has their own idea of how programming should be completely reimagined. This podcast gives us the opportunity to kind of go around and talk to a bunch of different people with different visions about reimagining programming and get their thoughts or to reflect on the work that they've done or to look at some interesting bit of history that might inform the work that we do as we push towards a better, perhaps more imaginative or my favorite weird future. Today's guest is Ella Heppner. And Ella first came onto the radar of our community back in the fall when she released a project called Vloser, which is a very

novel style of visual interface for the Clojure programming language. And it features these nested circles to represent the nested structure of Clojure's Lisp code. So whereas in Lisp you have parentheses that are used to group together expressions and nest them to form larger expressions, in Vloser these are circles nested within circles, and it sort of gives a visual spatial character to the code that is there in the Lisp, it's just not there in the text in the same way. And Vloser also does a bunch of really interesting things with the user interface. We will get into all that and some of Ella's other very interesting work in the interview. I want to thank our two sponsors for

today, Glide and Replit, and you will hear more about them at the Midway sponsor break. Without further ado, here is Ella. When I first learned coding, primarily, I was learning it because I was interested in doing generative art. Actually, that was kind of the way that I got into programming initially. In the high school that I went to, I was lucky enough that they were able to like give all of the students laptops. And that was wonderful for me, because, you know, when I was sitting in class, instead of listening, I could be doing something actually productive and, you know, learning to code. So, and, you know, I would, you know, sit in class and make little like generative art backgrounds for my computer and other people's computers. And that's kind of what got me into coding in the first place.

In terms of like my professional background, I went to Virginia Tech, graduated from there about two years ago with a degree in computer science. Up until a couple months ago, I was working at the University of Virginia in the School of Medicine, doing like web development stuff for research into online health interventions. And then for the last month or two, I have been taking some time off because I've got a couple of projects that I'm working on, Lojur being one of them, that I wanted to have some time to work on. Since it sounds like you learned how to program kind of in the later part of high school, and I think by that point you were probably, you know, like well aware that programming was a thing, do you remember at all what you thought that programming was like back before you actually

learned what it really was and how those two things compare. Hmm, I can't remember anything like any particular impressions of what programming was like, other than just it being like, very, you know, opaque, and kind of, you know, almost, almost magical, you know, like, there were people called programmers, who did, you know, some, some magical thing that made computers do interesting stuff. But, you know, beyond that, I had no idea of the details. So it turns out it's in reality very much like you expected. Exactly. Yeah. Yeah. There's a lot of dark magic, at least. Yeah. I have this tendency when I when I write the questions for one of these interviews, to leave the very heavy, involved, deep stuff

to the end, and then kind of front load the interview with a bunch of lighthearted, fluffy, fun questions. And what, what inevitably ends up happening is we spend, you know, two hours talking about some wild tangent from one of the light, fluffy questions and then never get to the actual real meaty stuff. So I am, I am going to want to get into talking about Vloser pretty quick, but just before we do that, the one kind of lighter thing i wanted to talk a little bit about were the uh generative art youtube videos that you've made there are three of them so far for one as a as an act of listener advocacy i want to suggest that everybody listening to this actually go to the show notes for this episode i will have links to these videos i might even like embed them on the show page because they are absolutely beautiful and the music in them is just delightful

you could just like zone out and look at them and the generative art that is going on in them is just beautiful to look at and the music you know accompanying them works perfectly but then you're also talking through how the algorithms work and it has this very hypnotic quality to it. And I've just absolutely loved watching these videos. And it made me wish that there was like an infinite stream of them where I could just, you know, put them on and have it run for hours and hours. And so in in pursuit of that kind of a goal, maybe, are you planning on doing more of these kind of videos? What did you do to make these first ones it seems like they took a lot of work where did these videos come from and and do you think you'll

be carrying them on into the future uh yeah i'm certainly planning to um and thank you for your kind words it's uh very nice to hear you say um and all the feedback that i've received on them has been uh really wonderful so i'm absolutely planning on continuing. I've been doing generative art for a long time. And you know, I don't use like social media stuff all that often, I just find it kind of like boring. So like, I would rarely like post a lot of the generative art stuff that I did. Oftentimes, I'd, you know, show it to a friend or something, and then it would just sit on my laptop for, you know, a year or something, doing nothing with it. And eventually, I thought, you know, I should, I should actually publish some of this stuff and make something cool out of,

you know, all this time that I'm spending. And so that's, that's what the idea behind the YouTube channel has been. And they have been a lot of work to produce. And it's been about maybe two or three months since the last one. And since that one, I've been trying to rethink the way that I'm producing the videos in order to kind of make them less taxing to produce. Hopefully since that one, I've been trying to rethink the way that I'm producing the videos in order to kind of make them less taxing to produce. Hopefully that I can, you know, be able to pump them out faster, create that infinite stream of generative art content that you're talking about. But yes, I'm absolutely looking forward to making more of those in the future. And just for my own curiosity, what's the process like to make one of those videos? Because there's like a mixture of animated examples, and then also some cases where

you're actually like drawing out what the vectors are doing. It just seems like there's a lot of different elements that each need to be separately created and kind of brought together. Yeah, so generally, the process just starts with me having some, you know, topic in generative art that I find interesting. And I'd like to, you know, make a video explaining. And I will just kind of start making some code, writing up, you know, whatever is necessary for the piece. I just kind of make a couple of variations. Like, you know, the first video that I did was a video about flow fields, which are kind of a classical technique and generative art. Tons and tons of artists have used them, but they're still really beautiful

and interesting. So I thought that that would be a good place to start. And for that video, once I knew that, you know, that was going to be my topic, I just sat down and coded up a flow field algorithm. And once I was happy with the way that it looked and everything, I started writing a script and I would go through kind of different various examples of how the algorithm could be used. So I start with, you know, the coding and making the art. And then retrospectively, I kind of go through and try to find out what would be the best way to explain this. And then sometimes that involves, you know, writing more code or making diagrams, you know, like you said, drawing stuff out to kind of help people understand what's going on and then yeah film it record it get a friend of mine to make some music for me and uh that's basically the process by the way you were mentioning you liked the um the music

to them and if anybody is interested in in uh in that music then you should check out uh inconvenient body that's a friend of mine who does all the music that's been in my videos so far so i'm really grateful to her i'm gonna do the worst thing that every music reviewer does. And as a musician, I hate this, but as a person listening to music, I love it. It reminds me very much of like the early Aphex Twins sort of selected ambient works. Having that kind of music accompanied with this really beautiful generative artwork is just like such a such a great mood to find oneself in so i've i've really enjoyed discovering the work of this musician through your videos as well so i'll have a link to their their stuff in the show notes also i'm a huge fan of apex and so is my friend so she'll be she'll be very happy to hear that what tools are you using to do

this generative art is it like a like a closure library or using processing or what kind of what kind of generative art tools are you fond of in this work? So yeah, I do most or really all of my generative art nowadays in closure, or usually closure script for the videos that I've made so far. And I'm using a JavaScript graphics library called pixie.js as one of my main, you know, graphics libraries. And I also do some stuff using various Clojure graphics engines. I've been using one recently called just Clojure 2D, which is just a great little, you know, pure Clojure graphics library. I've used a ton of different graphics libraries over the years, and I'm not particularly picky. I can just, you know, find something that works, build my own wrapper around it and get going. So speaking of closure, let's get into the,

the reason that, uh, you came onto my radar and why I wanted to bring you on the show, which is your recent project Vlosure, which I, I would normally do a little spiel where I'd explain what it is, but I, I actually want to let you explain it, like what it looks like and what it would feel like to work with it for somebody who is listening to this and might not have seen your video introducing the project. Sure. Yeah. So the basic idea behind Vloser is that it is a programming interface for programming in a normal Clojure script, right? So there's nothing special to the language itself. But what's special is the interface. So you know, normally, writing Clojure involves a text editor, you have, you know,

nested parentheses, and you know, brackets and all that kind of stuff. But in Clojure, the hierarchy is instead represented with a structure of circles. So wherever you'd have a pair of parentheses in normal Clojure, you'll instead have a circle. And kind of subforms within larger forms are represented as smaller circles within a bigger circle. And of course, since Clojure includes not just lists, but also things like vectors and maps that have their own special kind of encapsulating characters. Vectors are represented as octagons in Clojure. Maps are kind of represented as these circles with little spikes on them. And so basically, Vloser is a sort of drag and drop visual interface for programming ClojureScript. It's hard to describe the idea of it in words. But thankfully, it's

up on Vloser.io. So if anybody is listening to this and can't quite picture what I'm describing in their head, just go look at it for yourself. I worked a lot on trying to make it very intuitive and easy to pick up for people who already know ClojureScript. So it should be very easy to kind of see what's going on if you're familiar with Clojure. the demo video that you created in the show notes for this so that anybody just happening upon the website for this podcast can watch that presentation you gave, which serves as a really wonderful introduction to this environment. It's a really concise video, like I think it's only like eight minutes or so. But even in like the first three or four minutes, you cover so many really interesting things that feel like they could be explored in great depth in a longer form. So that's why I wanted to get you on the podcast. Talk a little bit about how you

arrived at this idea of evaluating by grabbing a form and dragging it to one of the corners. Was that something that just occurred to you? Was that something that you tried some things and then arrived at this one? How did that come about? From the start, I knew that, you know, this was going to be a visual interface. And the most natural thing that popped out once I kind of had this idea of, you know, representing S expressions as circles with smaller circles inside, was that clearly this interface would be great with like a drag and drop system, rather than, you know, any kind of other editing system. And so when you're building a program in Vloser, you're sort of dragging circles around, moving them around, copying them, deleting them.

So just naturally the whole thing lends itself to a drag and drop based workflow. So with that in mind, the idea of just kind of having this REPL zone or eval zone where you can just drag any piece of code and see what it evaluates to just kind of seemed like the most natural thing to do once the basic idea was there. And deleting things goes the same way. In the bottom right corner of the screen,

there's the eval zone. And in the bottom left corner, there's a discard zone. And if you ever want to delete something, you just click on it and drag it to the bottom left corner. So I tried to keep the whole thing very drag and drop focused, in part because I want this to work not just on computers, but also on mobile devices. I'm aiming for this to be a very cross-platform tool, that kind of drag and drop system seems like the best way to go. It's nice because it's not just cross platform but it's also like cross interface paradigm or whatever term you want to use like using drag and drop as a way of of invoking action

rather than something like command return or whatever like you get out of more traditional repl is nice because it's just another way of bringing some spatial character into the way that you interact with this programming tool and it feels like that's sort of a missing ingredient that a lot of programming tools have now that we're kind of into the mobile world and maybe someday into the vr world where if you want to be doing programming that is fully sort of fluid and conversant with the medium that it's in if that medium is very spatial in nature the programming also needs to be very spatial in nature. And so having that way of taking some of the very familiar conventions from a traditional REPL-based environment, but just adding that little bit of tangibility by making everything about dragging and dropping

just feels really nice. It's one of those things that seems very obvious in hindsight but it's the kind of thing where if it was so obvious in hindsight like we'd have so many more tools that are doing it and so that's why i think lozier is really interesting especially because you didn't just stop at grabbing a form and dragging it to the eval corner and then there's your result or to the discard corner and then it's gone those two corners also hold on to the last value that was put there so in the eval corner it's whatever the result of the last evaluation was and in the discard corner it's whatever you discarded and so in the discard corner it acts almost like an undo stack if you change your mind you can grab something back out of there and bring

it back was that something that just kind of occurred to you as you were building it? Or how did you arrive at that idea? What sort of led you to have that as a capability? Like you said, you know, the idea behind Vloser is a relatively simple one. It's really the core is just a way of representing S expressions as kind of like this little visual circular structure. And I feel like, you know, once you have that idea, the rest of Lozier just kind of writes itself. So, you know, once you've got the drag and drop little REPL zone, you know, of course, that's going to display whatever the result of the eval is. And so it just seemed obvious that, you know, if clicking on that and dragging to some other area,

that's not going to do anything anyways. So, you know, we might as well make it so that doing that drags out the result of the evaluation. Yeah, that just kind of jumped out to me. It seemed like the obvious thing to do. And one thing I should say in the future, I'm planning to make it so that you can pull out not just like the last eval result or the last thing that you've discarded, but you'll be able to click on the eval zone or the discard zone and see like a history of all the evaluations that you've done or all the things that you've discarded and pull any of them back out into the main program. Yeah, that would be super nice. That seems like another one of those things where once you've added the feature of

being able to pull things back out of there, it's just like the next feature writes itself. And I love seeing environments that are designed that way where it's like, hey, if we do this one little twist, then it shakes all of this fruit out of the tree. And then this is a terrible analogy because it's not like after that fruit falls, the rest of the fruit is easier to get. But it's the kind of thing where you can just keep pulling on the thread and like more and more and more cool stuff comes out. So I like that a lot. like I'm happy about this environment. It's because it's like giving me this feeling of excitement about like, there's so much possibility here that I really love seeing whenever I find one of these new environments that has some fresh ideas in it that are waiting to be explored. So it's always fun to get to look at one of these things as it's early in its evolutionary process.

Yeah, no, absolutely. I kind of share the excitement. You know, I feel like I'm kind of in the same position as the rest of the people who are seeing this where, you know, I didn't do anything particularly special to come up with the idea, you know, it just struck me. And, you know, I've just kind of been doing the obvious thing ever since, you know, I came up with it. So yeah, I'm very excited to see where this goes to. And more than anything, I'm excited to see kind of the ideas that other people have on like how to expand Flosure in the future. So far this has just been like a solo project

but you know some people have already forked the repository on GitHub and stuff so I'm very interested in seeing you know what directions other people take this. Have anybody shared any good ideas that you're particularly excited about playing with? I've gotten tons of great suggestions about different things to add you you know, just little, like UI features, like, you know, you

should be able to drag form bars around or stuff like that. Speaking of the form bars, let's talk about those, because that's another part of the interface that feels like it's a very simple seed of an idea. But the way that it sort of manifests is, I think, quite rich and quite nice. There's a little sort of, you know, edit the interface mode that you can go into along each of the four edges of the screen. At the center of those four edges of the screen, there's a little plus button, and if you click one of those plus buttons, it'll make a little bar appear along that edge uh with a couple of like empty circles in it and then uh when you're back and working on your lozier program you can grab any form from your program that's the sort of nested construction of circles at the center of

the screen and drag it down onto the form bar along the edge where you where you click that plus button and made the little form bar appear and you can have these form bars along any any side of the screen and you can also when you're in that edit mode add a new form bar kind of on either of the three sides sort of like the like the hemisphere away from that edge of the screen and keep stacking these form bars up and make um you know like a larger sort of interface element there if you want to be saving a lot of stuff and i think in the video that you showed you have form bars along the left and right sides of the screen that have a quite a few different things in them like some common closure functions that you might want to use how how are

those here i go i'm trying to find a way to repeat the same question that i've asked three times already but uh uh here's what i'll do i know how i'll approach this i i have a two and a half year old and her favorite thing right now is to is to reference something and say talk about it so i'll point at the conceptual notion of the form bar and say talk about it sure yeah yeah so i'm happy to talk about it. Yeah, the form bars are, I think, a really interesting part of Lozier's design, because they act kind of both as like a toolbar of, you know, like common functions that you'll want to reference frequently. And you can drag them out and, you know, keep them or add them wherever you want. So it can act as that, but it can also act

as something much more transient, like sort of a clipboard, you know, because at any time when you're writing a Flosure program, you can click on a circle and drag it into one of your form bars and it will stay there for as long as you want it to. And then you can, you know, drag it back out later to kind of copy that form you dragged into another place in your program, or you can throw it into the REPL or, you know, discard it from the form bar once you're done with it. So the form bars act as kind of a combination of like a normal toolbar that kind of like has a bunch of fixed things on it. But also a kind of very versatile clipboard, where if you have something that you you've written a little piece of code, and you say, Okay, well, I'll probably want to write stuff like this, you know, all over the place, then you can throw that into a form bar and, you know, use it whenever you like. So yeah, the form bars are one feature of Loader's design that I'm definitely happiest with. And as you said, they can appear on any side of the screen. And I've tried to make

it so that they're as flexible as possible in terms of kind of how the user interacts with them. You can make them bigger or smaller, move them around how the user interacts with them. You can make them bigger, smaller, move them around to different sides of the screen. You can have however many of them that you want. And a big reason for that kind of amount of customizability is that, like I said, I'm aiming for this to be useful on a bunch of different platforms. So, so far, I've mostly just been using this on my desktop, but I want it to be useful on tablets and phones as well. And there isn't like a, you know, a computer version of Lojure and a iPhone version of Lojure or a mobile version. It's all just the same code base that just kind of dynamically adjusts to whatever aspect ratio you happen to be using. And so, you know, in a

landscape aspect ratio, it might be more useful to have your form bars on the side. But then if you switch to landscape, like if you're on your phone, then of course, you'll probably want to have them on the top and the bottom of the screen. Because, you know, having them on the side will end up taking a lot of space in the UI. So yeah, that was kind of the idea behind the form bars, just having something that's very, very flexible and makes it easy to kind of copy and paste stuff to different areas of your program and to have something that can store kind of useful functions or whatever in a long-term way. And they're very similar in purpose to some things that we've seen in other tools that we've looked at on this show and that I'm sure you've seen in your own explorations of

the wonderful world of software, where normally this kind of a panel would be something that's stuck on one specific edge of the screen. So like there'd be a left sidebar or something like that where your symbol library goes and i'm thinking here of sort of some some of toby shockman's projects like recursive drawing for any of the listeners who are familiar with that where the left sidebar is where each new shape that you create gets a little slot on that left side and i'm curious ella if you consider doing an interface for vloser that was more rigidly defined, like on the left there is this, on the right there is that. Or if right from the start you were thinking about, no, I can't do anything where a specific side of the screen is reserved for a specific function because it's going to be cross-platform in the way that it is now.

The latter, yeah. I always kind of knew that I wanted this to be a cross-platform in the way that it is now? The latter, yeah. I always kind of knew that I wanted this to be a cross-platform project. And for that reason, I wanted the UI to be as customizable as possible. So that was always kind of the intention. And it's neat, the design that you've arrived at with the form bars, because they are sort of an interesting alternative to the sidebar panels. But of course, sidebar panels aren't flexible in the way that these form bars are. And they're also similar to floating windows like you'd get on a windowing desktop operating system

in that they don't carve out their own space from the main canvas. They kind of float on top of what's there. And you have that little bit of flexibility to put them on any edge or to have like four of them, one along each edge and like four, you know, nested structures of form bars. Cause it's not just like a single bar. You can have them kind of stacked. And is it like an arbitrary depth? Does it only go like one or two or three deep or is it just as many as you want? Oh yeah. it's arbitrary depth right now so you know if you stack too many of them then it'll start just kind of like overlapping

all the other ui elements uh so it's easy to kind of you know if you add five form bars to the to the left side of the screen then you're probably not going to really have any room for anything else um but there's no like hard limit on it right now. So, you know, add form bars at your own risk, I guess. Yeah. And I love that. Like it's this nice balance between the structure that you get out of a rigid UI, but the flexibility that you'd get out of floating panels in a way that sort of seems to take a little bit of the best of both. And so I think that these form bars are just like for such a simple conceptual ingredient,

there's a lot of interesting detail to the result that you've arrived at that I think is worth other folks perhaps listening to this. And hopefully the people out there in the world who are seeing Vloser picking up on that and thinking about that. Because there's some cool qualities to this UI that I think are a little bit distinct from what I'm used to seeing in other similar projects. Another aspect of the design that I wanted to ask about is the arrangement of top level forms is sort of a long single axis. So you can have a bunch of different top level forms that are each, you know, able to be evaluated separately and in your video by default you have this this sort of main axis that these forms are arranged along going from left to right and so each time you make a new top level form it kind of goes to the right of the previous one or

what have you and you have the option to rotate that main axis so you can have your top level forms going up and down the screen or at a kind of an arbitrary angle of your choosing and that is interesting to me because if i were building flozier i would have immediately gone nope it should be a free spatial 2d canvas and you can put your forms wherever you want them to go and just have free space and you know zoom in and out and move things around and organize them and what you have is something that is more constrained. It's more structured. And I'm wondering if that's something that you did deliberately

where you thought maybe about doing a free 2D canvas and decided not to, or if it was just something you did because that mirrors nicely the way that Clojure works as a sort of a linear language being text-based, or if it's just something where that's what you did for now and you might change that in the future as the environment grows and evolves. The idea behind that was mainly just to kind of keep coherence with the way that people normally write Clojure code. You know, in a Clojure file, the forms that you write, the top level forms aren't arbitrary. They have an order to them, I really want to maintain kind of compatibility with kind of the normal structure of code in Clojure. I

don't want to take this too much into like kind of being its own environment. I'd really love for this to just be an interface for the existing environment that is Clojure and ClojureScript. So I think having, you know, just kind of this open 2d canvases is like purely about aesthetics and and more of that impulse i alluded to earlier about wanting to burn everything down and start over again so totally yeah no i i totally see where you're coming from with that impulse i've had to fight that uh myself uh in in developing closure um i actually made a prototype uh not called closure i don't think i really had a name for it, but of this basic idea of, you know, like a visual Lisp with kind of a circular interface, nine months or a year ago.

And in that one, I wasn't using Clojure as kind of the underlying language, I was just sort of defining a new Lisp ad hoc. And so that was kind of the first, you know, direction that I tried to take this project. But then I realized, like, oh, it's really hard to, like, design a whole, you know, good programming language. So, you know, I kind of stopped working on that and kind of took a step back and thought, well, you know, Clojure already has so many wonderful tools. There's no point in reinventing the wheel here. I can just make this work as an interface for programming closure, rather than, you know, trying to build my whole own, you know, programming language in addition to trying to build my own, you know, visual interface. On the aspect of it being a sort of a mirror of existing text based closure, in that there's also a little button in the top right corner, you can click on, and it shows you the textual representation of the code that you're working on.

So you can flip back and forth between the sort of the graphical circles version and the text version with parentheses and indentation and such. Are there any features from a more conventional programming tool like a text editor or terminal or something like that that isn't present in closure that you're sort of or that you have thought about wanting to find a way to bring across into this more visual realm yeah absolutely um so i think that uh you know one one big thing in in lisps and uh or closure in lisps in general is structural, you know, you're, you're not really thinking about this program that you're writing as a text file, but instead it's this hierarchy, you know, defined by, or defined in the form of a text file. Um, but, but ultimately, you know, you're, it makes more sense to think of your

program as, uh, this kind of like tree-like structure rather than a long string of text. Um, and so structural editing, of course, you know, means kind of editing things from that perspective rather than from the perspective of text. And Vloser, in some ways, kind of lends itself very nicely to structural editing. You know, you're not dealing with text at all. You're just dragging around these different forms to various areas. But of course, there are many kind of elements from normal structural editing, like plugins and stuff that are missing in Vloser. Some, you know, obvious kind of structural editing tools

are there, you know, like dragging things around, but there are all kinds of interesting kind of structural editing operations that aren't super easy to kind of translate to the kind of drag and drop interface that Vloser has. And I'd very much like to kind of include more features to make the editing more powerful. And what I'm planning on doing right now

is I'm going to be adding certain... So the form bars right now just contain forms. But, you know, a toolbar or whatever normally doesn't just contain, you know, things that you can drag into your program, but it'll also contain like, you know, options and different commands that you can click on. And so I'm planning on adding kind of special forms that you can add to your form bars, where maybe one of them will be like, if you click on this, you know, little button in your form bar, and then you go and,

you know, click on some form, then it'll maybe kind of wrap that form in an enclosing circle or something, you know, or whatever kind of structural editing kind of operations you might find useful, could, I think, be built into just sort of a button that could live on one of your form bars, wherever you'd like it to live. And you can click on that, and then it'll pull up a little, you know, interface or whatever for doing kind of structural editing stuff. So that is something that I'm working on right now. And I think that'll be a big improvement to kind of the UI of Lojure. But for now, it doesn't have anything too special like that. I think that would actually be a really interesting way to add the idea of tools

like when i think about tools i mean that's such a you know a broad term um but tools in the sense of like uh in photoshop or something like that like the brush tool and the selection tool and the and the other brush tool and the other brush tool you know all the brushes yeah those tools in in a in a photoshop like program they're modal so you activate one of them and then clicks into the canvas invoke that tool and the tool does whatever it does to your pixels in the canvas and i think that it would be really interesting in a vloser-like environment to have like a tools palette like interface, but where the tools are, rather than being modal,

like you invoke them the same way you do the other things in Vloser, where you just grab one and drag it and drop it onto the thing you want it to have its effect on. Right, yeah. Especially if they're things that are sort of discrete in the kind of effect that they perform, rather than like a continuous

operation um something that is like invoking this tool does a thing it seems to me like there could be some very interesting ways to sort of logerify that way of working and so i i'm i'm personally excited to see how that goes as you as you explore that bringing editing tools into this because yeah that that does seem like that would be a nice way perhaps to do some more complicated edit operations that aren't perhaps as enjoyable to do just with like directly manipulating the the nesting of the circles right yeah because like one one you know relatively basic thing that's a bit of a hassle to do in flojure right now is you know know, if you have some form, and you want to wrap that form in, you know, another set of parentheses, or in the case of Clojure in a circle, the only way to really do

that right now in Clojure is to kind of drag an empty circle into, you know, wherever you want your thing to be, and then drag the old thing into that empty circle, and then go back and delete the original thing, which, you know, is like a three step process for something that should be one click, you know, so so I think I'm hoping that having this kind of like form bars that can contain these little commands will will really help things be more convenient. Another angle that I like to look at these tools from when thinking about how they could grow and evolve in the future is there's the angle we've been looking at so far, especially when it comes to ideas like here are things we can borrow from more conventional text-based or structural editing tools. And those things from borrowing from those tools are about editing the structure of the code the other sort of half of this is about bringing in some aspects from the execution side from the runtime side from the evaluation side and bringing those more into the environment and right now what

has is very similar to other conventional repls where you can evaluate a form and get the result back. But there's nothing in Vloser yet that is about visualizing the evaluation or adding more leverage over how the evaluation happens. There's nothing, as far as I'm aware, like a step debugger or anything like that. There's no way to kind of blur the lines between the structure of the code as written and then the actual operational semantics of the code as written and then the actual operational semantics of the code as it's executed. And I'm wondering if you've had any thoughts about that or if that's an area that you're interested in exploring as you continue to work on this project. Yeah, no, absolutely. I think stuff like that is definitely kind of on my radar and more

on a more basic level. Right now, Clojure doesn't really provide you with anything in terms of like errors. If you drag something into the eval circle, and you know that it encounters an error during execution, it just kind of gives you a big X in the eval zone to tell you like, hey, something went wrong, but it doesn't give you you know, the name of the error or whatever. So so you know, the first thing I need to do in terms of, you know, making debugging an easier process is just, you know, having kind of some way for the user to see, you know, what kind of errors occurred. And when it comes to things like kind of stepping through code or, yeah, more kind of like specialized debugging tools, I'd love to work on something like that eventually for Vloser, but that probably won't be something that I get to all that soon. I've got a bunch of other features that are kind of on my to-do list. But in the long run, I'd love to see something like

that. And one thing that I haven't talked about much, but that I think might kind of help understand the perspective that I'm coming from, is at some point, once Clojure is a little bit more mature, I'd love to try to make a version of it that works as like an in REPL plugin. So that, you know, it's not just something that you can use for ClojureScript in your browser, but it's also something that you can use for, you know, JVM Clojure, where, you know, it'll basically act like a plugin tool. So you'll just run some command on the command line and it'll start up a little web server that connects to a running Clojure process and it will let you kind of basically have a REPL

for JVM Clojure, but using the Clojure interface. And so I think when something like that is done, it'll be a lot easier to kind of integrate Vloser with existing kind of debugging tools. So you know, maybe you can, even if I don't do anything specifically for, you know, like a walkthrough or like a step through interface in Vloser, you know, you can be doing your stuff in Vloser. And then if you run into some error, and you want to go, you know, dig through the stack trace or whatever, then you can do that in some existing tool that's kind of more specially built for that kind of thing. That's nice. And that's, that's an approach that

I'm becoming more and more fond of is this idea of building these new programming environments as part of an ecosystem, like explicitly rather than as their own siloed thing allowing them to be opened up to interoperate with other tools so that it like relieves you of the burden of having to address every use case and every need and every special way that people like to work by just saying this tool you know has its strengths and the things that you would want to use it for but it's not going to do everything for everyone. And so just plug it into your existing workflow and use it for the things that it's really nice for, and then go and use your other tools for the things that they're better at. Which I think, and this might sound a little bit confrontational, but I think it's an interesting lens to look at things what would you say are the

strengths of lozier compared to something like a traditional text editing tool like if i'm already using some you know like um parent for something like that uh and i have some nice way of writing lisp code that i'm happy with why might i want to Vlosure as a programming interface to work with? What things would you say that it offers? I'd like to hear you sort of articulate, like, here are the strengths that Vlosure has that make it compelling to use compared to a conventional tool. Yeah, that's a great question. So I think it'd be, you know, way over ambitious for me to expect that, you know, any expect that any professional Clojure developers are going to abandon their current workflows and start using Clojure anytime soon.

But where I think that Clojure probably does have more potential is for people who don't yet know Clojure or maybe don't even know programming at all. The biggest benefit that I think that Clojure has um for for a lot of people who are kind of looking to to learn programming just the idea of like opening some text file and editing text and and you know running console commands um that stuff is all just really intimidating but just saying oh well here you just drag these circles around and you know uh you can do normal programming but through this uh kind of much more friendly looking interface i think that that's where revlosures biggest strength lies so i'm very interested in kind of making this an educational tool for people who are just learning closure script or perhaps

just learning programming for the first time i could see that being very appealing in that way because it's it's something that i think is a wonderful strength of visual programming in general is that it's, if nothing else, disarmingly different from what people might expect when they imagine what programming is like, which is kind of why I'm fond of that opening question. Because when you're confronted with one of these visual tools, it feels more like maybe the sort of thing that people would think programming was like if they were just looking at like you know hollywood where people are waving their arms around and making robots do all sorts of things yeah people might look at that if they are very self-serious and they're very professional and they might sort of dismissively say oh that's not programming because

it doesn't look like a you know a black and white text editor with a blinking cursor and and and they might be kind of stuffy about it but on the other hand i think there's a bunch of people for whom it's its visual character is just sort of if not charming then at least kind of uh it would dispel some of the fear or the intimidation that they might feel by being confronted with the colder forms of programming that are you know text in a buffer and a need to like, you know, explicitly run some terminal commands in order to compile your code and go through all those hoops. And so I think that that's a really wonderful thing that these kind of visual tools can offer. And I'm curious, I'll phrase this question in a slightly different way, because

I'm genuinely curious. I don't have any thoughts about this, any leading thoughts, I should say, which is do you perceive Lozier having like a potential future where it grows in a way where you would find yourself using it for your own work? Like, is there some part of it that really speaks to you

and the way that you like to interact with computation or interact with software, something about it that sort of is driving you towards creating it that you think you might want to use? Or is it something that you're creating for a different reason? Yeah, that's a great question. So I guess one strength of Lojure that I didn't mention in the last question is just that, you know, sometimes, you know, I'll be, you know, out and I'll have an idea for a little generative algorithm or something.

And I'll, you know, have some code that I want to write, but I'm nowhere near my computer. So, you know, I just have to try to remember it. But I always think it would be great if there were some convenient way to just pull out my phone and do a little bit of programming, right, and then save that code for later. And so that's kind of one use case where I can maybe see even kind of professional, you know, experienced developers using something like Vloser as kind of a more convenient way to kind of interact with Vloser on a touchscreen. This episode of the Future of Coding podcast is brought to you by Glide. Glide's mission is to create a billion software developers by 2030 by making software dramatically easier to build. We all marvel at how successful spreadsheets have been at letting non-programmers build complex software.

But spreadsheets are a terrible way to distribute software. They are an IDE and the software built in it rolled into one, and you can't separate the two. One way to think of Glide is as a spreadsheet-y programming model, but with a separable front-end and distribution mechanism. The way it works right now is that you pick a Google Sheet, and Glide builds a basic mobile app from the data in the spreadsheet. You can then go and reconfigure it in many different ways, including adding computations and building some pretty

complex interactions. Then you click a button and you get a link or a QR code to distribute the app. The data in the app and in the spreadsheet will automatically keep in sync. For the Glide team, that's just the beginning. Glide needs to become much more powerful. Its declarative computation system has to support many more use cases without becoming yet another formula language. Its imperative actions don't even have a concept of loops yet, or of transactions. Glide needs to integrate with tons of data sources and scale up to handle much more data. To do all that, Glide needs your help. If you're excited about making end-user software development a reality, go to glideapps.com and apply to join

the team. My thanks to Glide for helping bring us the future of coding. I would like to thank Replit for sponsoring the transcript of this podcast. Replit is an online repl that gives you a very immediately productive environment to get up and running with any number of different programming languages and frameworks and tools that you can use including git integration they have a multiplayer feature so that multiple people can hop into the same repl and work on the same project together it's all very batteries included, very easy to get started with and easy to scale up to much bigger projects. They've created this just fantastic sandbox for trying out new programming ideas, trying out new languages. They just have this constant stream of

new things that they're adding and new ideas and new tools. And they're just absolutely firing on all cylinders when it comes to making this environment robust and productive. Even if it's not the kind of environment that you find yourself working in frequently, it's the kind of thing that's useful to have in your back pocket

if you need to pop up something on the go and just test something out. Or if you want to, as a weekend project, stretch your legs on a new language, but not spend half the weekend installing the compiler and all the dependencies and getting your environment set up and going through all that work. So go to replit.com to check out their programming environment and all of the tools that come with it. My thanks to Replit for sponsoring the transcript and helping bring us the future of coding.

So I have one more kind of light and fluffy question about Vloser that I'm curious about. So it's a visual environment, and there are these circles that are nested within other circles, and it's like the presentation of the code is two-dimensional in its nature. But because of the nesting circles, and because of the fact that it's a zoomable interface it made me feel like um as you are zooming in on the circles nested within circles that i'm actually going like into the screen and that there's some depth to it and i'm curious if you also think of it that way where you sort of imagine it as like going into the distance and sort of like into a cave as you are going into your code. Or do you see it maybe as like a tower coming out, though that kind of, I guess, defies the sort of depth perspective? Or do you visualize this as just like a flat thing where the things are smaller and smaller?

Yeah, I think I do mainly just visualize it as, you know, something flat where, you know, just to see more detail, you just have to zoom in. I don't quite think of it in like a three dimensional like, like depth way. But that's, that's a cool perspective. I like that. Yeah, yeah, I think I mainly I just think of it as like a way of representing a tree, you know, that's fundamentally what the core idea of closure is, it's not even particular, or it's not even like specific to, to lists, you know, like any, any time that you have a tree, you,

you could represent it with, with something like closure. And I think it's my, my feeling that every time I'm, I'm, I'm descending into nested forms that I'm going into a cave is probably aided by the fact that the user interface is sort of a dark theme, like a dark background with like slightly lighter shades for the

foreground elements and so it does have this kind of feel that i'm like i've got a you know a hard hat with a little uh headlamp on it and i'm going into the the coding cave as i'm exploring these deeply nested shapes and so that's uh there's more of a comment than a question i guess but uh it's something that i'm I'm curious if that was something that you also felt or not, but I guess not. And that brings me to the end of the questions about Vloser that I had written down that I've remembered and the ones that I came up with along the way. And I'm curious if you had any other aspects of the project that you wanted to talk about before we move on to the last couple questions that I've got for this interview.

Sure. Yeah, actually, there is one thing I'd love to mention. As I mentioned before, I'm, you know, very much a generative artist. That's how I learned, you know, to program in the first place. And I still do that kind of stuff all the time. And as I said, I want Vloser, or I hope that Vloser can be used as kind of an educational tool. And I think that one thing that would really contribute to that is having a sort of built in connection between Vloser and some kind of like JavaScript graphics library, so that you can, you know, be doing kind of like generative art through Vloser,

you know, so maybe the left half of your screen is what Vloser currently looks like. And then the right half of your screen is like an HTML canvas that you can use, you know, p5.js or Pixie.js or whatever to kind of interact with, you know, on your phone or on your computer or anything. So I'm very excited about that. I'd love for this to be kind of just a little

kind of cross-platform generative art studio that you can use on any device. I think that'll be really exciting once I've got that working. That was actually something I had wondered about as well because it feels like a – and it's one of those things where when I look at the different tools that are made by the people who like doing generative art and they tend to make these unconventional interfaces in ways that play nicely with generative art one of the sort of decisions that they always end up having to go one way or the other on is does the generative art live in the same space as the visual code or is it in a separate space and so that was um something i was curious about

where if you were going to be using this for generative art would you want the generative art to be on its own separate like its own partition of the screen or something like that or have you thought about having the generative art exist in the same canvas as the as the code that you're doing because there's like i think so many different interesting design decisions that can come out of going either direction with that decision. And I don't think that it's one of those things where there's a, an inherently right answer. And so I'm always curious about seeing which way people go one way or the other with that. Yeah, that's a really interesting point. So far, I just kind of been assuming that it'll be something like, you know, one half of the screen

is Vloser and the other half is your canvas. But I suppose you could have something kind of different than that. And I'm just kind of thinking through this in real time. But the one thing that comes to mind is rather than having, you know, like a fixed background color, the kind of background that you're seeing could just be your, you know, HTML canvas. And the code, you know, kind of the structure of your code is just overlaid on top of that canvas. And maybe if you just, you know, sit there and don't touch anything, then it kind of fades out. So you can see, you know, what your piece looks like without a bunch of code overlaid on top of it. But yeah, I don't know, I'll have to think about that. Another thought that occurred to me, since we're doing the thinking through it in

real time thing, which I love to do on this podcast, is you could have it so that any the result view and if the um if the form is evaluating some kind of a renderable thing like a like a bitmap or something like that like it could just draw right within the the circle itself and so you could have um whether it's like this one circular construction that i've made that's running this algorithm is you know i could flip it to a different view and see it as the rendered result or whether it's like maybe there's like a a second top level form that's like this form is where my render target is and so i'm going to be in the one circle over here building up the algorithm to you know constantly recompute what the pixels are in that target and then i you know on the next circle over on my main axis is where the actual imagery is being displayed. Like, it feels like there's just a couple of different ways that you could sort of squeeze that idea of showing some of the result of the rendered image that you're creating kind of right in line with the code itself. Though that does like a little bit deviate from what's nice about having the eval

corner too yeah no no i haven't um really thought about sort of having like just sort of live evaluation you know like it whenever you change something it just kind of re-evaluates the entire form because i've very much been thinking of this as like a repl driven workflow you know like you write your code and then whenever you want you send it to the repl to to evaluate it um but but that i think i totally see where you're coming from. That could be a big benefit for something visual where just, you know, as soon as you change any bit of code, you can see, you know, what effect that has on your image. That would be pretty cool. It's one of the things that I harp on about a lot on this show is like,

show data in the code, show the data and the code in the same place so i'm sure i sound like a broken record to a lot of folks listening to this um uh sorry not sorry so the the one other thing i wanted to get into um now that we're into the back half of the episode and we can get into the kind of deeper more nebulous kind of stuff is your project broth oh right yeah um because it um it's i'm coming Oh, right. Yeah. blog post sort of talking about it um but i definitely didn't read the post closely enough to come to a good understanding of how it worked and what it is but what i found interesting about it was it looks to me like uh like and just a very high level summary it's it's a sort of a a virtual artificial life environment that you've created where you set up um and and correct me if i'm wrong but you set up this sort of graphical

environment where you have a number of sort of entities in this environment, and they all have their sort of rules guiding their behavior. And it's the system where you sort of let an evolutionary process play out. Is that kind of a fair, like very high level summary of it? Yeah, yeah, no, that's, that's absolutely right. Yeah. So broth is was a project in kind of artificial life. And broadly, the the idea behind artificial life, or at least kind of the digital part of it is to create simulations that kind of capture aspects of biological evolution. So you know, random mutation, followed by, you know, some kind of natural selection process. And the hope is that we can create simulations where things just as kind of rich as the, you know, biological organisms

that we see in real life can evolve within a kind of simulated world. And so that's what Broth was an attempt at doing. And I found that project and I'm, you know, similar projects, like I'm sure a very familiar, if not maybe directly related example that we could point to is like Conway's Game of Life. Yeah, absolutely. taking that and the idea of cellular automata and sort of spinning it into all these different permutations and these different things that have grown out of it. And even so far as I think Wolfram is doing some kind of sort of model of physics and trying to unify. Right. Yeah. Theory of everything. Using like something derived from cellular automata as like a foundational principle or something like that. There's some, some wild stuff going on there.

Yeah. Yeah. Stephen Wolfram is a, is a really interesting guy. I am kind of skeptical that anything is going to come with that theory, but it's definitely interesting to read about. Um, yeah, he's, he's, he's done really wonderful work on kind of the theory of, of cellular automata in general. And, um, and kind of the, the, the graph based, um, kind of model that he's using for physics um is is really interesting normally cellular automata run on kind of like a fixed um square grid or something or sometimes a hexagonal grid or a triangular grid or whatever but it's a very you know kind of fixed regular uh kind of background um but what uh wolfram is

working on right now um instead of having kind of a fixed background, you have this graph that kind of represents both space and the things within space. And so kind of the structure of space itself kind of evolves over time, according to the rules of the simulation. And so that I think is like a really interesting kind of elaboration on the idea of cellular automata. And I find that work really inspiring. There was a recent article that he put up about this work. It was interesting to me because it was presented as something that was talking about sort of fundamental aspects of physics and looking for maybe something even more foundational that he could use to express

what we now have of physics in terms of a more foundational thing. And what he arrived at was something that I couldn't make heads or tails of as a non-physicist, but it was very meaningful to me in terms of how it made me reflect on the behavior of programming interfaces and computation, especially the sensations of time within programming, which is a thing that I've been thinking a lot about lately. And so what I liked about looking at your project Broth, even though the aspects of it, which are about artificial life, went well over my head,

it looked to me like a very interesting environment for coming up with a programming model. Like it's got it's, it's, it's got that nice relation that, um, you know, like people have made programming models based on cellular automata. That's a, you know, a classic ESO Lang kind of project that folks do. And so I'm curious if in your work on the broth project, if you see anything in there that feels to you, like it's relevant to folks who are working on programming languages, and if there's any parts of that that you might point to as things that you

played with or things that you found where it's like, this was interesting for simulating artificial life and as a simulation of something, but it's also something that would be neat to work with as a way of expressing computation more generally? That's a really interesting question. I'm not sure. Unfortunately, I'm not sure if I have an interesting answer. But the idea behind Broth was it was more of a research project than anything into just kind of some kind of ideas about how to achieve some goals in artificial life. So I hadn't really thought about it in the context of what it might how to achieve some goals in artificial life so i hadn't really thought about it in the context of of what it might contribute to programming interfaces yeah so i'm not sure if

i um have much else to say there unfortunately that's totally fine and that's that's one of the things i do on this show is i occasionally will throw a kind of a um i won't say a curveball or i'll just throw a kind of a like a hey what if kind of question at folks won't say a curveball, or I'll just throw a kind of a, like a, hey, what if kind of question at folks and see if there's anything there. Sure, sure. Usually there's not. But that's, that's what we're here for. We're trying. We're trying. Yeah, absolutely. Are there other projects like this that you've created where you've sort of been playing with, like dynamic visualizations and interactivity and interfaces and that sort of thing that, uh, that might be of interest to the folks listening to

this show to go and check out? Uh, yeah, no, absolutely. I've, I've got a bunch of cool stuff that I haven't gotten around to kind of writing up or making videos about yet. And, um, I, I suppose one thing that, um, one technique that I've been thinking a lot about recently, which is useful in generative art, but also I think might potentially be applicable in like a wide variety of areas, is this concept of aesthetic selection. And aesthetic selection is a technique that's very much inspired by kind of evolutionary ideas. Whereas, you know, in biological evolution, you have basically the two steps are mutation and then natural selection. So mutation is just kind of blind variation of some like existing, you know, genome or whatever. And then after you've created a bunch of variations, you kind of narrow down this pool of variations by natural selection.

But of course, you know, there are, you don't have to just use natural selection, you can use any kind of kind of selection mechanism. And there are, you know, all kinds of useful techniques in evolutionary computation and machine learning that kind of apply different kinds of selection techniques to get sort of different evolutionary systems. And aesthetic selection is a technique where you have some system that is producing kind of variations. And then in addition to that,

you yourself kind of act as the selection mechanism. So a good example here is a project called Pick Breeder by Kenneth Stanley that anybody can Google and it's very fun to play around with. And the idea is that you start off with just kind of a random image generated by an algorithm.

And then the system creates a bunch of variations on this image for you. And you go through and select the ones that you like the best. And then you say, okay, we'll create some new variations of these. And this kind of iterative process where this algorithm is generating kind of variations for you, and you're picking out the ones that you like the best.

And so it's kind of this evolutionary process that involves both a computer and a user. And you kind of get to just use your aesthetic sense as kind of the selection mechanism in this evolutionary process. And I've been exploring kind of systems that use aesthetic selection a lot in my generative art recently. And I think it's a really fascinating technique because, you know, art in general can kind of be thought of as, you know, you're kind of conjecturing new ideas about what might make, you know, an interesting piece of art, you're kind of saying, well, maybe this would work, maybe that would work. And then you say, well, no, that probably wouldn't work, or that wouldn't work. And the way that I think of aesthetic selection is that it's essentially getting a computer to do the conjecture part for you, or at least help you in doing the conjecture part. And then your job is just limited to kind of

criticizing to saying, okay, well, I don't like that one. I do like this one. So, you know, give me more that look like that. And so, you know, the applications of this technique are most obvious in generative art, but I think that it might have applications much more broadly. Like I'd love to see, you know, systems where maybe, you know, a user can like customize their avatar on something by, you know, doing a sort of aesthetic selection where it takes their current avatar and kind of modifies it and does stuff like that. So that's a technique that I've been thinking a lot about recently

and I think has a ton of potential. It made me wonder kind of two things, and maybe you're familiar with some of the work in this area and could say whether these things exist or if they've been shown to not work very well the one thing i'm wondering is um if there's any interface that would allow you rather than just picking like the example that you prefer the most if there's some way to like sort of enrich the dialogue between the the computer side of the system and the human side of it to say like what it is that you like

or dislike about a particular thing that you're selecting yeah i'm not familiar with any systems uh that do something like that and and that that uh would require kind of a pretty big um kind of modification to the sort of normal way that aesthetic selection works um because aesthetic selection fundamentally is a very simple process. You know, it's just that you have this algorithm for kind of blindly generating variations, and then the user gets to pick among those variations. And so if you wanted some kind of system to sort of not just have it be blindly varying, but kind of trying to vary in the direction that a user has kind of specified, then that that would make things a lot more complicated. And one thing that I think is a really important insight

that Kenneth Stanley, who I mentioned before, has made about kind of evolutionary systems is that it's often counterproductive to kind of be too focused in advance on like what you want. When you're interacting with kind of an aesthetic selection system, if you have like a goal in mind, like I want to evolve in an image that looks like, you know, a cat or something, you're never going to find something like that, right? It's just it's just way too hard to kind of search through, you know, enough of the space to find something that fits any particular

preconceived goal that you have. And instead, aesthetic selection works best as this very exploratory process, where you don't have anything in particular in mind that you're looking for. You're just kind of seeing these different ideas coming at you and just looking at whatever piques your interest. You know, so it's not I have this particular goal and I can, you know, articulate the things that I like about, you know, these ones rather than these ones. Instead, it's just, oh, that seems interesting. Let me explore that space more. Or this seems interesting. Let's go explore that space more.

And is this technique, is this idea of aesthetic selection, is it something that is so far being applied only to images and sort of generating art or imagery? Or are there people applying this idea to other media as well? As far as I know, all of the stuff that I've seen has been kind of in the medium of visual art, either images or like 3D models, or, you know, short little animations, but it can absolutely be used in kind of a broader way. Something that I've been exploring recently has been kind of using it to generate sounds. So I've got this kind of little prototype of a sound design tool that

I'm working on, where it will, you know, you click a button, and it kind of plays you a random sound. And then, you know, you can play a bunch of these random sounds, and you pick the ones that you like best, and it'll generate variations on those. And you can end up finding some really interesting and unique sounds using a tool like this. And so I think, you know, anything that can be sort of generated by an algorithm can in principle be used with aesthetic selection. So historically, I think it's mostly been used with images and other visual stuff,

but I think it can be used for absolutely anything. It reminds me a little bit of this uh sound effect tool that was very popular in the indie game scene um i mean it's still popular but i think it had its heyday maybe 10 or 15 years ago called sfxer oh that's right i you know i i used that like 10 years ago and and you're you're just jogging my memory now and and that did have a way that you could um could kind of vary a sound randomly right yeah because it had it had like uh it's just a very very simple sort of uh original nintendo style synthesis tool where it had a you know like a noise oscillator and a and a sine wave and a triangle and it had you know a couple of parameters for each of them like amplitude and

frequency and some adsr kind of stuff and and so it had like you know 20 or 30 different sliders and you could hit a button to randomize all the sliders and you'd get just like all sorts of weird random noises or you could um pick one of the presets like you know this is a sound for a an explosion or a laser or a coin pickup or that kind of thing and then there was a button that was like, vary all of the sliders just a little bit. And so you could jam on that button over and over again and get like slight permutations. But it wasn't directed, it wasn't like, generate it in a in a particular like, oh, I like this variation better. So I can see this aesthetic selection being like a much nicer interface for using a tool like that. Yeah, no, absolutely. I'm glad that you brought

that up because I had forgotten about SFXR. And that's, that's a great example of kind of a simple little aesthetic selection tool. But yeah, like, I think the main thing, probably that that was limiting in SFXR was, as you said, it, you know, the way that it did the variation was that it had 20 sliders or something um and it just kind of like jiggled them around uh randomly for each one um and i think a much more powerful model of kind of evolutionary computation is um rather than having the the object that you're varying being like you know a vector of numbers that you jiggle around a little bit um you can actually evolve programs themselves and lisp is actually a great language for doing this

because you know a list program can be thought of as a tree and it's very easy to create variations on programs that are expressed as trees you can just kind of like delete a little part of the tree and fill it in with some random operations or you can kind of take two trees and kind of you know snap a branch off one of the tree and kind of shove it into the other tree. So what I'm most interested in with aesthetic selection is kind of evolving programs like this. And the programs can be used to generate images, to generate sounds, whatever you want. But the fact that you're kind of searching the space of all possible programs effectively means that you're kind of exploring the richest possible space that there is to explore. Whereas if you're just kind of like varying, you know, the positions of

20 different sliders or something, then you're pretty limited in the kinds of sounds that you're going to to produce. Now, you do say the richest possible space, but I think I, you know, given the choice between exploring programming and exploring like procedurally generated chocolates, I think I would much rather have aesthetic selection guiding me towards the ultimate tasty treat. But on a more serious note, the idea of using aesthetic selection with programs is very interesting, because it makes me wonder all sorts of things like what would you have to do to ensure that the programs were sound? Is it doing some kind of like, you know, doing a quick halting problem, like check on each of the variations that it's generating? Or is it using, you know, type systems or something like that to ensure that the programs that are being generated

are actually valid in some way? Like, what do you know about how to, how to make it so that these generated programs are actually going to actually produce output? Or is that just part of the selection criteria that you as the user of the tool are going to evaluate these programs on the basis of? Yeah, no, you raised a bunch of great points there. And if anybody is interested in this kind of idea of kind of evolving programs, I'd highly recommend John Coase's book, Genetic Programming, which is kind of most of what's inspired my kind of views on kind of evolutionary computation. And he goes over kind of different algorithms for evolving list programs by kind of, you know,

swapping different bits of the program trees. And so when you're looking to, well, so you mentioned first off of the issue of like types of, you know, how do you ensure that, you know, the right kind of data is being fed to each function or whatever. And one very easy way to do that is just to kind of have it so that your language that you're working in only has one type, right? So maybe all of your functions take in like a list of, or like a vector of numbers, or, you know, maybe multiple vectors of numbers and return a vector of numbers. And so if you're limiting yourself to kind of a situation like that, then you know that no matter how you connect these functions in a program tree, you're going to end up with a valid program. And you also brought up the problem of like halting, you know, how do you

know that the programs that you, you know, evolve are actually going to halt. And it depends on how you define your language. Because often in genetic programming, you're not technically exploring the space of all possible programs. Because of course, if you're exploring that space, then you're inevitably going to have some programs that don't halt. So instead, you know, rather than exploring, you know, the space of all programs, you can explore, you know, some smaller set, like the set of all primitive recursive functions or something, so that you know that everything is guaranteed to halt in a finite amount of time. Or an alternative approach would be to say, okay, we will include things like recursion and all this stuff

that leads to potentially non-halting programs, but we're only going to run it for a thousand steps worth of computation, and at that point we're just going to stop it wherever it is and take the current result, you know. So, you know, this is a very kind of nuanced topic, and there's a lot of different approaches to all the different problems that you mentioned. But thankfully, it's a well-researched area, and there's a lot of good literature on the subject. Are there any specific projects that you're working on or any ways that you're working to apply these ideas to your generative artwork or other work that you're doing that you feel like talking about at this point?

Yeah, absolutely. I'm hoping to make some videos about kind of the way that I'm using aesthetic selection soon. So hopefully, that'll be a good opportunity to kind of go into a lot more detail about this stuff. But one thing that I've been exploring recently is just using it to generate kind of little looping animations. And in particular, the way that this works is that you're evolving a program, right, which represents a function, and you generate an image from one of these functions by looking at each pixel on an image and kind of taking the x and Y coordinates of that pixel, feeding that into the function, and then the function returns a color, and you assign that color to that pixel. And then you go to the next pixel, you know,

and do this for all the pixels in an image. Depending on kind of the character of your function, you'll get, you know, vastly different results. Pixels that are nearby one another tend to have, have like similar outcomes from the program so there's still like visual coherence to the whole thing um and and well that that process uh technically would just produce a static image you know because that doesn't take time into account but if you want to create uh something like an animation rather than a static image

then you do something similar but rather than just taking in the x and y coordinates you also take in a time coordinate so that you're you're, you're producing a video rather than just a still image. And the process that I'm using there is very similar to plenty of artworks that have been done in the past. I think I mentioned pick breeder before by Kenneth Stanley, that it's essentially the same process there that I'm using in my work right now. But I'm just applying it to kind of animations rather than still images. I look forward to seeing a video on your channel about how to use aesthetic selection to generate a YouTube channel where we can have an infinite supply of these videos from

you about things like aesthetic selection. Yeah, I'll make a collaborative aesthetic selection program so that everybody can design their own generative art YouTube videos. I think that would be a beautiful thing to bring into the world. Well, Ella, thank you so much for coming on our podcast and doing this interview with me. Of course, thank you for having me. And that brings us to the end of the episode. Thanks again to Ella Heppner for coming on the show. Thanks to Replit and Glide for sponsoring. Thank you to you for listening. And if you would like to find links to any of the things that we talked about, you can find the show notes for this episode at futureofcoding.org slash episodes slash 54.

And there'll also be a link to that in the description for this episode in your podcast player. Now it's the end of the episode, so you'd better go fast if you want to find that link before it disappears from your feed forever and is deleted and vanishes and the bits go into space and get thrown into the sun and they dissolve and they're gone and you'll never hear them again. And this is the last time you will ever know about the show existing because it'll be wiped from your memory. It'll disappear like all things into the great unending sands of the cosmos.