Future of Coding - You Should Consider Some States Kevin Lynagh

Episode Date: June 12, 2018

Kevin Lynagh is a designer specializing in user interfaces for complex systems. He co-created Subform, a CAD-inspired UI design tool, with Ryan Lucas, which got a thousand backers on Kickstarter. He recently created Sketch.systems, an interactive playground for designing system behavior using Statecharts (hierarchical state machines). In this conversation, we discuss direct manipulation, Statecharts, challenges of layout engines, visual programming languages, the Clojure community, constraint systems, and the three different types of programmers. futureofcoding.org/episodes/25Support us on Patreon: https://www.patreon.com/futureofcodingSee omnystudio.com/listener for privacy information.

Hello, and welcome to the Future of Coding. This is Steve Krause. Today I have Kevin Lina on the podcast. Kevin is a designer specializing in user interfaces for complex systems. He co-created Subform, a CAD-inspired UI design tool with Ryan Lucas, which got a thousand backers on Kickstarter. He recently created Sketch Systems, which is an interactive playground for designing system behavior using state charts. And state charts are basically hierarchical finite state machines. In this conversation, we talk about direct manipulation, state charts, the challenge of layout engines, visual programming languages, the Clojure community, constraint systems, and the three different types of programmers. Of course, there are more than three, but we talk about the three different classes

of programmers that get into Hacker News arguments. And after our conversation, we continued chatting unrecorded, but then a few minutes in, I realized that we should be recording what we were talking about. So I hit the record button. So this conversation goes for like an hour, and then I'll frame what we were talking about, and then I'll play the epilogue, basically, afterwards. And without any further ado, I bring you Kevin Lina. Welcome, Kevin. Thanks for having me. Yeah, thanks for coming on. So you were just mentioning that you have emerged successful

from using some regex. So I'd be curious to hear about what you were doing that for. So this is actually part of Sketch Systems, which is the project that we'll be talking about later and one of the things that's really nice and i enjoy about releasing projects and putting them online is that people use them if you're lucky and they use them in ways that you didn't expect and one of the maybe the easiest ones or common ones on the web if you're doing things right is that people from other parts of the world will use it. And so someone had written in and they wanted to write in acrylic, the Russian alphabet Cyrillic, I guess. And they couldn't do that.

And I was really surprised. So I started looking into it because we have a whole grammar for how you define state machines. So I'm kind of digging into this grammar and I realized that we're using just a, you know, regex character classes. So like backslash W to match word characters. Yep. Yeah. And my kind of naive English speaker programmer assumption was that that meant all word characters and the nice people at Mozilla and Google and things like that were just going to make that work under the hood for whatever word characters mean outside of ASCII. It turns out that's not the case. It's just ASCII, uppercase, lowercase, and digits. And so I started looking into how do you match word characters, you know, outside of that.

And it turns out that it's a whole mess because of JavaScript's UTF-16, and you actually need to match the two parts of the Unicode code points separately. And, you know, there are nicer character classes in Java and C Sharp and in JavaScript 2018, in ECMAScript 2018, but they're not here now. So you actually have to make, there's a nice tool to take those character classes that specify this and then sort of decompile this succinct form into an extended form that just matches code point literals and ranges. So you get a little regex and then it becomes 10,000 characters long because it's matching all these different human languages

in terms of the byte representation. So I was very happy that that didn't add five megabytes of source code to my project and it actually worked. So that's what I just came out of right before I called you. Wow. Yeah. Congratulations. So how did it not add so much? Like, how did it not add all the size to your project? Yeah, it seems like it would. Right. I mean, my guess is that, you know, in the byte representations of the Unicode code points,

and this is all sort of outside of my domain, right? This is, I'm doing web programming and occasionally have to jump into this stuff. But I guess those ranges are consecutive in many places. And so you can say, I want to match code points from, you know, 10,000 to 50,000. And maybe that covers a bunch of word like characters. I mean, the people who work on Unicode and stuff are pretty reasonable and intelligent. So I would imagine that they've set things up in a way like that to work out kind of nicely. Or just that GZip is very good.

So yeah. Cool. That makes sense. Part of why I want to ask you about Regex is whenever I use Regex, I use like a Regex playground library thing where you could type a bunch of examples. And then as you're typing a Regex expression, it like live matches them. Right. Is that sort of thing you use typing a regex expression that like live matches them right is that sort of thing you use to do regex i've used that in the past um and and this is actually

this is a great point right to i guess what i was saying earlier about the joy of shipping projects is you know it's sort of like doing property-based testing uh where the there's only so much that you can actually imagine and so so as an English speaker, it actually had never, I'd never imagined an example where I tested this assumption I had about backslash W matching characters outside of ASCII. And it was only when someone was using my project and they were trying to type in their own language that they were like, hey, I can't use my letters, you know, and I don't even have those letters on my keyboard, so I don't know how to type those letters.

So it's not something that I think would have come up in one of those sort of playground libraries, although those sorts of playground libraries are very helpful in terms of helping visualize what's going on in the regular expression automaton, things like that. Cool. So now let's transition to talking about what you're actually working on, this new Sketch Systems product.

I really like this tweet I saw from you where you said that you're following the playbook that you learned from the Clojure community, read papers from 30 years ago, and implement them. So first, I'd be curious to hear about, like I know Clojure is based on like immutability and stuff, but I'd be curious if you could like maybe kind of go into that more, like what papers, like what research or researchers with Clojure based on and then what research and then maybe talk more about state charts and sketch. Sure, sure.

So I came into the Clojure community pretty early in my career. I had been doing a lot of data visualization work on the web using JavaScript and libraries like D3, which had sort of just come out. And I got into ClojureScript, or into Clojure more generally, because they had released ClojureScript, which is a Clojure to JavaScript compiler. And I had sort of run into these limitations of my ability to write large programs in the semantics of JavaScript. And so I was very happy to kind of see this other language that had these semantics that I found more comprehensible. And so that's sort of what got me in that community. And I spoke at a couple of conferences and went to a couple of Clojure conferences. And what I liked about it was Rich Hickey, who's the author of Clojure.

He, in a lot of his talks, would reference papers and ideas from a long time ago. And I think he set the tone in the community in that way. And I think this might be true in a lot of sort of emerging languages and early language communities because you have very technical people, usually, who are the kind of people who usually start building new languages. But I like the tone he set where it wasn't like a popularity thing or like, this is what's kind of cool this week. But it was much more of a like, here are these ideas from this paper in the 70s. And, you know, we took some of these ideas and put them in a language. So, you know, I mean, Lisp is maybe a good example of that. It's like actually a very old language, one of the first languages. And, you know, there are kind of just a lot of things like that. And

so just seeing a lot of these talks where people talked about these older things just got me Googling for some of these papers and reading about these ideas. And it's interesting because there's some papers that are very, you know, they seem very old, like from the seventies, um, you know, or you get these PDFs that are scanned typewritten pages and they'll have ideas about programming or they talk about problems in programming that are still very relevant. And I'd read these, you know, very old looking papers and be like, Oh yeah, that's totally still a problem. Or like,

it would totally be nice if I had a language that helped me, you know, address some of these things. So that's one of the things I really enjoyed about working in the closure community and being friends with a lot of those folks. And so, yeah. How did you find the state chart papers that you're now like, you know, obsessed with? Sure. So I guess for your listeners who aren't familiar, state charts are a generalization of finite state machines. So those might be familiar for folks who've taken formal computer science

classes and then also working programmers who've used things, I mean, actually like regular expressions is a very good example where a lot of languages have regular expressions built in. And, you know, at the end of the day, regular expressions define the state machine that tries to match characters and kind of move through all these different states. So I found out about state charts because I was working on an embedded project. There's some separate stuff you can Google about to find this. But for a period, I was very interested in building my own cell phone. And as part of that project, I knew I needed to do some hardware work and basically have a microcontroller. So a very small computer on a chip kind of thing,

talking with like this cellular chip that would talk with the SIM card and communicate with the network and things like that. And all of that was in, it was an AVR microcontroller. So those are the little ones from the Arduino. So there, you have to write in C and you have, you know, like 100 kilobytes of memory and things like that. So you definitely can't use a dynamic language like Clojure, or Ruby or JavaScript or something like that, because there's not enough space. And you know, you just have to write all this stuff in C. And I was like, I'm not smart enough, right to juggle all this kind of stuff and write all this in C in my head, like, and so I started looking around for like, how do you organize programs and think about things in this lower level. So I found a book about using state charts

for embedded devices. And I read that book and it was very interesting. And that's sort of how I got into it. And state charts themselves were written, you know, and kind of developed in the late 70s and I think early 80s by, I think, a mathematician or applied mathematician of some sort in Israel. And he was actually doing consulting work with the like an aircraft manufacturer, I believe, in Israel. And they had just piles and piles of these written specifications. And so he spent, you know, all this time trying to figure out what, how the aircraft was supposed to behave. And then he'd be talking with engineers and pilots and all this kind of stuff. And because he, I think he had this mathematical background, he started sketching out these different diagrams and then eventually

wrote kind of formal semantics for them. And then they turned out to be very useful for, you know, not just implementing systems, but also like drawing these pictures to communicate and show people what's going on and like have this conversation about, you know, how things should behave, but doing it in a much more rigorous way or like self-consistent way than you can do when you're just like writing individual, you know, paragraphs in a notebook or something like that. Cool. And so, and I guess you found enough success with them that you decided that it would make sense to make a tool to help other people use these. Yeah. Yeah. So it was, it was, it took kind of a while. So I did the cell phone project in 2014, 2015.

And I sort of put it aside. So I'd used this embedded library that someone else had written for doing state charts on this microcontroller. But the idea was always in my head. And, you know, during, you know, other projects on the web or, you know, in Clojure or something like that in those sorts of environments, I occasionally would use state machines explicitly to think through stuff. And a lot of my work is in user interfaces. And so that's a place where it's actually, stuff gets really complicated really quickly in user interfaces, you know,

and especially on the web, right, where you essentially have this distributed system and you have these asynchronous events that come in, right? The server is going to talk to you at some unknown time or people are clicking around the screen and, you know, whenever like these things happen at any time. And as soon as you have any kind of complexity, it gets very difficult to define and manage that, you know, by just writing kind of straight line imperative code or like nesting callbacks and things like that. So I had some experiences where I was, you know, using state machines or state charts in that environment. But what really got me into it again was about a year ago, I mean, maybe six months ago, I was working on a graphic design tool with my business partner, who's a designer.

And we, so this tool is called Subform. And so it's kind of like Photoshop or Sketch, right, where you draw rectangles and you move them around and things like that. But our tool had a layout engine in there. So you can define these layout constraints and kind of put things in this hierarchy. So it wasn't just rectangles on the page, but you would actually nest things together, kind of like what you would in HTML or in a programming language, right?

And what we wanted to do was let people have this direct manipulation interaction with some of those things. So, you know, a lot of people who were using the tool wanted to like grab a rectangle and they wanted to, you know, with the mouse, like drag it over and move it somewhere else. And because we had these more complex semantics where things were nested in trees and we had, you know, these layout constraints where you could put things in grids or stacks and stuff like that. It actually wasn't always clear how to interpret people's movements, right? So if they click on a rectangle and they like move it,

you know, are they, if it's in a grid, are they trying to move it within the grid? Or are they trying to like break it out of the grid? You know, and things like that. And so we started thinking about like, how can we let people express all these different things, like if you're holding control to duplicate it or you want to hold shift so that you can move it into a different place but keep it in the grid and things like that. And as we were trying to talk through all this stuff, it got very complex really quickly, even just me and my partner trying to define the behavior between the two of us, much less implementing it.

And that's really what got me back into using state charts, because we can actually define all these different cases. And then the specific, you know, transitions that happen as you move between these states to say, like, okay, when the control key, you know, goes down, and you're holding something, and this should happen. And then, you know, because we represent all that in these explicit states, it also made it much easier to implement in terms of giving people feedback, like in the UI, right? We could say like, if someone's in this state,

we're going to draw highlights around this target area that they're hovering over and things like that. And so what tools did you use to do state chart work before sketch systems? I mean, so state charts are really, it's just a, it's a graphic notation first and foremost. What was that? Doing pencil and paper.

Yeah. Yeah, absolutely. I mean, the original, if you read the Harrell, David Harrell is the original creator and if you read his paper, he has a lot of examples of just these just these diagrams. So, you know, that's sort of the original thing, which, of course, you can't execute those, which is fine, right? Because in many cases, it's very valuable just to draw those to make sure that people are on the same page, right? Because it lets you define just the act of defining these different states, it sort of forces you to think about like, what are the different possibilities? And then, you know, you sort of get this exhaustive list. And that was very helpful for us when we were building out this direct manipulation in the graphic design tool, because we could say like, okay, well, we could like, we could make sure all the bases were covered in the UI. Like we could say, okay, in this case, we're going to, it's going to be highlighted red. In this case, it's going to be highlighted blue. And in this case, we're not going to have a highlight. We're going to do this other thing. And like, we could

just look at it and define all of it. So it was incredibly valuable in that sense, even before translating it to code. But in that particular project, we were able to implement it in code. And the way that I did that was with a graph database library in Clojure called Datascript. And essentially what it was, you build a graph, right? In the computer science sense of graph, you have these nodes and edges. I don't know if you've used graph databases before. I think Neo4j is one that's been around for quite a while. But you can also think of it, you know, in terms of creating an object graph in an object oriented language where you have mutable attributes.

So, you know, in JavaScript, you can make all these different objects and you can set properties so you can sort of point them at each other and then walk this graph, you know, walk between the objects and things like that. So that's how we did it in Enclosure and ClosureScript using this graph database to sort of say, here's the topologies, so here are all the nodes or the states in my state chart, and then here are the edges between them. And this graph database was very nice because you could sort of programmatically do queries or things like that and say like, okay, I'm at this state and I got this event, you know, do I have a transition that corresponds to this event or does my parent have a transition that corresponds to this event and so on and so forth. Very cool. Yeah. There's been an explosion of libraries, I think, that are specifically designed to handle state chart tech kind of transition-y things in the last, like, maybe a few months or something. I feel like you've had a hand in, you know, causing, like, causing this new interest in this idea. Have you seen these libraries I'm talking about?

Yeah. Um, yeah. X state is, uh, yeah, it seems that David, David K piano, I believe this is Twitter handle, although he has some, he has some other last name. Um, but he's been popularizing state charts. It seems like in the JavaScript community for the past year or two. Um, yeah, I had written about, and I gave a talk about state charts a couple of years ago. It's interesting, you know, like like what I was saying with the sort of the trick that I picked up in the Clojure community, like this think is still very popular in embedded systems world, because in some sense, they don't have the hardware capacity to just change the code and see if it works and like deploy hotfixes and production and stuff. I think they're a little more conservative and they have more limitations that they face up front. And so they'll design their system in

advance using state charts or state machines. And then also, you know, if you do it that way, you can, I think, make very efficient representations in embedded systems to actually compile the state chart down into, you know, sort of optimal, you know, just giant switch statements or something that map very well to certain hardware architectures. Yeah, so it's by no means a new idea but it does seem like it's getting more popular maybe i i don't know in some sense i'm actually not i don't really follow the javascript community

super closely um you know just because it's like it's always there because i work on the web but at the same time you know i i couldn't't set up the Babel or whatever for like the life of me. So I don't know what folks are doing in that space. But it's not surprising, if you're making complex UIs, or you're working in a distributed system, being able to think about the states your system might get in and what it should do in those different states in response to actions is an incredibly useful way of thinking about stuff. Yeah.

So I want to talk, I have a question about a design decision you made in Sketch Systems. So clearly, because you are working on this tool Subform, which is like a direct manipulation design tool that we'll talk about in a bit you're no stranger to direct manipulation yet with sketch systems you made the decision to have a like less direct manipulation like you're not directly manipulating the the uh state machines and s state machines like in a graph sort of way where you can drag and drop nodes and connections it's a language that you parse, like, you know, what you were talking about at the beginning of this conversation with Regex. Why did you choose a language over a direct manipulation graph tool?

Sure. So I guess just to give a little background for any of your listeners, the tool, it's called Sketch.Systems. And that's, apparently,.Systems is a top a top level domain so you can go there because we bought it and that's where it is and you can try it out and play around with it but when you go there what it looks like is you have um kind of a split pane so one of our inspirations was tools like uh codepen or js fiddle where you know you just get in there and you can kind of sketch something out very quickly,

and then you get a URL to share it with people. And what Sketch Systems does is, unlike CodePen or JSFiddle, where you're making just like some JavaScript or HTML or CSS, and you kind of have a blank canvas, ours is very focused in that what you're making is a state machine or a state chart. And what it looks like is in this text area, right, you type in this little textual specification of your state chart, and then you get a little abstract diagram that says this is what your state chart kind of looks like. And you can click on the different events in the states of your diagram to sort of progress the system through.

So that's what the tool does. And to your question, like, why isn't it a direct manipulation thing? You know, it's one of the things that we wanted to do with this tool is make state machines and state charts more accessible to people who don't have a strong background in thinking about abstract systems. So we wanted this idea of sort of precisely defining behavior in this representation. We wanted that idea to be accessible to more than just like programmers or mathematicians or CS folks. So, you know, on one hand, like you could argue like the tool doesn't really actually do that much, right? Because you can always sit down with a piece of paper and you can draw like circles and you can draw lines between them. Yeah. And what's interesting is like when you draw something like that, and it's very easy to forget this i think as as a

programmer you know it's easy to forget what you've learned and internalized um when a programmer sees something like that or they see some code on paper i think what we do is we start executing it in our head because we sort of have the semantics already loaded in our brain to say like oh like i'm you know this i'm going to start in this state and i can follow this arrow if this event happens and i'll be in this state and then i'll follow this arrow you know and then i'll be in this other state and uh it was actually i had a lot of these experiences where i was talking with designers and friends of mine who weren't programmers and like a friend of mine's a ux designer and i was talking with her about this about state charts and stuff and she read her l's paper and she was

like yeah i get it this is cool and then whenever she'd actually try and make her own they wouldn't sort of be they wouldn't make semantic sense because although she thought she understood the formal semantics she actually didn't and it's very hard to you know when you're learning something to learn all the rules and be able to do it on your own, like without feedback. Right. It's like, if you, if, if this is why math class, for example, is very hard, um, I think for pretty much everyone, like you, it's very abstract and it's very hard to get feedback about like why things work the way they are and kind of play around with numbers. Um,

and so what we wanted to do is like, just draw this little diagram for you that you can kind of click through and it'll say like you're in this state and then if you click on stuff it'll like show you like oh now you're over in this state and that's something that any programmer can do in their heads but for folks who are learning we thought that'd be very important but to actually sort of circle back to your original question like why is it in this why do we have this textual specification language rather than a drag and drop kind of thing?

That's actually coming from the other direction, which is that I think a lot of ideas actually get obscured when you try and make them a little too point and click direct manipulation. And in particular,

state charts, they're an abstract idea, like the idea of, of a state, you know, or like a state space is a completely abstract idea. And we kind of thought like, if we're going to have people start drawing rectangles or lines, then they're going to be focused on this sort of graphical layout or graphical look of that. And like, they're sort of going to miss the forest for the trees, right? So that's one reason that we wanted to like under, you know, underplay that angle. It's sort of the same reason you might use a wireframing tool, you know, if you use like balsamic, or if you if you want to draw, you know, like design, like information architecture for an application,

you know, like a user, like how, how is the user going to interact with your website? Very early on, a lot of designers will use like a big Sharpie to do that, or like drawing stuff on the whiteboard because it helps sort of underscore, like, it doesn't really matter what the line width is and like what the colors are like the point is that there's a title bar at the top and like we're gonna have this button down here right um so that's sort of one one of the reasons and then you know the other is very pragmatic which is like it's very it's very hard to do direct manipulation right because you need to understand like what are the what are the things that people are going to manipulate how are they going to manipulate them um and especially if you're trying to maintain like a

sort of two-way mapping between multiple representations it becomes very difficult right this is the same problem with a lot of um you know like visual programming languages it it's like this is actually very it's very hard to you know take the idea of iteration and somehow make that visual and then then interpret like if someone's dragging you know an index outside of the loop like what does that mean like what should happen um so that's why we're like actually let's try and make it something that you know there's going to be this little abstraction hurdle because the idea is inherently abstract but let's try and make that hurdle sort of as small as possible um while at the same time sort of retaining the benefits of something like text right what's nice about text

is like you can check it into source control you can diff it you can copy paste it if you can like you know a lot of programmers and stuff have their own favorite editors and things like that so it's it's actually very fast. Um, I think for people to, to use it and interact with it in a way that maybe direct manipulation wouldn't be as fast. So, you know, you can just, you get this blank text area and the syntax is very simple, so you can just open it up and try listing out your States. Um, so, you know, the other inspiration that we had for it was, uh, like Markdown kind of same, same sort of idea where it's like Markdown is pretty straightforward syntax,

you know, at least at the high level and it's something you can just open up and you get the big ideas down very quickly about like, this is a headline and you know, this is, yeah. So, so that's,

we wanted it to feel like Markdown. So it would be approachable to people who have a technical background. Totally. As a, like, I guess where you're implicitly saying there's as compared with a rich text editor where you like highlight text and click the bold button in markdown you have to just remember the syntax of like surrounding a piece of text in asterix right right yeah yeah

that is a good point because i do love markdown and i never think like i wish this there was this was direct manipulation because i understand that the benefits I get are that it's just text and I can check it into source control and mark it up how I want with like a special markdown editor with whatever configuration I want. Okay. I kind of see, yeah,

there's a power in text that you give up when you do direct manipulation. Sure. Interesting. So there's one question I've had. I've watched a lot of videos on state charts over maybe a few months ago. There's this one question that I didn't see addressed anywhere. So maybe you could help me with it. It makes a lot of sense to me how state charts can help solve the problem of a lot of if statements and a lot of like nested

if statements. Like it definitely like nested if statements. Like it definitely, that's definitely hell. And this seems like the way to escape that, that particular hell. But I was wondering how more complicated state data structures fit into the state charts world. Like what, usually when I look at state charts, they like kind of leave off complex, like the actual meat of the data. Like if you have a to-do list, like the states that the to-do list could be in, like are you looking at only completed to-dos? That's a state charty thing. But the actual to-dos themselves, that seems like it's kind of out of the purview of state charts.

Right. Is that how you see it too? Yeah. The way I like to think about it is that states are, they define the sort of qualitative aspects of your system. So the aspects that are sort of fundamentally different and you want to use variables and data in your implementation for the things that are sort of quantitative aspects and so for example on the sketch systems uh tutorial page one of the tutorials is making an egg timer so like you know like one of those mechanical egg timers where you turn it clockwise and it's got like sort of winds up and then it sort of ticks down and then it rings at the end so you know what are the states you know how would you model that using something like state charts

and you know you can sort of in some sense you can capture all the behavior in a very little technical way um by making a state for like every possible second on the timer. So you can have a state for there's 30 seconds left and a state for 29 seconds left and so on and so forth. It's like ticking, you know, or things like that. And like, if you wanted to get more detailed, you could, you know, some of those timers, it's like you have to wind them past a certain point to build up enough energy. So you can make that, you know, those states to be like it, you would need to have an integer somewhere that would keep track of like how many seconds are left in the timer. Does that get it? Sort of the question you had? Yeah. So like, I guess another way to put the question is like a state charting library, like the one you used or X state isn't really like you also need something like Redux or some sort of extra state management. You can't do it all in a state charting system. It just handles the qualitatively different states.

Right. So that's right. So if you're going to implement something in many systems, you're going to actually have to have, you know, functions and data that exist outside of the state chart. That's not always going to be true. And there's some like sort of functions that you can define purely in terms of state machines, right? I mean like a regular expression is a good example about whether it's going to

accept some string or not. You know, you can compile that down to a finite state machine. But on the other hand, you know, the states of that state machine aren't particularly interesting or useful for people to think about, which is why we don't really ever dig into them or name them. You know, and to be completely pedantic, right? Like your computer only has a finite number of states that it can be in, right? It's like some huge number of how many, you know,

depending on how much RAM you have and things like that. But in some sense, like you could make a single finite state machine that represents, you know, your entire computer's execution. But the states in that state machine would not be very interpretable to humans. And if you think about state charts as a way for, you know, formally defining behavior in such a way that you can talk with other people about it, you want the states to sort of mean something to people and what that, you know, what they actually mean, it depends on both like your application, but also like kind of the level of detail

that you, you want to get into or specify so you know going back to the egg timer you might that might not really matter that you know you want to model whether the internal spring has enough energy for the mechanical timer to start ticking or not like that might be a detail that you don't really want to throw put in your state chart because like it's not going to change how you explain to people how it works or something like that um but that that's really a question that you need to think about you know depending on what your goals are yeah i that makes sense to me i think there's a part of me that just kind of wishes it could we could extend the state charts formalism to like also handle data so you could say like when

the timer has n seconds left like somehow like it could include variables and data structures too i don't know why i guess my brain usually just wants to right well the solution yeah so i mean stepping back um you know you you can do that um let's see how to explain this. For example, in the library that I used on my embedded system, which is called QP, I think, QP Nano, can put, you know, you can add function pointers to the transitions and this framework will run those function pointers when the transition occurs. And so you can, you know, modify data on the heap or whatever. And likewise, like if you're using a JavaScript library, you want to implement your own state chart library, like you can add your own, you know, sort of functions into there. Right. You can say like when you enter this state, you know, when you call this function, right? You can say like, when you enter this state,

you know, when you call this function and modify this data or things like that. So, you know, it's a way of structuring your program, like thinking about states. And so you can almost certainly add functions and things like that. And then lots of people have done that sort of stuff. I mean, state charts, you know, they're in UML

and you can do digging and find lots of libraries and different kinds of semantics for the details of stuff. I mean, state charts, they're in UML and you can do digging and find lots of libraries and different kinds of semantics for the details of it. But the very high level idea is that if you think about the states your system can be in, then it gives you this sort of design leverage that lets you communicate about it more effectively and then also build it more effectively you know how far that goes into the structure of the code like whether you um you know use an off-the-shelf state chart library to implement it or you just use the state charts as a design tool but then you implement it using you know whatever programming paradigm um you know kind of you can you can slice it a lot of different ways totally Totally. Totally. I think what was going on in my head that I didn't realize until I was listening to your answer is that,

um, I like had, I like held out hope when I heard about state charts that they could be the foundation for a visual programming language. And then you could like embed functions and data structures in the like visual specification of how states transition. Um, so,

so that, that's the part of why I want to combine them visually. I see. Yeah, it's tricky. I mean, I've been very interested in visual programming languages. And I think there are some applications where they work really well. But there are also a lot of great benefits to text and syntax that is very hard to make visual in a way where it's sort of more meaningful. I mean, a lot of visual programming languages end up looking like these spaghetti kind of nodes with inputs and outputs. But then the layout doesn't communicate anything about what the semantics are.

Like the nodes are kind of anywhere and you have these wires going anywhere. And so they don't really help. I don't find them helpful compared to working in a programming language, which might have more, more capabilities. And, you know, a lot of the, if you look into the UML state chart stuff, I mean, they have little notations for, you know, putting in certain conditionals and things like that but oftentimes you know you might just want to have a little snippet of code or you just want to sort of capture at a high level you know sort of the

function name so you might say like this transition you know you can you can have transition guards so like this transition can only occur when this event happens and you know this guard evaluates to true for example and that might be some some function that checks you know all these different variables you have been collating up right so you know you can you can do that but to try and have it all on one piece of paper in some sense i don't know you you might be throwing away some of the benefits because I think a lot of the value of state charts is that you get a high-level view of the semantics of the behavior that you want

without having to get muddled down by making it a complete specification down to the implementation, for example. Yep, yep. Cool. Well, I think now's a good time to transition to your other tool, Subform, that you mentioned briefly. It's a design tool, direct manipulation design tool.

It sounds to me like it's kind of like halfway between Photoshop and Sketch or Figma and on the other side, Webflow. Is that a good way to put in a box or maybe you could help yeah it's it's definitely in that space so we so my partner ryan lucas is um his background is in industrial design but both of us have worked together for a long time uh on on the web you know doing digital design. And we did a lot of consulting together for a couple of years. And we started working on subform because Ryan was very frustrated with the

tools that he had available to him. So he, you know, he knows HTML and CSS and a little bit of JavaScript, right. And he would feel this frustration. And I think a lot of designers have this frustration, especially the ones who kind of know both. They know enough about the implementation that they can sort of see the benefits of programming. Where we would be in these situations where, you know,

he wants to sketch out some ideas. So he used Photoshop or paper or Sketch or whatever. And, you know, those tools are nice because you can just throw down ideas very quickly you can just start drawing rectangles and colors and type and you can play around with all these things much much faster than you can um if you just went right into html especially in terms of exploring graphic design possibilities right like you can html is nice enough. You can do sort of high level information architecture and layout, you know,

pretty well. And that's usually what I do is I just get straight into code, but my background's more in programming and I don't care as much about the visual design. But if you care about the visual design, you know, you want to just be able to have like a type dropdown, right? You don't want to have to go look up web fonts every time you want to change the type. But he would feel this frustration because, you know, Photoshop or whatever your favorite tool is, is great early on when you're just getting some stuff on paper, so to speak. But then as you want to start refining your graphic design and you want to say like,

okay, here's my screen, but I want to like put real data in it, for example, or I want to see how this looks on a bigger screen or a smaller screen. Those tools don't really help you. Right, because they're all sort of pixel manipulation or like vector manipulation, but they don't have a lot of capabilities in terms of rendering out data or letting you do abstraction. You know, even the level of being like, I want to make this the primary color and I want to reuse it in all these places. It's pretty hard or impossible to do in these sort of graphic design programs. And so those things are all very easy to do, right? If you're writing CSS, because then you just change your one line of CSS and then your

colors update everywhere. And so he wanted something that was more in the middle that would let him sort of bridge this gap in his workflow more effectively. So to sketch out ideas quickly in a direct manipulation way, but then also once he's happy with the direction of the design to be able to start honing it in and like putting in more and more constraints or like abstractions to like make things consistent. Yeah, so that was sort of the problem we were trying to solve.

And so when we started working on subform, you know, the biggest sort of thing in it that made it different compared to everything else that was on the market, especially at that time, you know, this was in 2016 we wanted to give people, basically put a layout engine in there so that you can, you know, make your buttons size to fit your text and stuff like that. Really, you know, kind of straightforward stuff. So that's what we worked on. We ended up doing a Kickstarter, and we got about 1,000 designers

who backed the Kickstarter and then started using it. And that was a really wonderful and interesting process because, you know, their background, like the backgrounds of people were very different, you know, like a month between each other. And then from our backgrounds, you know, there are folks who were very familiar with the web and programming and they wanted certain things. And then there were other people who had only used Photoshop and were very upset when they

couldn't you know directly manipulate things or you know like for example in early versions like you had to pick the parent element and you would you would pick the parent element and you'd press enter to make a new element like that's how you made new elements and there are a lot of people who are like really upset by that because they were like, where's the like, where's the pen? Like, why can't I use the pen to like make a draw rectangle? And so that was like really interesting to kind of, you know, I understand why they wanted that. But then trying to explain, you know, and talking about vector graphics in general, a lot of people wanted the pen tool to do vector graphics. And then it would be like, well, you're making something that is going to work on the web and like web pages aren't really built with vector graphics. Like they're all built with divs, right? Like you don't go to a web page

and it's like all SVG, you know, control points and Bezier curves and things like that. So it was very interesting trying to sort of navigate people's expectations and needs and figure that out. And like one of the great things that came out of that process was when we started, the layout engine that we had was very similar to the CSS box model. So margins and padding and things like that. And a number of people were very confused by that which was interesting to us because you know we had spent so much time on the web we'd internalize margins and padding and absolute positioning and all this stuff but it was really great for me to see the perspectives of these other people who are trying very hard and this one person in particular i remember

you know he just wanted some white space, like between like a child and the parent. And he was like, why am I supposed to use margins? Am I supposed to use padding? Like, am I supposed to, you know, make it absolute position left? Like, why are there so many options to do like the same thing? And, you know, I, when he said that, I was like, oh, that's, that's actually a totally reasonable question. And I had forgotten, having worked on the web for so long, how strange it is why things are the way they are in the web. So based on a lot of that feedback, we ended up implementing our own layout semantics to try and simplify things in one sense and then make other things more possible or easier to do than they are to do in CSS. So I actually, I gave a talk about that at a conference called Deconstruct in 2017. So anyone who's interested can dig that up. But that was one of the, you know,

my favorite parts of working on this project was like thinking through the problem of layout. And I also have, you know, a lot of people like to hate on CSS, but like, I have the utmost respect for the people who worked on CSS because a lot of these problems are very difficult. And I totally understand, you know, why they made the decisions that they made because it is a very, there are a lot of very hard problems in there. So, you know, I'm, it's like, I guess,

trying to make anything will make you appreciate and respect how difficult it is to make anything and how complex things really are when you dive into it. Yeah, of course. Yeah, I don't hear people often giving respect

to the creators of CSS. So it makes sense. You're someone who's worked on a layout engine. So hearing that you worked on a layout engine. So hearing that you worked on a layout engine, it makes me want to ask, well, I guess you already said that it was hard, but sometimes I fantasize about coming up with a better abstraction or set of abstractions for HTML and CSS or like a better layout abstraction. It sounds like you worked on that.

Yeah, I don't know. Do you have those sorts of thoughts? Like one day, will we come up with something that's better or like a better compile target? Oh, well, I mean, knowing programmers, I'm sure we'll come up with something new and add it on top of the 30 other layers. Whether it's better is an open question. But yeah, in terms of layout,

there's a lot of research in this space in a variety of domains. And it comes down to sort of like how do you define better and what you want to do? And there are lots of different applications. I mean, most of what UIs look like now, it's all rectangular. And I understand why that is. But most websites are rectangular, and that's how people like to think about things for the most part. But you know, there are other cases around layout where you want to do things where you

rectangles don't matter at all. I mean, for example, we looked into, you know, early on some constraint engines that are used for doing things like, you know, like placing labels on maps is a good example. Like that's a longstanding and very difficult layout problem because you kind of want the labels to be on top or near the things that you're labeling, but you don't want them to be on top of each other, but you don't want to be too far. Right.

And like how you balance all those constraints is really difficult. You know, we looked at there's, if you're interested in kind of layout stuff, right and like how you balance all those constraints is really difficult um you know we looked at there's as if you're interested in kind of layout stuff there's a tool called uh cassowary which is out of the university of washington it's been around for quite a while it's actually the basis of um i think apple adopted it as part of their constraint system in ios i think it's called auto layout and we looked at potentially using that because it seems really easy because what you do essentially is you set up all these linear inequalities between all your things. So you say, okay, this, the width of this rectangle should be

half the width of this other thing. And like the space between this and this should be, you know, twice of this other thing. And so it's kind of nice early on when you have a small thing and we thought about maybe making a tool around that um but and like like many ideas you don't find out almost until it's too late um like how they scale or don't scale and so one of the things that's kind of hard with constraint layouts like that is like everything is very understandable and neat in your small examples because you can like center your box really easily but then if you have a hundred different constraints at some point the computer is just going to say like hey you know this matrix is not invertible like here you know your linear constraints are not consistent um but then you

don't you're kind of stuck right you're like well i don't know what to do here now um the computer might not be able to help you right it just you just gave it all these things to satisfy and it's just like i can't satisfy this and that's that's that so we didn't want to do do that um and what we ended up doing um there's there's some articles about this that we can put in the show notes but what we ended up doing is sort of a simplification of a generalization of css flex box and grid um so unifying all of that into one sort of system like absolute positioning and like a flex box and grids and and letting people handle all that with the same set of units and making it consistent so that's that's what we came up with. Um, and I'm, I'm pretty happy with it. Um, but I'm also very cognizant of things that you can't do very well

in it. Um, you know, like layout again is a very, it's very tricky problem, what people want. Um, and that was another kind of experience. I was a little surprised in some sense, working with designers, um, you know, cause they'd ask for certain things but then you know you try and dig deeper and a lot of people like that people say they want text to wrap for example um but then if you actually get into the mechanics of like how text wrapping works like where do you want to break the lines and like how much space should be between the lines and like is this a whole there's a whole separate like sub layout system in there um it's like it all gets very complicated very quickly so you know i think anyone who digs into this stuff it's again like i said it's easy to hate on css but if you dig into the problem you're like oh yeah this is actually really hard and it's not obvious what

the right answer is and i think most problems are like that and besides this cassowary from u of washington are there other papers or like approaches to this that you find particularly elegant that like might have the answer that you'd point us to? I mean, I'm very happy with the system that I came up with. I mean, there's but again, it it depends what you're trying to do. So, you know, in some sense, if you're making stuff on the web, your best bet is to, you know, make sure that the people who are using your website or your web app, you know, figure out what browsers are using, go to can I use.com. And then if enough of them support Flexbox or Grid or whatever is natively built into the browser, I'd recommend just using that, you know, there's, there's a technical problems of like, how do you define a nice layout engine, and you can write an academic paper about that. But then there's a whole separate set of sort of implementation questions about, you know, should I do this on my website? If it means we have to ship, you know, a half megabyte runtime, so that people can lay out stuff on a page, and it's going to destroy any iPhone that's older than three years, you know, or things like that. Yeah. I was talking more from the academic perspective of like writing a layout engine or writing

better stuff. Oh, okay. Yeah. I don't know. I mean, Casuar is definitely interesting paper. There's a whole separate, I mean, I think the field that you want to look for is, I think it's called constraint satisfaction problems. Most generally, like, and it depends if you're doing 2d layout there's a lot of stuff i'm sure there's

you could do stuff in 3d layout um you know i did some research into tools around um parametric design software so i don't know if you're familiar with tools like solidworks or inventor but those are kind of industrial design tools. And those are all built on top of constraints and sort of solid geometry kind of stuff. So it's sort of like layout in some sense, you're making solid geometry, but you're also trying to specify constraints between things. You know, what I really recommend is like, if you want to get into it, is like, try and think of some things to layout, and then try and express those layouts in a lot of different layout systems. And probably what you'll find is that there are, to do is to be able to have this tool that let

them play around with things so that they would see things at design time and realize that there are things they needed to work around rather than having to wait until production. You know, like one of the things that, you know, we didn't get end up implementing this, but one of the things I think is very useful is again, like how we started in this story with the regular expressions and the person who wanted to write in their own language. You know, I, I think a lot of people, I think it's very normal to draw from your own experience and not think about the experience

of others. And unfortunately, I think a lot of the stuff on the web and digital products are, you know, created by like young white dudes in San Francisco. And, you know, as soon as these things meet the real world, you have all these problems where it's like, oh, like, you know, a classic example is like, what happens if people don't have a first name and a last name? You know, it's a very Western kind of thing. And I still come across web forms where I can't do that. Or, you know, it wants me to put in a five digit American zip code, but like I'm not in America right now. So I don't I don't have five digits to give you. Like the best thing you can do as a designer is actually to kind of try and look around and go see how things are going to work and like look at UI and be like, is this going to work?

You know, if it's translated in German and all the words become like four times longer, um, you know, and sort of things like that. Last question about subform. I'd be curious to hear your experience. I got the sense that it was, you had a positive experience on Kickstarter. So I'd be curious to hear if you do it again, you'd recommend it. Any lessons learned? Yeah. So, you know, I'm very happy that we did the Kickstarter and, and what we did, and this is, I guess, a tip for anyone who's thinking about doing a Kickstarter. You know, well, first off, I guess if someone has a Kickstarter project and they want my advice on it, feel free to email me and we can put my contact info in the show notes. But one of the things that was really nice that we did was we actually had all of our backers, we had them sign up for a private forum. And that's actually how they got the software. So that kept everyone in this sort of private place where

people could ask questions and start to help each other out. And that was incredibly useful for us. Because instead of responding to these one on one emails or things like that, everyone was in this one place. And so they started helping each other out. But then it also helped us, as we were developing the product, have these conversations with our backers to try and say like, okay, this is what we're trying to do. Or like, can you please explain more about like what you want? You know, if you go on the subform forum, most of the, there are a lot of feature requests and, you know, oftentimes someone will sort of just do a drive by and be like, Hey, your color picker sucks. Like I want a better color picker. And then, you know oftentimes someone will sort of just do a drive-by and be like hey your color picker

sucks like i want a better color picker and then you know we have to um just go in and be like well like what do you actually want like what makes a better color picker like are you you know you want to have color fidelity because you're worried about like the rendering on stuff you know or you want to have palettes or you want to make sure that your designs work for people who have different kinds of color blindness like you want us to recommend certain palettes um for those kinds of situations like things like that and so it was nice having that forum to go back and forth and then have other people come in and talk about things they liked or didn't like and stuff like that so that's like a very practical tip i guess for someone who wants to do a Kickstarter.

You know, in terms of the downsides or what's tricky, you know, Kickstarter is very much a consumer facing thing. And even though, you know, we made a tool supposedly for people who are professionals, the market is very much a consumer market. You know, there are a lot of people like I think our Kickstarter, it was like $100 or $120 or something to get get subform and a lot of people were really upset because it's like a lot of money and it was just like hey bro like you know you're living in san francisco you're making like 80 grand a year drawing stuff in this tool all day long and yet you have this attitude that it's a lot of money you know it's it's very strange to me and like i think programmers are arguably even worse about this kind of stuff um and so even though i think we raised like 100 grand or something it's like that's not enough money to make a real software product right like you need to have professionals working on it for

like years to make make a thing um so i would i would be hesitant to encourage someone to you know make try and try and think they're going to fund their whole thing on Kickstarter. I feel like at Kickstarter now, the people who make the money, I think, are people who have funding from elsewhere. The Kickstarter is used more as a marketing platform to get people on board early for pre-sales and things like that. That's fine, but it's something to be aware of. Again, it's hard to make stuff it's very expensive uh it takes a lot of time so you know did just i think tempering people's expectations too was another interesting part of kickstarter a lot of people i think even though we we so we worked on subform with our own, like we had done consulting and we just saved up our own money and worked on it for a year. So when we did the Kickstarter, like we had a little video and all this kind of stuff and everything we showed in the video was like using the real tool that we had.

And everything we talked about was like stuff that was kind of already built. Like it was definitely half baked, but like, you know, like, like like the you had to press enter to make new boxes for example like there was a bunch of stuff like that and like the color picker was the system native color picker which is why some people didn't like it um but it was all there and it was interesting because a lot of people immediately downloaded the tool and then they'd be like there's no animation in here like you suck go away you know and it was like whoa we never talked about animation like what what's going on um so it was like, whoa, we never talked about animation. Like, what's going on? So it was interesting. And I think maybe part of that is from Kickstarter,

that it's a very aspirational thing. So I think a lot of people maybe saw it and they sort of imagined in this hazy, magical way what their dream tool would be. And they just sort of projected that onto our thing and then downloaded it and then were upset that it was not what they imagined um so that's you know that's something to be aware of i don't know if that's you know specific to kickstarter that's just going to happen uh in general with when you have have people looking into stuff but it's it's definitely a good experience to to interact with people and and and, and try and, you know, get them, get them on the straight and narrow and, and things like that. Um, you know,

it's, it's cool. Cool. Um, so we're about an hour. I, I wanted to give you a chance cause, uh, uh, related to find, I wanted to give you a chance to like, kind of give, like almost give you a chance to rant on like what you like and what you don't like about the desktop metaphor, uh, for operating systems. um yeah next time yeah i'd be happy to chat about um about finder real quick it has been an hour um so let's see what's the backstory on finder the find out all right oh go ahead well what i remember from Find Out is that, like, the thing that stuck with me is that we have all these apps on a Mac, and each app has a window.

Like, you can have a lot of windows, and on each window, you can have a number of tabs. And, like, isn't that insane? Yeah. Yeah, so that's something that's bugged me for a long time. So professionally, I use Emacs for most of my programming. And Emacs is very nice because the interface is very uniform. And there's a steep learning curve, but you know, you can, most of the operations will work at any point in the application.

And so once you get the hang of it, you can kind of navigate around very quickly and it's very, very fluid. And you know, with Finda, what I wanted was I wanted sort of a similar thing where I wanted to be able to move between things without having to do this mental context switch and think about like what application the thing is actually in. So for example, you know, like you said, switching between tabs in a web browser, or switching between like buffers and Emacs, or like opening a file on my computer, like, I know what it's called, I know what the title is. And I just want to like get it in front of me real quick. And I wanted to see if I could make an interface or make a tool that would let me do that without having to say like, okay, I need to like alt tab over to Chrome. And then there's like some other hotkey to like cycle through the tabs or like now I need to like get over, get to my mouse and like look at the top and start reading to

figure out like which of these tabs is the one I want to go to. I wanted to sort of avoid that and just say like, I know that this tab has the word mail in it. And I just want to like start typing mail and then press enter and like get to my mailbox so that was the the sort of product design um for Finda the backstory on it though is that you know I was working on subform and I was doing a lot of this very complicated direct manipulation stuff and working with a lot of these kickstarter backers who would want things to work exactly the way they worked in Illustrator or Sketch or whatever. And so on a purely personal level, I was like, I'm, you know, sometimes I'm just like, I'm tired of writing, you know, closure script and like dealing with

mouse, you know, mouse handlers and these complex state charts so that you can hold control and like duplicate your thing when you're dragging it somewhere else. Like I was super tired of all that. And I just wanted a break and I was like, can I just make a completely text-based interface that like has no mouse whatsoever and do all these other kinds of things. And finally it was also, it was my first project in Rust. So it really was like, I want a completely different change of pace so that on Saturdays I'll like

goof around in some other programming language that has, you know, no direct manipulation and it's purely text and all that kind of stuff. So that was sort of the personal backstory on where that tool came from. Fascinating. And what do you think of Rust? I'm super into Rust. You know, I deliberately picked it up because I wanted to learn a very different language. I'd been using Clojure and ClojureScript for the past 10 years or something, and I just wanted to try something that was very different. And I had done Ruby and I have to write JavaScript, and those are all kind of the same in the broader scheme of things. And so Rust was my first experience using a language that had a really kind of powerful

static compilation time type checking. And so that was kind of interesting for me, especially compared to Clojure. So in Clojure, all the data structures are immutable, and it's very idiomatic to share everything with everything all the time and in rust it's sort of the opposite extreme where you have to be very deliberate and explicit about like i'm you know i'm now transferring ownership of this object to you or you know i you know you have to start caring about like is this on the heap or on the stack and and you know what are the types of these things and so it was very different from Clojure,

which was very, at first it was very frustrating, but then it sort of became very refreshing as I learned how to do certain kinds of things. And then the community was also very excellent in Rust. I mean, it's very refreshing because it's clear they care about the user experience of programming. So I was a beginner for the user experience of programming. So, you know, I was a beginner for all these kinds of things. You know, I did not think about this. I didn't, you know, write a ton of C. I don't know a lot about the particular differences of the stack and the heap

and all this kind of stuff before that. And so the fact that they had this really well-written book and the tooling was very, it just worked out of the box and the compiler was very helpful with its error messages. Like it would just point me to be like, Hey, Kevin, like you're trying to do this thing and you can't do it because you know, this is why, um, like that, the whole experience of, of getting on board with the language was great. So they've done a great job. I've been really impressed with them. That's awesome.

Yeah. I've heard things about it, but I kind of disregarded it. I don't know. I'm just not interested in low-level languages. But I recently found this essay by the creator of Rust, Great, and Horror entitled What Next, where he talks about the remaining open questions in programming languages. And it was just so much brilliance in such a short essay. And he was pointing you to all the like front frontiers all the people who are tackling these questions in academia and industry i was just blown away so i now i'm now i'm kind of my curiosity's peaked and i want to see what this what this you know it's like kind of like you watch a rich hickey video and you're like like i want to play with this guy's language

now yeah yeah yeah no i definitely recommend checking it out and it's interesting because i don't consider it i mean i guess low level means different things to different people. And Rust is very interesting to me because there's, if you want to, you very much can. And this is an explicit design goal, I think, where you can understand what it's doing on, you know, maybe not the level of the hardware, but pretty low on the stack in terms of like, like I said, where your data is being allocated. So in that sense, it's low level, because you're caring about these implementation or these mechanical details that are completely obscure to you if you're writing in JavaScript or Clojure. But on the other hand, you know, the Rust language itself and the type system, it's very powerful and it lets you express very high level ideas. So like the trade system is somewhat similar to, I guess, Enclosure has protocols and I guess it's, I shouldn't even speculate about languages I don't

use, but it has these sort of higher level things where you can have these express ideas that are very hard maybe to express and see or you'd have to do it um you'd have to build your whole virtual dispatch dispatch table yourself and all this kind of stuff and and in rust you can express that very succinctly and the compiler sort of does the right thing um so it's feels very dynamic in some sense almost like kind of like like like Ruby or Python or closure, where you can do some of that stuff. And then, but then at the same time, it's going to compile it and make it perform efficiently for you. So it's been very interesting. It's not low level in the sense that like, all you've got to get are like some registers, and everything else is up to you. So this is where our conversation ended at more or less the hour mark.

But then after we stopped recording, Kevin and I continued talking. In particular, we were talking about something that came up in our conversation where programmers complained to Kevin that Sketch Systems doesn't scale to working in production. Put another way, when you formally specify your states in Sketch systems, you then have to transfer those states into code manually. There's no automatic translation layer. And he was saying it's annoying that programmers kind of want that layer to happen automatically.

And so we kind of debate the different perspectives of having like a single source of truth versus having multiple different representations that humans manually convert between. So that's more or less the frame of this conversation. And then I'll let Kevin take it from there. This question of like, what's the use of things that aren't going to be put into production

that programmers bring up often is something that I've, I've thought much more about as I've been working on subform because on one hand, the thesis of subform is that, you know, in a nutshell, a lot of people who are designed doing graphic design for digital tools are using tools that were designed for print. So Photoshop, illustrator, any, you know, sketch, like vector graphics, all this stuff,

that's all print tooling for the most part. And they've sort of been slotted into, or people are drawing pictures of screens in there, which is fine. That's a graphic design thing. But those tools don't let you effectively capture some of the inherent qualities of digital products. In particular, that, you know, some of the inherent qualities of digital products, in particular that there are these notions of composition

and reuse and dynamic layout that digital products kind of have to have now, especially on the web where you don't know how big the artboard needs to be, right? Like until you're shipping the code. And at runtime, it finds out that it's on some phone that's like whatever pixels wide. And so our thesis with Subform was that designers should have access to tools that let them think about some of those problems while they're doing their design work. So you can, you know,

resize the window and be like, does my graphic design still work? And, you know, what's tricky with that is, you know, resize the window and be like, does my graphic design still work? And, you know, what's tricky with that is, you know, how, like, where do you draw the line in the tool? Like, at what point do you no longer have a design tool, and you have like an IDE? Right? And it's not clear where to draw that line. It was interesting, because this is something that people brought up on the forum. Some of our Kickstarter backers were like some, you know, there was one person who essentially was like, I want every single CSS unit or like get the fuck out. Right. Like they wanted like REMs and viewport widths and like everything in the CSS spec. And we're like, well, you know, we're making a more general design tool because, you know, there's life beyond CSS,

right? Like people might want to use this tool to design native apps for iPhone and there's no VW unit or whatever. So some people wanted IDE stuff, but on the other hand, there were a lot of designers who, you know, coming from Photoshop or Sketch were very rightfully like, I don't want to think about this stuff. I just want to like try and draw my thing because that's what I'm doing right now. And it made me appreciate much more. There's this, you know, there are multiple steps in this process. Like when you're designing an artifact where there's this early step of divergent design where you want to just explore different ideas at a very rough or high level. And that's the sort of design I think that happens when you're at a coffee shop with your friend or whatever, and you're drawing on paper, or you just open up Sketch and you're like, you know, or Photoshop or whatever. And you just say

like, hey, you know, what if we look like roughly like this? And that part of design is like, you're just getting ideas out and you're trying very different sorts of things. And to support divergent design, you want things to just be fluid and easy. And that's why most of the time when I do that, it's I'm either in like a blank text file, or I'm drawing on paper, because you don't want lots of constraints, and you don't want syntax errors, and you don't want any of that stuff. And that sort of, you know, Sketch systems is designed to support that kind of thing to say, like, let's just try and list out our states. And like, let's just kind of click through them and see if this makes sense at a very high level. On the other hand, you know, as you as you start refining those ideas, you get into this stage of convergent design where you're like, OK, well, we have this concept.

This is cool. We drew some sketches sketches we have some concept art whatever but like now we actually need to implement this and like that's the part of the process where you say like okay well what are our constraints like we need this to work on google chrome and firefox and you know we have 30 of people are in china and so it needs to be translated into mandarin or whatever um and that's where you start getting into all these details where you're like, okay, well, like, how is this thing represented in the code? And like, is this the same title bar widget, you know, across these different screens, or is this the same React component or whatever? And so there's a whole separate set of questions

and concerns in that part of the design phase. And that's where you want maybe more of an IDE or you want a source control system and you actually want all this complexity because it's inherent in the problem. Like you have to deal with that complexity. And trying to figure out, like trying to bridge those two things, like those two stages is very difficult. And one of the challenges that we had with Subform

is that we actually realized there are a lot of people, a lot of designers who actually had no interest in the production or implementation side of it, right? Like they actually, in some sense, did not want to ever think about what it actually means to, you know, have a responsive layout, right? They're like, I'll draw a big one and I'll draw a small one. And like, I don't actually want to think about the exact place that breakpoints happen or, you know, whether, you know, how things move, because it's just too complex. Like, I just want to, you know, draw my shadows and like move the boxes around and the developers can like figure it out. So that was like sort of a sort of challenge for us or something I didn't anticipate, you know,

even though in hindsight, it's sort of obvious, like, these are different parts of the process in the same way that like, even before that process, you have your like, the CEO or your marketing vision guys who like don't even want to draw pictures of anything, right? They're like, they want to start from a press release, and then like throw that to their creative director to figure out. So these are all different stages in the design process. And I think one thing I learned in subform that I think a lot of that's going to at least influence my own work with tooling is approaching that question early on and like knowing where you stand on it. So with sketch systems, you know, that's, again, one of the reasons we're like, you just get this old markdown e-text thing, like it's not precious. We're not going to let you, you know,

twiddle everything to your heart's content. It's like, you get this markdown thing. Like if you want to do the work and you, if you know JavaScript, you can like type in some JavaScript in here, do whatever you want. Like we're not going to help you out too much because it's not really the point. Like we're not going to make a little app building framework for you in this tool. And that's again, the value of something like CodePen or JSFiddle.

You're not making a whole app in CodePen. The value of CodePen is you sketch out some idea, and you can just email it to your friend or your developer and be like, yo, this is what I'm thinking. What do you think? And you're facilitating this earlier part of the process, and it's okay that it doesn't sort of connect directly to production. Yeah. I see what you're getting at. And that's how things are.

Like you were saying, it's like how things are done at companies. You know, one group of people produces some assets and then sends them to another group of people who like reads the asset with their eyes and then translates them into like a whole different set of assets. And there's no connection. The connection between the two sets of assets are the translator in the middle who's doing the manual translation. So I definitely understand that it's how it's done and it's possible to do things that way.

I think a lot of times programmers want a single source source of truth representation that like you can have multiple views over or you, or like you, like a lot of different teams can all use, like refer to a single source of truth definition. And then if you want to change everyone's assets all at once, you just go to the source of truth and change things there. And then like the changes kind of propagate. Right.

Right. Yeah. But like you were saying, it's a weird siren song for programmers to want that you know and i've definitely found myself wanting that and then tried to build those kinds of things and i feel like it always ends up failing because you have you have you heard this term the wicked problems no so so wicked problem is one that you you can't even really articulate until you try and solve it and as you're solving it you sort of learn more about the problem uh sure and you know it's sort of related like the programmer uh was it you ain't gonna need it kind of thing it's sort of related where if you over abstract early on or if you're like oh i know like i'm gonna make i'm gonna

define you know in my platonic truth space and and I'm going to define some mapping that's going to map out to any possible representation from this platonic truth space. Like, you're going to have a bad time doing something like that, because, you know, it turns out it's really hard to do, or you want things to actually be different, you know, in these different representations um and you know i it's more and more and it's the other thing too to point out is that there are trade-offs to trying to capture all that stuff like one of the reasons i wanted to learn rust was because i wanted to work in a language where i could think about certain things sometimes um that i couldn't think about in Clojure. But I don't want to, like, I'm not trying to be like, oh, I need a third language that, you know, gives me the best of both Rust and Clojure, because I don't think that question is even coherent, right? Like, I can be in Rust when it's like, I want things to be, you know, formally verified as like,

consistent. And I want to think about the bit level representations, memory and all this stuff. And like when I'm using closure much more often, I'm like, I just want to like throw some ideas down. I want to sketch it out. And it's like, fine that I'm paying,

you know, whatever, some performance overhead for it to be interpreted or like, or it's not provably optimal. And I don't know what the types of any of my functions are, but it's like, I don't care because it's easy for me to sketch stuff out and like explore some ideas. And if, you know, it turns out that this function is too slow, like I can go and rewrite it in rust or assembly or

whatever I need to, but like, I'll do that later. Um, you know, and, and to force someone to do that upfront, you might give up the benefits of, you know, the flexibility of, of doing stuff in a dynamic language. Yeah, no, I definitely see definitely see that perspective um but i think then i'm just trying to think of like examples of where single representation sources of truth are like successful like one that came to mind is like why people like using react native or phone gap like technologies where if you write it in one technology it can like compile down to a lot of different platforms. That's like kind of a canonical example of, of why single represent or like,

I guess like, uh, yeah, the J JVM, like the Java virtual machine, I guess maybe that's not a great example. Yeah. I mean,

I mean, it's, it's tricky because those will work up to a point. And I mean, it depends like if, if your goals are within that point, then that's great. Right.

I mean, like closure is a great example for me because I write closure. And for mean, it depends. Like if your goals are within that point, then that's great, right? I mean, like Clojure is a great example for me, because I write Clojure. And for the most part, I don't have to care about whatever's going on in JavaScript, because the ClojureScript compiler is going to handle it for me. But then there are cases where like, there are things that I occasionally need to do in JavaScript that have no clear mapping in ClojureScript. And so it's sort of more work for me, because I have to go reach outScript. And so it's sort of more work for me because I have to go reach out and understand. I have to sort of break this abstraction to go and do interop or whatever.

And just like, you know, and this is true for any language because, you know, there sometimes are cases where you have to care about the machine. One piece of advice or perspective that I really like that i read on a blog somewhere is that there's actually three kinds of programmers programmers have three different schools of thought and a lot of times conflicts between programmers in the hacker news comments or whatever are because people belong to these different schools of thought but don't realize it and the three schools of thought are that you know one

one group of people just want to they just want to ship their thing right like they just want to they want to make a product they're like they care more about whatever they're building and they care about how it's built and they just want to get something out the door um the other group of people care about computers in like a mechanical sense. So they think it's very important to think about how CPUs work. They want you to write everything in C as if that's how hardware works, which it doesn't. But, you know, there's like the mechanical sympathy crowd where you're like, I need to know about cache lines. I need to think about how much time it takes to get data off the stack or off the heap and all this kind of stuff. And then there's a third crowd that think

of computers as this wonderful way to express these elegant mathematical ideas. And so you don't want to think about hardware and you don't want to think about performance or whatever, but like you want to make these amazing abstractions that are going to like handle all of your stuff. And these are the people who are always talking about category theory or whatever types. And it's interesting because those are all, you know, different perspectives. And once you realize that, it's helpful because then you can realize like, well, which one do I care about right now and why? And it can also explain if people are mad about stuff, right? Like, I mean, there are people who get mad about Clojure because they're like, this is, you know, unethical and unsafe because you don't have a type system to show you that your stuff is correct right and it's like well yeah but there's there's trade-offs to having type systems

too right and like sometimes it's actually valuable to move through you know impossible or illegal states as you're exploring ideas um in the same way that it's like sometimes you want to draw stuff without a ruler right it's like that's that's there there are times when that's fine and there's times when that's not a good idea. And I think it's helpful to think about all your tools like that to be like, well, what are the things that this tool makes easy and makes hard?

And are those aligned with like my own goals for what I'm trying to make? I think just to go back to our original spark to this conversation as like, why are people bugging you uh to like link their the state charts they create in sketch systems to uh to like their code it's just it's so close and i think it's the same reason that like state charts inspires me to like make a visual programming language it's like so close to a visual programming language and so close it's

like a formal specification it's so close to code that like it's like the programmer the mathematical programmer in me the third of the categories you listed like he's just like but why like why like you know why doesn't this just bridge itself automatically right right yeah i mean you you definitely can i know i'll probably add you know i'll probably add some export so you can take your your state chart and use it in whatever you know, get it back as JSON or something so you can play around with it that way. I bet there's like a very simple like X state integration that you could make if you want. Yeah, I've already written that on the train. So that'll come up at some point um but it's i guess i would push back though about it being close because you know it's interesting well on one hand you know we don't want these to be too precious

so like it's not going to be that hard if you you know like the value is that you thought about the states right so if you have to spend five minutes just typing it back into whatever your programming language is, I think that's not the worst thing in the world, right? I guess just to add, I think for me, a lot of the benefit of the sorts of things you talk about, like thinking like, is that the tool forces you to think about all the possible states. And so like you have like a certain level of assurity that like you didn't miss out, Right. Like there's a certain correctness, uh, to it. And so I think people like are nervous that if you, you, you, you, you're in this wonderful, correct world of state charts and then you move, like you use your brain to move to like code, like all of a sudden your confidence is a little bit shaken. Got it. So this is interesting. Have you worked with tools like TLA Plus or Alloy?

No. Okay. So this is really what you want to look into if that's the angle that you're interested in. Because like I said, I mean, our goal with state charts is to sort of make just the basic idea of like, hey, bro, you should think about some states. It's helpful. Like make that accessible to non-technical people. So, you know, designers and product managers and all that stuff. I think that might be, I think that might be the title of the podcast. Hey bro, you should think about some states. Sure. I mean, it's, it's a good, it's a good idea. Um, so TLI plus is a tool from, uh, Leslie Lamport, uh, that, so he's very famous

distributed systems researcher, computer scientist. And that takes this kind of stuff, you know, it's in the realm of formal verification. But if that's the angle you're interested in, you really want tools that will actually check invariance in your state space for you, which Sketch Systems does not do. And we do not plan on doing because there are very good tools for doing that. So TLA Plus very famously, you know, essentially what it does is it, you make not a state chart, but it will compile a specification into a state space and then exhaustively search it, I believe using like SAT solvers and things like that. And you can have certain invariants and it'll give you these traces

that say like, you know, you might say, hey, you shouldn't be able, like, you know, the bank account balance should never be negative, right? And it'll say like, okay, given these transitions and things, like here's a sequence of things that lets me make the bank account balance be negative because it's exhaustively searching the entire state space. Alloy is a similar kind of tool, but those are the ones where you'll actually, you can formally verify that your specification is correct. But interestingly, those things also do not generate code. So that's still sort of left as an exercise to the reader, but there's a lot of value in thinking through the state space. Because again, you know, and there, there,

there are languages and I haven't used these languages. There are languages, I believe, where, you know, you, you can do these formal verifications and it'll generate code and do all this stuff for you. Um, but my suspicion, I could be totally wrong. My suspicion is that it's going to feel like my math class felt as an undergrad, which is the first time I was exposed to, to actual mathematics with a capital M where like, I just wanted to do some arithmetic. Right. And we spent like three months proving that like zero is the additive identity. And like, it's important to do that, you know, if you're going to do actual mathematics, but like I ended up majoring in physics because I was like like i don't have time for this i'm i'm gonna just divide by zero and like

you know wing it and it's fine um and those are different attitudes and it depends again what your what your goals are um i i think um i'm totally with you i think there's a part of me that hopes that we can like bring a sort of formal like confidence that you that you have um in a math class to something like sketch system so it's like it feels like you're just like messing around but if if you do something that's impossible it's just like it's just like quietly is like you know by the way this is impossible for this reason you know like so like i don't know if it's possible but i i feel like i can like emotionally i want there to be this marriage of the two. Oh, yeah.

So you can go as fast as you want, but you have this brilliant SAT solver on the background making sure you're not doing impossible things. I mean, in some sense, that's the experience. Again, it's very similar, I think, to working with language like Rust or Haskell compared to JavaScript or Clojure. You have the compiler in the background checking certain invariants for you. And sometimes it's very helpful and sometimes it's very annoying. Yeah, well, I feel like there's something in the middle. There's something where it's as fluid and amazing as Clojure. And then in a way that it doesn't prevent you from like it doesn't slow you down

like you know you can go as fast as you wouldn't closure but somehow in an inobtrusive way it's like you know just by the way like this is why what you just wrote like doesn't quite make sense and here's how we suggest you fix it you know i'm i will i'm looking forward to uh to seeing that on a future episode and hearing about it. Okay. Yeah. Sounds good. Yeah.

I don't think I'm going to be the one to make that happen, but I want that too. Yeah. Because I think the three programmers you mentioned, the three different schools of thought, the third one is, is kind of where I'm at, but I think there's like, I don't know if this is a fourth school or this is more kind of like the third and the third and first schools combined. Like there's a Brett Victor direct manipulation, rapid feedback school. Um, I guess it's probably the first school and the third school combined, but, but anyways, I feel like I want the mathematical beautiful abstraction stuff combined with rapid feedback like closure

stuff so um i don't know what school that is yeah and i mean computing is a very young field so what by no means and then you know this is i'm not trying to discourage you i think that would be a very hard problem but i think if you dug into it and a wicked problem yeah i mean definitely a wicked problem but i bet if you dug into it, And a wicked problem. Yeah. I mean, definitely a wicked problem, but I bet if you dug into it, you know, you might be able to find, I mean, almost certainly we'll be able to find ways to improve the status quo. Right. And again, this is something that I think Rust has done very well where I think from a, my understanding is that from like an academic perspective,

Rust is not doing anything super crazy, but they've taken some ideas from the seventies and eighties and like, they put that stuff in an accessible format, right? Like you can just download this compiler and it gives you these nice error messages and it like exhaustively checks your, you know, your match branches and your enums and stuff like that. Like that's not groundbreaking from an academic or research standpoint, but it is groundbreaking from a practical standpoint in that that is

accessible to people who previously were not able to access that stuff. And that's, that's the kind of thing we're trying to do with sketch systems too, is just make these ideas more accessible because these old ideas are powerful, you know, and there's plenty of good ones, you know, even before you start trying to come up with completely novel things. Agreed, agreed. Cool, man. It was great talking. I probably need to wrap it up, but I will, I'll send you some links about TLA because I think that's something that you'll be very interested in based on what

you're saying about your desires for sketch systems. Cool. Yeah, sounds great. Thanks. Yeah. Thanks a lot. And yeah, thanks again for taking the time. All right. Go, man. See you later. you you you you you you you you you you you you you you you you you you you you you you you you you you