Future of Coding - Bringing Explicit Modeling To The Web: David K Piano

Episode Date: July 17, 2018

David K Piano is bringing explicit software modeling to the web with his xstate library. He gives talks around the world about statecharts, and is cooking up a new SaaS service that will help developers model and understand their application using statecharts. In this conversation, David and I discuss the benefits of declarative languages, such as CSS, the principle of least power,  musical notation, and Facebook Origami. futureofcoding.org/episodes/27Support us on Patreon: https://www.patreon.com/futureofcodingSee omnystudio.com/listener for privacy information.

Hello, and welcome to the Future of Coding. My name is Steve Krause. Today I have David K. Piano on the podcast. David is a pianist and a passionate developer of CSS and JavaScript. He currently works for Microsoft. In the past few years, he has given talks about state charts all around the world, and he has created the Xstart xstate javascript library he also co-runs the state charts community online if you caught my conversation with kevin lina a few weeks ago you will find this conversation a nice follow-up

david is a bit of a celebrity on the codPen community online for his CSS animations. He is one half of the Keyframers livestream duo, where they take animations from Dribbble and re-implement them in HTML, CSS, and JavaScript. And as we discussed in this podcast, he is working on a new SaaS startup to help developers understand how their applications work visually from a state charts perspective. It was a really fun conversation, and without any further ado, I bring you David K. Piano. So today we have David Korshid on the podcast, also known as at David K. Piano on Twitter. So welcome, David. Hey, thank you. Glad to be here.

Glad to have you on the show. So I wanted to start, as I often do with these episodes, with your background, how you originally got into programming, which I think I've heard elsewhere on the internet is a particularly interesting story. Yeah. Well, I've always been interested in programming, just ever since I was at the library when I was around 14, 15 years old, and I found this book on doing number theory with basic and so of course I was trying to actually do these exercises just because I was very into math so I've always sort of had a knack for programming but I actually went to college for music performance because I do piano hence

David K piano and there was this intro to music technology class. And in this class, our final project was to make a marketing page that essentially marketed ourselves. And we were told to use this program called iWeb, which is a drag and drop interface, I don't know if you remember, similar to Dreamweaver. And I did not want to do that, mostly because I didn't have a Mac and I didn't want to drive all the way to school so I figured hmm there must be a way that I could just do this myself and so I just studied up and looked up this HTML CSS JavaScript stuff and I went through all the basics and made my own website.

When I presented it to the class, my professor said, you need to be doing this as a career. And so I did a little research and I found like, oh, hey, it pays a little bit more than whatever my music degree would get me. So yeah, I got a sort of a part-time job at the College of Sciences at my university. And today I'm here. Awesome. And then, so I feel like that's, it's interesting to hear about how you got into programming. But then I feel like you're one of the, you know, I guess because I'm interviewing you on this podcast, you're one of the people who's interested in like pushing programming forward and making it better. So how'd you get into that kind of subfield of programming? Oh, just being lazy. Well, actually, music has a lot to do with it just because music forces you to think creatively.

That's what all music, performing arts, visual arts, any sort of art is about. And when you think creatively, you try to come up with different solutions to common problems. And I wanted to apply this to programming in a way. And I think that musicians tend more towards the declarative side of programming, because if you think about what music is, it's a very creative, free-flowing form of expression, usually done through, you know, sound, of course. But we represent this using music notation, which is very declarative and very exact. So we don't have someone saying, play this note. Okay, now

play this note. Now play that note. Now do this. Now do that. It's just right there. Like this is what the music is. Interpret it as you may. And so that music could be adapted between instruments or adapted to even more than one musician. And so it's a very declarative, flexible format. And I thought, we should be applying this to programming more and more. And so that's why I tended towards a more declarative approach towards programming. And I was doing a lot of research. And then I realized that thinking about programming this way and doing programming in a declarative way actually saves a lot of time and makes all of our applications easier to digest and understand and more maintainable too. Fascinating. Yeah.

To kind of push forward this music metaphor, if it's a metaphor, is I was thinking about how musical notation is very much not like put your finger here on the instrument, like take this breath, then breathe into the instrument. Instead, you have the notes like almost on a stream, which I'm sure we're going to talk about. And streams are like these very declarative visualization over time. So I don't know, something about that, like the difference between,, imagine how much worse musical notation would be if it was imperative. Yeah, exactly. I don't even know how you would represent that. It's just so hard to think about. uh, like confirm with you on, um, is that many of the people who I interview on this podcast got their start in this field of improving programming from people like Brett Victor or Alan Kay, those sorts of folks. Oh, of course. Yeah. Okay. So you're, I was going to, I was wondering if you are familiar with them or if they were part partial influences or not. Definitely Brett Victor. Um, and I'm sure he's influenced a great many people uh the the main

thing about brett victor is that he's moving towards a more tangible way of doing programming doing developments and representing ideas and concepts and that's why he has this uh i think it's called dynamic land project where programming is actually tangible. But in the past, he's had all of these met with resistance just because it takes developers more silly like coding seems like just typing stuff. And hopefully if you don't make a typo, everything works. And so that seems a little bit silly to me. Yeah. Um, I'm with you. Uh, so it seems like, uh, there's been a lot of activity, a lot of people recently, particularly in the JavaScript front end user interface community of people talking about this state charts thing. And so I'd be curious if you could walk me through kind of how, like originally where those ideas came from. And then I'd also be equally curious to hear about like how those ideas have kind of resurfaced today. Like who,

like who are the people who originally started talking about them and then how you found out about them and kind of what you've been doing with these ideas? So I've been, back when I was starting like my first, I guess you could call it a big boy job, developing, it was at this electronic prescription company and we had these complex workflows for these medications. And these workflows had a complex sequence of steps that could change depending on what you a declarative way. And through that, I started researching on finite state machines and seeing that these sequence of steps can be represented as finite state machines. And of course, that led to learning about state charts as well.

And so I've been researching them just quietly for a while and just continuously thinking about how we could actually use these in our applications. And it turns out that they're actually in use and they've been in use for decades in other industries, just not the web development or native mobile development industry for, I guess, whatever reason. And so I was researching these ideas more and more. And I submitted a talk to React Rally in August of 2016 or 2017, I believe. One of those days. Actually, I think it was 2017.

And surprisingly, they accepted it. I thought that, you know, no one is going to want to hear about this, you know, this weird computer science topic that you learned in school and, you know, immediately forgot because it didn't apply directly to the software that you were building. But there was a lot of traction over it just because a lot of the promises were there that it makes developing software not only easier, but more robust. And the idea of having things visually represented was just a huge selling points to a lot of developers. And so that also paired with Kevin Linna's Sketch Systems. I remember he actually sent, right before the conference, he sent an email from Subform, which Subform is a design tool similar to, sort of similar to Sketch. And in that email, he was actually talking about state charts, just, you know, planting that seed in, you know, people. And so I emailed him, did some back and forth and told him that I was going to give a presentation about this. And so I guess between the mix of those two things, state charts and state machines have risen in popularity in the front-end community.

Yeah, it's fascinating. Some other names to just throw out there. I guess David Harrell is the original guy. Yes. And then Ian Horrocks is the guy who wrote the book on applying state charts to user interfaces. And then there were two other names adam sola solov i think yes i'm solo he wrote an article that that was related to this i feel like he and kevin might be friends i i'm forgetting and then guillermo ranch also um is is responsible for getting these ideas out there um i i i just wanted to kind of

fill out the uh like the picture of how these ideas kind of got germinated and spread yeah so adam solo he wrote an article called pure ui control which is a is based on guillermo roch however you pronounce his last name his essay called pure ui and in this uh pure ui essay he's talking about just the this abstract idea of instead of imperatively saying your app does this and then it does that and then it could do this and do that instead breaking it out into states and so so Adam Solove took that article, and he wrote a follow up article in which he defines a very concrete concept to this, which is state charts. And yeah, so Ian Horrocks, that book, Constructing the User Interface with State Charts, I read that book, actually actually like probably twice now, like cover to

cover. And what I love about that book is that it describes even modern development so well, even though it was written back in, I believe, 1998. So really, really far back. Well, at least it seems far back. And of course, Harold was the inventor of state charts for the original paper in 1987. Cool. And then I think just to get fully up to speed, I think there's Eric Mogenson has created this world of state charts website of resources and like a community of people. So I just wanted to put a plug in for that too. Yeah, Eric Mogenson, he actually gave a presentation about StateCharts in user interfaces all the way back in 2014, which I found really interesting. And that's actually how me and him connected because he was one of my influences as well and sort of solidified the idea of

how useful state charts could be in user interfaces. And so he and I were, you know, we run a great community over at spectrum.chat slash state charts, where there's actually a few really passionate people who we discuss the ideas of state charts and using them in all sorts of various applications on there. Cool. So I guess to get a little bit more concrete, you've taken these ideas about state charts and you've created this library, XState. So I'd be curious to hear what that's about. So XState is a library that basically I wanted a JavaScript library that encompassed all of these ideas of state charts. But at the same time, I wanted a library that embraced the ideas of functional and reactive programming at the same time. So the various other libraries that existed, I thought that the API was either too

separated from the original ideas or too limited in that they didn't encompass all of the original ideas of state charts or that they might have, such as like Scion Core, which is probably the biggest inspiration for this library, but it tried to control everything. And instead, in this day of various modern frameworks and libraries, I wanted to give the developer the freedom to just use, you know, use a state chart as a pure function. And so you give it a state, you give it an event, and it just returns the next state

deterministically, and it will always be the same next state. And then you could use that next state to determine which actions to perform afterwards and how to change the state on your own. For example, in React, it would just be this.setState. In Vue, you would just mutate the object directly, I guess. And in Angular, you would use RxJS. Got it. I remember seeing in a talk of yours, you showed that almost like a prototype of XState was just an object, you know, you just have objects that point to other points in that object with like object selectors. Right? Yeah. So so why do you need a whole library? It seems like it's a pretty simple

idea. That's a that's a very good question. So if you are doing just flat state machines then of course you don't need a library and i tell people you do not need a library you could just use an object and it just becomes what i call an adjacency map which is similar to an adjacency list and that's just pointing like basically we're describing a graph right and the graph has nodes and edges so edges connects nodes and so if you have an object where all of the keys are nodes then you could relate them to different nodes via just another nested object so it goes node edges and each edge points to a node, which is just a string. And then so you could just do a simple object lookup to determine what the next state is. However, doing state charts, it gets a bit more complicated. And the reason is because state charts introduce concepts such as nested states, parallel states, and even things like on entry, on exit,

transition actions, transient actions, or sorry, transient transitions, conditions, all sorts of things that make it a lot harder to simply do in objects lookup. And so that's why a library is necessary. Got it. So I want to drill down on how XState and React, sorry, and like, I guess React or more specifically Redux play together, because I know that's a common question for people. Yeah. So where does XState live? Does it live in the model and like the store? Does it live in actions, in the reducer, in the view? How does it interact with those things? So XState at its core is just a function. So you define the machine. So machine, you give it the JSON

configuration, and then you have a transition method. And so that's your pure function. And so that can pretty much live anywhere that you need, anywhere that you need to determine the next date. In Redux, it sort of works like a reducer, because in Redux, a reducer is you have a state. And based on the action, you get the next date. The transition function works in the same way, where the transition, you give it the current state, and then you give it an event. And then the next state and various actions to be performed are returned. And so it could live as part of the Redux reducer, depending on which specific reducers you're using, or it could be a standalone reducer as well. Now if you're not using Redux

I'm not sure how it would work in MobX, but you could certainly use MobX DateTree to accomplish similar things But in React you could have a method on a component Like send and so that send is responsible for taking an event getting the current state from the component state, and setting the next state and also performing any actions that need to be performed. And that's something that I actually found was sorely missing from Redux and why so many supplemental libraries exist for Redux to bridge the gap of what to do when you transition to a state, but you also need to perform actions. So one simple example is, let's say that you are building a search application, and you had to transition from idle to loading when you started the search, but you also need to trigger a search, which sounds imperative. You need to perform that action of actually executing the search. So in Redux, you cannot do that within a reducer.

The reducer function must be pure. So you have to have another way of doing that. And so what ends up happening is spaghetti logic within your components where you're both dispatching a state change and you're executing that search action. Or you use various solutions such as Redux Saga or Redux Observable or Redux Thunk to do that imperatively, which both of them just sound very spaghetti. And so how does XState solve this problem? So XState, actually, I mentioned Redux Saga, but it does a similar approach. And what it's called is declarative effects. So instead of executing the action, your state that's returned from the transition function of like you know you're given the current state in an event that state objects that's returned contains both

the next state value such as loading and also an array of actions and so those actions are not executed they just say by the way, when you get this date, don't forget to execute all these actions. It does not execute it itself, but instead it gives that metadata over to the developer and the developer can do what they please with that. Got it. So I'm curious to know about the like current status of xstate like what i guess is it ready for production uh are people using it so xstate is being used in production and i was actually going to um try to dig around on twitter or github or wherever and uh figure out where exactly people are using it in production but i do know know one example. If you go to workshop.me,

Ryan Florence, who worked on that site, has created a, it's actually a pretty complex payment flow in the site. And he did it using Xdate. And I was actually really surprised to learn about this. So I went to workshop.me and I looked through their minified JavaScript code looking for that XState specific code and I found it I copy pasted the JSON into a visualizing tool that I made and I saw the state chart right there and I was really surprised

it's a very in-depth state chart that describes this complex flow wow that's a great story. Yeah. And so I do know that it's being used in other places. I don't have specific company names. There are people who say, you know, who are telling me like, hey, I'm using this in production in my company, and this isn isn't working or I would like this feature. So I do get a lot of those. So I do know that it's being used in the wild. And of course, I'm using it in my own personal projects as well. Yeah, I'm looking on the GitHub page,

you have 1800 stars. So that's like pretty legit. Yeah, yeah, very surprising surprising i'd be curious to know if you're like how did that happen was it from talks um do you have a sense of like how you gain stargazers uh well so first of all i don't really use stars as a proxy valuable metric it's just a glorified bookmarking tool. However, well, it's also somewhat of a sign of popularity as well. But I've given a few talks on state machines and state charts before, being React Rally at JSConf Iceland and also at ReactConf in Finland.

And so I guess I sort of advertise at those different places um yeah i i don't know where the other stars i i wish that github had a feature where it could tell you where these like how those stars came to be like where the people came from well you mentioned that you don't i i agree with you they're not like a great metric for anything um are there other metrics that you keep track of to know like how it's doing in the wild sure so i also look at um both insights and dependencies on github uh the insights are good for showing uh where a lot of the traffic is coming from and so a lot of it comes from the documentation page that i uh that I have up on davidkpn.github.io

slash xdate. But also on Smashing Magazine, someone wrote an article mentioning xdate, so not even directly related to it. And I guess there's some from news.ycombinear.com, which sort of scares me. Don't want to look there. But in some various Medium posts as well. In fact, there was one recently from, her name is Eli Schutz Ramirez. And so it was an article talking about, it's called Vintage Concepts, Fresh Applications, CS, or Computer Science in JavaScript. And it was talking about how, you know, this simple concept of state machines from computer science is extremely useful in doing UI development for the web. And so I guess that, you know, Medium and other various sources. Yeah. Yeah.

Cool. And so what's the future of this X state library? Oh, there's, there's a lot. So I'm continuously working on it and just always listening for, you know, issues and feedback. And one of the things that I'm trying to, well, I say I, it's more of a we, because there's a bunch of other contributors to XState. Eric Mogenson, you know, who made statecharts.github.io, he's been a huge help and a huge contributor as well. But I want to avoid design by committee, which is when just if someone

has some sort of clever idea and it sounds halfway decent, we just include it. I want to stay away from that because there's actually a standard called SCXML. And this standard has been around for about 10 years. And it describes state charts and it's interoperable because it's XML, it's interoperable between basically any language that implements some sort of SCXML interface and way to build state charts using SCXML. And I know C++ could do this with Boost,

I believe, Java as well, and I guess various other things use SCXML from the state charts that anyone would write in, you know, JavaScript and have that like viewable in any sort of visualizer or program that takes SCXML, which, I mean, there are a few that do that. Cool. I think I saw somewhere that two bullet points that I wanted to ask about. I saw that you, you mentioned that there might be like an interpreter of some sort. Yeah. So, so for now, and, um, just to give a definition to the name, an interpreter is, um, is something that takes the results of, or something that handles the transitions.

So since the transition function is pure, or it's more of a method, but it's pure, it just gives you the state and then you do with it as you please. So that doing with it as you please, that's a job of the interpreter. And the interpreter is in charge of actually modifying the application state and also performing any actions that the state returns. I was thinking about just including an official interpreter, just because there's various concepts such as delays, such as, you know, delaying an action and doing activities and just dealing with time. So I just wanted to make one as a starting point for developers. But mostly people who have been using XState,

they've made their own interpreters because they have some special business logic or they want to handle state changes in their own way. And so, yeah, that's definitely one of the goals. That makes sense. Another thing I saw you mentioned was like editable state chart visualizations. Yeah, yeah. So right now, I have on bits.ly slash xdate-viz. It's a weird URL. If you click it, it's even a weirder URL URL because I just whipped it up quickly. But it's a tool that you could copy and paste your state chart JSON from XState, and it visualizes it for you.

And you could also, you know, just go through the states and see how everything is working. But I am in the process of developing an application that not only lets you visualize state charts, but also modify them, you know, manage them, save them, test them. So generating automatic tests based on the state chart definition, but also like do analysis as well. And so if you think of a state chart, as a graph, you have many different paths that you could go on. And when we think of analysis, like just generally, we think of things like funnel charts, where we have a certain user flow, and we want to check that the user gets from point A to point B. However, you know, we're finding more and more that that's not sufficient because there's many, many, many different ways that a user could navigate an application. And this is where state charts come into play. And so if we could visualize just the entire graph of how users use an application, then we could derive much deeper insights into

how users use the application so that we could further enhance the application. So I'm trying to create basically a service that does that for developers. And it sounds like an analytics service? Well, analytics, testing, visualization, editing, storage, just a whole ton of things. Not unrelated to software like this that exists in other industries. It almost sounds like TLA Plus, or is this unrelated? Not exactly TLA Plus, because TLA Plus is more about formal verification,

which with state charts, it is somewhat possible to formally verify them. I know that there have been a few papers written on that, but it's more for practical usage. So some of the tools that are similar are like Qt Creator, also Mathworks Simulink Stateflow software, and Yakindu Statechart tools. So those three, obviously never heard of before in the web development space, but they are similar tools that exist forState open source umbrella or it almost sounds like it could be like a SaaS company? Right. So I'm planning on making that part a SaaS company. know for developers to use but i also think that there's a lot of value in in having software that basically saves tech teams lots of time and lots of money because that's fewer tests that you have to write and fewer headaches with determining what the complex business logic is and also it makes it easier to to enhance the software you know, as an adding features, modifying features, seeing what happens if you remove a state and just manipulating, you know, just manipulating states.

And so obviously it's going to be a SaaS because the data needs to live somewhere and that's not free. But my ultimate goal is to make something like this completely free for any open source project. Oh, cool. And so this service, I'm curious how you see it interacting with the other code that developers write? Or do you see it totally replacing code? Or how does it hook in? That's a really good question. So I guess this is more of a general question for X-Date as well, because I've had people ask me like, you know, X-Date, this whole state chart thing, it seems really cool, but

I have legacy software. I don't want to redo my software. I'm not building anything new. So unfortunately, this isn't much use to me. And so just to give a story, I gave a workshop over at React Finland, where we had to build an app and I wanted to show them how state charts worked and how it could be used in applications. But the first thing I told them was I gave them the business requirements, which it was pretty simple. It's a feedback form.

So first it asks you a question like, how's your experience? And then you can click good or bad. If you click good, it says thanks. If you click bad, it asks you, or actually if you click bad, it says thanks as well. But later we changed that so that if you click bad, it asks you why, and then you had to input why, and then it took you to the thanks page. And then of course, you could exit that screen. So I told them, those are your business requirements, open up React, and just do it however you want. And so they did. And so that was me showing them that

it doesn't matter how you write the software. It could be legacy software that you wrote 10 years ago. It could be Greenfield software, doesn't matter. What we did from there was we learned about state charts. And then we built a state chart that describes the application in a very abstract way. So we said that the initial state is this question state, and then there's two actions. You could either click the good button or the bad button. And then, sorry, events, good or bad events. And then those events transitioned to, say, a form state

where you could say why your experience was bad, or a thank you state where you just had this confirmation. And so we modeled that as a state chart, or a state machine at this point. And this is completely separate from the application code. Now, using this state chart code, we took that and we generated all simple paths. So all paths from the initial states to every single other state that a user could navigate through.

And so I think since this was a simple application, it generated a total of 24 or 26 full integration tests. We used those tests combined with a couple of heuristics to test our application that we made. So remember, no code was modified, no code was touched. We don't even know what the code looks like. And for most of the people in the workshop, they said like, okay, awesome, all the tests pass, which means that the state chart accurately describes the the business logic of our application and so for those who um for who

the tests didn't pass it turned out that they had bugs so they went and fixed those bugs and then the tests magically passed and then from there we were able to refactor the code confidently, knowing that our state chart described our application in a robust way. So it makes a great refactor target, basically. Got it. So I'm trying to like kind of build an analogy of how the service you're building, the SaaS service you're building, so it kind of holds on. It's almost like a GitHub, but only for your state charts, and then your other code lives in the actual GitHub. Is that kind of the separation? Sort of.

So at the end of the day, a state chart is basically just a big JSON file or an XML file if you're using SCXML. And then you could use those files however you want. You could use them within your app, put them in an interpreter, and then use that to deterministically manage your application state and any actions that your application, you know, may use. So it's all up to the developer. And so that's why I think a tool like this would be valuable because it's framework agnostic and it's just going to simplify and visualize a lot of the process. Plus, if you're using it within your app, I was thinking that there could also be ways to debug or not debug, but just analyze an app in real time. So as a user is using the app or,

well, I guess due to privacy concerns as a developer is using the app, they would be able to see on a visual representation of the state chart, what state they're in, and what events happened. Got it. It's almost like a developer tools experience, like the Redux developer tools experience. Right. Redux, but less text, more visual. Yeah, yeah. I'm totally with you.

I'd be curious to hear how you think your tool would compare to Kevin Lina, his Sketch Systems tool? Yes. So I actually talked to him about that. And I went back and forth with him on his ideas. So I think that Sketch.Systems is a fantastic way of prototyping systems and getting that much closer to bridging the gap between design and development. Because unfortunately, we have all of these awesome design prototyping tools, and nothing really that fully describes the behavior of components. So all of these prototyping tools that you've seen, such as Sketch or, I guess, Envision or Figma,

all of those are mainly focused on how different components of your application look. But they're not focused on how they behave. And so this is where Sketch.Systems bridges that gap because now it becomes possible for you to describe how your components in visual layout, just various parts of your application should behave, which I think is even more important than how it looks. And so it's very useful for bridging the gap. With my software, what I'm planning to do is have it be much more on the development side.

And so it's sort of a carryover of just designing the behavior of components, but actually being able to use that meta information directly in your code. Got it. Well, I'm curious because right now you can use Sketch.Systems to export into your XState library. Yeah. So, like, what like, your visual editor, like, how would it differ? Would it use, like, the same

kind of syntax, or would it be more direct manipulation, editing nodes? Or would it be, like, because I know you have a JSON XML thing you've been mentioning. So is that the way you edit the, say, chart? So I actually want it to be very visual. And we were talking about this earlier.

But I'm very influenced by programs such as Facebook's Origami, in which it's just a complete visual manipulation design tool, and you could visually see how various parts are connected. It's also similar to Unity's tool for managing finite state machines, and actually Unity does hierarchical state machines as well. And so, you know, even in the game development world developers are using visual tools like this to uh with direct manipulation and just that huge visual aspect in order to create these complex animations for games and so that's what i want to do as well just have a very visual interface like i i would love to just even open up my, you know,

my Surface Book Pro or my iPad tablet and be able to drag around different nodes and, you know, edit an application from just touching. Cool. I got it. So I want to move on to a related but slightly different topic that we're also both interested in, functional reactive programming and so i um i saw uh two in two different places i saw these quotes of yours in one place you cited connor elliott and paul hudak's original paper about

functional reactive programming so um that tells me that you're like you know one of the original you're into the original thing you're not just like a bandwagon React person. Right. And I saw you said elsewhere that React is not really reactive. It's more of a scheduler thing, which is well. So I'd be curious to learn about kind of how you see FRP and I know there are a lot of different flavors of FRP and I don't know just I kind of want to get a sense of how you how you see it and what you think is important about it right well if you really dig into it FRP is uh very very mathematical and there's probably people who know a lot or well there's definitely people who know a lot more about it than me, especially the original, quote unquote, functional reactive programming. Because functional reactive

programming is based on continuous signals and things such as RxJS are more based on discrete signals and push versus pull, blah, blah, blah, all that. But so with reactive programming it's just the idea of of treating time as a first class citizen and so i think that pairs nicely with state machines and um and yeah i actually forgot the original question no yeah i didn't really ask a great question um but you said the um you you answered it the way i was hoping you would uh um what i um and and you actually mentioned the thing that i wanted to ask the follow-up question on um so um this interaction of like x state and state charts and reactive programming i find fascinating um because like part of what's so great about observables, and I think I saw you explain this in one of your Keyframers videos,

observables are like events, but instead of a one-at-a-time kind of world, you can grab all the events of all time for forever. Because like you said, functional reactive programming lets you kind of play God with time. But then on the other hand, state charts and X date, it almost feels like in those worlds, you kind of take events one at a time like actions. It feels a little bit in the old world to me. Do you see what I'm saying? Like that kind of contradiction that they like don't seem so compatible? Yeah, I don't really see that as a contradiction just because, well, for one in RxJS,

I use the scan operator a lot, which scan basically works as an intermediate reducer, where, you know, given a state, you take, you know, some events, and then you transform that into a new state. And so that's exactly how these date machines and state charts work. And so I think that they actually pair quite nicely together. And I've actually been working on an interpreter for XState based on RxJS. I plan on creating a simpler one just based on native JavaScript, just because that's easier to do. And people don't really like dependencies. But with RxJS, like you said, you could play God with time. So if I want to test an event that happens, that's delayed by 10 seconds, I could manipulate time and say,

hey, that 10 seconds is really 10 milliseconds, just so that my tests run faster, of course. And so I actually think that they do pair, you know, pretty nicely. Also, state charts, when people try to manage and refactor their application into state charts, they end up sometimes coming up with like questions about how you model certain complex things because the state chart could do this and that and you know having parallel states but according to both the original paper and scxml state charts can communicate with each other and so it's this whole idea of event passing where one state chart can admit its states to another state chart who could respond to that state and then you get this like network of state charts sort of like you know galaxies in the universe

right you know you have all these solar systems and every they they form these groups and then you have like these big galaxies that's sort of the same thing with state charts i don't know if that was a good analogy sort of get the idea but and so reactive programming um it basically works the same way it implicitly uh creates a a directed graph of like just observables that are dependent on other observables and um so it actually makes makes event-based programming a lot easier to think about because you're not forced into this hierarchy of, like say in React, you have to pass from the top down with observables. It's just you connect to whichever observable you're interested in. And so one of these state charts could easily be an observable where it takes an event and it emits new states and the actions to be performed, which can be mapped to more events. something you mentioned earlier in this conversation uh you mentioned how like facebook origami is kind of like an inspiration for this other software you're working on to visualize state charts so i kind of want to dig into that because i also am a software developer not a designer who

origami was built for but i'm i'm also interested in origami as how it points towards like building better products for software uh engineers because i think I think it really does a good job of pointing towards the future of programming. Right. Yeah, I played around with Origami. Of course, I'm not a designer per se either, but what I like about Origami is that it really highlights

just the intuitive parts of functional reactive programming. So for those who aren't familiar, origami, you have these things called patches, and these patches have a series of inputs and outputs. And just to be clear, patches are just nodes and patches with wires. Right, right. So you would input certain, like uh so you would have source patches right and then those would emit certain values to other patches and those patches would

react to those values and just output other values and so now you have this created uh this connected network of different patches uh which all just do very specific things. And it's mainly used for just describing the complex reactive animations that are present in mobile apps, which is why it's very useful for designers so that they could accurately describe to developers what sort of reactive effects they're looking for,

where they can't just make an After Effects movie or some sort of GIF representation. They could actually show them an origami project and say, like, for example, when you swipe to the right, these move faster than this one, or this shows, this hides all of that complex logic. Totally. Yeah.

And so that fits in very well with functional and reactive programming. And whenever I play with Origami, I think, man, we could definitely make something that works with RxJS or most.js or whatever your favorite functional and reactive programming library is. And we could make a visual tool for that too. And so I definitely have a lot of ideas around that as well. And there is a language that does that. It's called luna-lang.org.

So Luna is a functional programming language. And it's also a very visual language, too, in which you could both code and have things represented in terms of a graph. Interesting. Yeah, I've played with Luna as well. My gripe with Luna is that it it's missing the source events like you can't derive events from like button clicks and key presses and stuff. You can't actually make interfaces, which is what you and I are like kind of all about. But I agree that it's, I totally agree with you. And I actually kind of feel like we should talk more offline about

that visualization of X, like RxJS and stream libraries, because that is, that's kind of what I'm working on these days. Really awesome. Yeah. My current thesis is that um the the the problem like part of why origami works so well is because it uses a limited form of programming functional reactive programming which doesn't have a lot of the features of imperative programming um but uh origami doesn't scale to like do all the complicated things you'd want to be able to do

and my thesis is that you, we have to limit programming even more. Like there are certain things in origami, like pulses that aren't very functional reactive that we, that we, if we like continue to like make origami have a model that looks a lot more like kind of Elliot's like original conception of it, then we'll be able to actually do it.

So I'm not sure if we'll be able to build the visualizations we want in like RxJS or the JavaScript-y function, like FRP observable libraries, because I'm not sure they're pure enough. And I think you need the purity to make visualizations, but I'm not sure about that. That's my current thesis.

Yeah, no, I agree. Well, so one question i wanted to ask is um about how origami kind of relates to state charts because i can definitely see how origami would inspire you to have that vision you mentioned earlier where as you're interacting with an app you can see yourself being moved through through the state chart because in origami as you're interacting with an app, you can see yourself being moved through the state chart because in Origami, as you're interacting with the app, you're seeing the values like numbers and objects flowing through the application and it's like unbelievably beautiful in a brand victory way.

So how does that work with state charts? How does it come together visually? So state charts embody the idea of just events-based programming where everything is an event and everything you do is an event that could cause state changes as well. And so I think Origami, it's more the visual side of just connecting nodes to other nodes via edges and doing so in a directed way. But also the idea of combining the ideas of, like, functional reactive programming with state charts.

And so that's something that Origami is sort of missing. Like you said, there is an upper limit, and there are limitations to just how complex logic you could describe in origami. And I definitely think that state charts can fill that gap. I remember I was... I do a lot of fun animations on my CodePen where I would see something very intricate and very fun and animated on dribble and I would try to recreate it and I've definitely run into roadblocks before where just the logic was very complex because it's like if you swipe up one way then some animation is going to happen depending on which states you're in. And so once I refactored to use state machines and state charts,

then things became a lot more clear. For example, just like swiping or just certain taps or using two fingers to zoom or just any various complex animations like that sort of requires you to think about it in a modeled way and think about what's going to happen if I'm in this state and the user does this versus when I'm in some other state and the user does the same thing. And so that's where state charts can definitely come into play. So I just envisioned this visual interface where we have many state charts. And so these state charts look like these origami patches

and they're all connected via links. And then you could just see your application in real time visualized, very similar to how origami visualizes applications. And it's hooked up to your live application, so you don't have to... Yeah. You got it yeah

cool right so that's the future i envision awesome yeah my future i envision intersects with the future you envision awesome um so you mentioned codepen and i want to talk about that a bit too because um i know you're big on codepen and it's this weird kind of community so i wanted to hear about you know how you got into it and what you think of the community. And also you have this live stream YouTube thing, Keyframers, and I wanted to talk about that. Yeah. Yeah, so CodePen, I got into it a long time ago, I think over three years ago. The thing that I like most about CodePen and the thing that I find most valuable about it is

its community. It's very much community-based where you're not just creating a throwaway pen, but you're actually showing something off to the broader front-end community. And CodePen will highlight these pens by either picking them or putting them on their Twitter page or things like that. And you could also fork other people's pens and work based off of what they made. I started off creating CSS-only concepts with CodePen, just taking a look at these awesome animations that I found on Dribbble and thinking there's got to be a way to do this in only CSS, so no JavaScript at all. And so that became a really fun challenge for me and a challenge that other people found very interesting too, because now I was using CSS in these new and different ways where it's not typically used, such as checkbox hacks or just various transition functions and things like that.

But it became a really fun challenge for me, and it got me thinking more and more about how we could declaratively represent these complex animated concepts in a declarative way. So at first it was only CSS, and then I realized, okay, doing it with just CSS is not going to cut it for everything. There are some things that are quite frankly impossible unless you use JavaScript. But I wanted to carry that declarative nature of CSS over to JavaScript,

and so that's when I started thinking about using RxJS for animations and using state charts and state machines to represent animations as well, which there are quite a few animations on my code pen where if you dig into the JavaScript code, you will find very explicit state charts and state machines within there. Yeah, I wanted to ask you about CSS because it's a declarative language. And as we talked about, into declarative languages, but a lot of times when I am spouting the virtues of declarative languages to other people, they come back at me with, you know, like, well, CSS is declarative

and CSS is the worst. Or they'll say, you know, instead of CSS, they'll say the same thing about SQL. So, but it sounds, you are someone who spent a lot of time with CSS. I guess you're kind of on the cutting edge of CSS. So I'd be curious to get your sense of, you know, what you like about it, what you don't in terms of declarativity. And if you had any thoughts on it, like how you would improve CSS, if you could. Right. So CSS is, I mean, I guess this is the gripe of others. CSS is underpowered. However, it is deliberately underpowered. In fact, on the W3C page, it cites the principle of least power, which means that, you know, a domain specific language should, you know, should limit itself to do, you know, only the bare minimum of what it needs to do. In CSS, it's, you know, styling in a

very declarative way using selectors and different rule sets and just all of these specificity aspects as well. But that's important because having a fully declarative language is that some things do become impossible, especially if taken in the perspective of, oh, I could just do this in JavaScript and, you know, it would be easy. But people who say that are not seeing the full picture. And the full picture is this. Let's say that you weren't allowed to use CSS and instead you had to use Canvas for everything. Okay, sure, you could do it. Canvas has a bunch of imperative APIs.

You could do basically whatever you want. You could draw anything. But then try to debug something in Canvas or try to inspect like just a circle or a shape that you made and immediately you see the downsides of doing something in a fully imperative very powerful way which is using javascript with css because there's essentially only one way to do things in one way to achieve layout and style and color and fonts and all of that you have these very rich dev tools. Even for animations, you have these dev tools where, like, I love the animation dev tools both in Firefox and Chrome because you could scrub through animations, you could slow them

down, speed them up, modify them, and just visually inspect them. You cannot do that with, let's say, Canvas or using Request Animation Frame because basically it's a free-for-all. There's no declarative representation of what you want to do. And to me, those visual tools are extremely important, especially when developing complex software, especially in a visual way. So that's one of my favorite parts about CSS and having these declarative languages. And I think that developers are too spoiled in thinking that, yeah, I could just do JavaScript however I want, just write imperatively.

And they think that the end goal is that their app works. Sure, they could manually write hundreds of tests or whatever, but they're just like, okay, my app works. It's done. But to me, that's not being done. Being done is, can I analyze my entire app or my entire layout and everything, and just be able to understand how the entirety of the app works and be able to generate tests automatically for it. Even with CSS, you could do this as well.

You could generate visual regression tests, but also you could just analyze like different sizes and colors and just make sure that things haven't changed, you know, just between revisions and also that things work responsibly and all of that. And so I think that there's a lot of value in having a declarative, a fully declarative language and sticking to declarative concepts. Now with CSS, I do wish that it had more powerful declarative animation tools, because there's a lot of tools that sort of fill that gap, such as GSAP, which is very imperative. I must be the only front-end developer who doesn't really use GSAP.

I try to stick with CSS animations as much as I can. But yeah, I think that CSS, you know, one day, maybe it will get more powerful animation tooling, in which you could describe more complex animations. But CSS variables are good for that, too. Yeah, are there other JavaScript animation libraries or CSS animation libraries that you use and you like better that than gsap so for dynamic animations um i will either use rxjs like i'll just straight up use that just because gsap isn't really useful for dynamic animations it's more for static timeline animations um and for for those animations, I would try to use the Web Animations API as much as possible, especially because now most of the API is implemented across Firefox,

Chrome, and Safari as well, or at least Safari technology preview. And I think it's just going to come out in the next version, which is extremely exciting. Not Edge yet, but you could easily just fall back to no animation. In the description of Keyframers somewhere in the internet, I saw that you talk about, like the point of the show is to construct interesting user interfaces. I'm someone who also was obsessed with the construction of user interfaces, but I sometimes have trouble explaining to other people why I think it's an interesting field, like as opposed to the construction of databases or just like other software things. So I was wondering if you had any eloquence on that topic of why the construction of user interfaces is a compelling study.

I think it is because it exists outside of a vacuum. We have many different browsers that users use. And like, for example, with databases, you could control the environment with which it's in, especially with tools such as Docker. You have that all nicely contained. You could predict exactly what's going to happen. With user interfaces, it's a bit different. We have all these browsers, all these different devices. A user could be using your website or web app using a mobile phone, tablet, laptop, desktop,

computer with a, I don't know, 40-inch ultra-wide screen. I don't know. And so there's a lot of different considerations to make when constructing user interfaces where it's just not just a simple user interface. You have to take a whole bunch of different variables into effect. And so there's a lot of environmental concerns like what's going to happen? What the user going to do and how do we deal with different types of users uh for example accessibility like how do we treat our visual user interface uh and make it work for someone who's blind or deaf or hard of hearing or just a combination of all these things and so there's's, there's a lot, a lot more considerations with doing user interfaces. And that's what I find so interesting. Cool. Yeah. Same. Well said.

Um, all right. Well, I had one other question for you. That's kind of, um, out there, but it brings this conversation full circle. So we've talked about improving programming and making it more like musical notation. I was wondering if you, who were thinking about all these ideas of improving notations, have any ideas on improving musical notation itself? I know it's been around for a while, but I was wondering if you had thoughts there too. There have been many attempts to just, you know, modernize music notation and create, like, you know, basically use colors or different shapes to describe different things but I think that what a lot of people don't know about music notation is that

it's evolved over centuries like you know we started with like the original Gregorian chants where you know you had these weird boxes and lines and then that eventually evolved into modern musical notation and even with modern more abstract music um we didn't change musical notation instead we added on to it you know with certain symbols like especially for things that had no time signature key signature or you know atonal. But basically, it evolved from these core concepts over many, many, many centuries. And I think that comes back to programming too, because a lot of people are trying to invent new ways of, states or just events or whatever. But there's all these concepts that are literally decades old, if not half a century old, that we could just apply to our current work,

and that other industries have been applying to their current work with success. I mean, the reason you and I are alive today, probably, is because the hardware that we relied on you know didn't accidentally kill us and the reason that that happens is because the software and hardware was modeled in a way over you know many many many decades using these principles in such a way where we could you know just predict any sort of situation that might happen and be able to thoroughly test these products and make sure that it does what it needs to do. So sorry, just to be clear, what industries are you talking about?

Well, just various like embedded electronics, hardware, automotive, avionics, just anything that deals with the combination of hardware and software. And what kind of tools do they use? Do you have names or something more specific? I don't have names specifically, but I do know that, like, for example, I was talking to my friend who does audio engineering, and he said that some of of his colleagues use for example math work state flow uh software uh in order to you know just model like the for example the design

of chips like electronic chips that's all essentially state charts and state machines and so i you know i might be wrong but i'm not too knowledgeable on you know just these chips but it's it's very like it's a circuit you know you you have like different nodes and it's it could be modeled as a graph and uh yeah and and so like the the example i give for state machines is always uh traffic lights that's like the hello world of steam machines and so i mean imagine a javascript developer programming a traffic intersection that would be pretty scary so you're saying that um you want to bring some of the like tools that other software industries are using to web development or Or basically using the same concepts.

Yeah, the same conceptual tools is what I meant. Yeah, yeah, which is essentially just explicit software modeling, which is something that we're sorely missing in the web development and mobile development industry. Cool, I'm agreed. On that note, I wanted to close up and give you an opportunity to mention the places you exist on the internet, or how to reach out to you. I know you have a Patreon, so if you wanted to mention that. Yeah, sure. I'm on Twitter, or I'm basically everywhere

at David K. Piano. So Twitter, GitHub, Facebook, Instagram, all of that fun stuff. But also I do these animation screencasts with my friend Steven Shaw. And so we're over at twitch.tv slash key framers. And we do have a Patreon set up because we're trying to get better equipment for streaming and all of that. And we basically break apart just these fun, complex animations that we find on Dribbble. And we program it together, just do some insane pair programming. And for some of them, we do use state machines. And I show people how useful both state machines and reactive programming and how useful they could be. Sweet.

So thanks so much for taking the time. This was a lot of fun. Yeah, thank you.