Future of Coding - Research Recap Four

Episode Date: August 28, 2017

After coming back from Boston, I did a deep dive into Jonathan Edwards, Jaime Brandon, Conal Elliot, as well as spending a full day reviewing Eve (Chris Granger). Towards the end of the week, I released a simple prototype for StreamSheets and send it over to Andre Staltz for ideas and feedback. Notes here: http://futureofcoding.org/journal#research-recap-4Support us on Patreon: https://www.patreon.com/futureofcodingSee omnystudio.com/listener for privacy information.

Hello and welcome to the future of coding. This is Steve Krause. So over the past two weeks I was able to follow through on my plan that I set during the last research recap from two weeks ago. I mostly spent the last two weeks reviewing other people's research in this space, including Jonathan Edwards, Elliot Connell, Jamie from Imp, I researched Eve, I spent a whole day on Eve, and a bunch of other scattered links. While I was able to be pretty productive over the past two weeks, I did lose a day or two of research to my other project the coding space my day job we are gearing up for the fall we're trying to teach 400 students a week and we're going to need about 40 part-time teachers in total we've hired about 15 or 20 at this point so i had to do we as a

company have to do like 50 or 60 more interviews so So I was taking a big chunk of that on my plate. So that took up a lot of my time. But lucky for me, my teammate Sophie is taking that over starting this week. She took that over. So I can get back to focusing on my research, which is great. For a hot second there, I thought I was going to have to put my research on pause for a little bit longer to get that off the ground, but luckily I can stay focused. So let me tell you about some of the research I did.

So I met Jonathan Edwards in person in Boston a few weeks ago, and I really enjoyed getting to meet him. I didn't quite realize until I did research in preparation for meeting him how similar his vision and my vision are. I think really the main difference between him and me is that he's been doing this for like a decade, like a decade longer than I have, which is amazing for me that I can learn from him and kind of read his blogs over the past decade and use him as a resource. So that's really lucky that I'm connected with him, that he exists and that he's been

working on this for so long. I was surprised by his manifesto. He has a few different manifestos, but I was surprised by how each of them are similar to my goals for the future of programming a decade later. Things haven't changed very much in the last decade, and the goals for people who are thinking about the future of programming are pretty similar. One thing I did notice is how much time he spends talking about the other hot new companies and fads, and how much time and energy and headspace those things take up for him. It's easy for me, looking back, to see, oh, that was just a fad, and discount something but I could see how for him fads you know are big deal you can't tell that they're fad

in the moment it feels you know more like a wave and you want to catch the wave and everyone you talk to is talking about the wave so it's like impossible to tell that's a fad or it's very hard. I guess, like right now, blockchain and AI seems to be hot topics that everyone's talking about. And it's hard to tell. Are they fads? Are they real? Do they have real momentum and weight behind them?

My gut is that both of them are overhyped to some degree. I don't exactly know what degree. I think blockchain is definitely overhyped. I think the ICO craze is insane and there will definitely be a crash at some point soon. And then as far as

AI goes, there definitely has been progress, but it doesn't seem to live up to the hype. But I think it's more in line with reality, the AI hype, and the blockchain hype is just kind of a bit out of hand. But it's hard. I don't really know. And there are other things that I don't think are fads, but they might be, like unidirectional data flow,elessness like the react architecture um strong types to me these things seem great and and everyone else happens to notice it right now uh but but you know who knows they could be fads i could be wrong another thing i noticed about jonathan is

that he spent a lot spends a lot of his time submitting papers to for to conferences and worrying about the and getting them back and getting rejected and worrying about it and and now he like you know runs a lot of these conferences and chairs them and reviews other people's papers and it's unclear to me what kind of value you get out of that like I said last time in the last research recap, when I kind of recapped my trip to Boston, I'm not sure if those sort of conference-y things are kind of my scene, or that useful to me. I think I can get the value from a conference more a la carte

by reaching out directly to people who are interesting to me and asking them more specific questions rather than a more spray-and-pray approach of going to a conference and talking to random people. So I don't think I'm gonna go down that line of inquiry. It's also really interesting to see how much he jumps around, how he changes the name of subtext, and he has syntax and he has no syntax, he used to work on frontend things, on backend things. It's just interesting to see how he jumps around. And there's a lot that I can relate to there. how he jumps around. And there's a lot that I can relate to there.

So he has about 100 blog posts over the past decade. And I read them all. It took me about a whole day, maybe more. Jonathan spent a lot of time switching languages and frameworks from Java to Scala and Back and JavaScript and Dart, etc. As I'll talk about in a bit, the Eve guys also switch their runtime and frameworks a lot. Jamie from Imp also does that. It seems like you can spend a lot of time doing that,

waste a lot of time doing that. So I would like to avoid that if possible i haven't really fallen into that hole i usually just stick to some flavor of javascript to prototype it and then i decided i don't like the prototype and end things there but i guess it's you're not wasting that much time to rewrite things so so yeah he there are a lot of other specific notes about things Jonathan Edwards worked on that I found interesting. And I have those notes on my website. If you'd look, I'll link to them in the notes for this podcast. But that's the synopsis. So after Jonathan Edwards, I moved on to Conal Elliott, who I thought was really interesting, especially as a foil for Jonathan Edwards.

Conal Elliott, he's like much more of a Haskell guy. And Jonathan Edwards, although he knows about that world, I think he said Jonathan Edwards says he used to be kind of a Haskell guy, Jonathan Edwards now thinks that Haskell is much too much like math and programming should be more like conversational. Colonel Elliott thinks that beautiful abstractions is kind of more the way to go. And so he has some really interesting ideas about how to design things. He has this thing called denotational design that Paul sent me,

which I like the idea of, but it also feels a little bit too pie in the sky. For example, I'm thinking about how to design stream sheets, a prototype that I've been thinking about for a little while. And it's, you know, I think about the API and the types, about the core data structures, and I did that, I did some denotational design for that, but at the end of the day, the interface, the metaphors,

the things that are really important to me, I find are hard to articulate from a denotational design perspective. It's a useful technique. I find myself more on the Jonathan Edwards side than the Colonel Elliott side of my way of thinking. Although I do think there's a lot to learn from the Colonel Elliot side of my way of thinking. Although I do think there's a lot to learn from the Colonel Elliot side. I read and watched almost everything I could find of his. In particular, he has this language, Fran. It's like a Haskell animation library,

which is super similar to the idea I've had for making an Elm animation library or a more reactive version of WoofJS. So that was really exciting. I think so much of the work I was going to do on that has already been done for me by him. So that was really exciting. One of the things that Connell Elliott said that I really can't get out of my head is that if we do things right, abstractions will constrain us and disable us from getting

certain kinds of information or doing certain things. For example, if you have an image and you can do certain computations on that image, you shouldn't be able to know how many computations have done an image because an image plus a computation is still just an image so it's not an image plus a computation it's just you know it's just another image so part of me likes this because it's a non-leaky abstraction and it allows you to make really cool like optimizations under the hood and it's pure in a lot of ways. But part of me worries that it's too constraining, that you can't build a metaprogramming system

in something like that. Part of me worries that I want the metadata. So that's something that I've been thinking about a lot, that particular quote. so last wednesday i spent the entire day with eve i've been meaning to do this for a while now because i've been admiring chris granger and light table on the eve team for years now i really love chris granger's blog post coding is not the new literacy and i the whole team, I love basically everything that comes out of those guys' mouths. I love everything I've ever seen them do. Those guys are definitely my role models in so many ways. So the fact that they have a prototype that's out in the world is super exciting. And yet every time I went to go and try and figure it out,

I got really confused and stuck and just gave up. I don't think I got more than like 20 minutes in at any time over the past years that Eve has been around and able to be played with. So I decided that last Wednesday, the whole day would be reserved for figuring out Eve. And so I did it. I'm very proud of myself, as you can tell, that I stuck with it. And I think I have a better sense of why I had so much trouble sticking with it all those other times I tried, but didn't reserve the whole day for it. It's pretty buggy, their interface.

And it's tricky. And they're figuring a lot of things out and they're making it better. So I guess what I'm trying to say is that it's bad. It's not intuitive. It's not easy. And it's buggy. So it's really, really hard to learn and get your head around. There are a lot of parts of it that I love.

Those guys are really smart and they wouldn't build something bad, but it's super frustrating. And I barely made it out alive kind of thing. If it was any buggier or any worse, I may have even given up, even though I reserved the whole day for it. It took me a lot of concentration and focus to make it through.

So let me tell you about it a little bit more specifically. So the first problem is that their Quickstart tutorial, their version 2 that's on their website, is really buggy. So it's unclear when the code is being run and when it's not like it's unclear when their code is just when their interface is not running your code and when your code is broken it's entirely not clear so i did a lot of like adding random enters white space to like and then hitting the refresh button and refreshing the page just you know every time things didn't work i had to like kind of monkey with everything to make sure

that like you know to reset the state on their end to make sure that uh things weren't working because of me and not because of them so that that was that just like multiplied the amount of time it took to get anything done by two or more so So that was a bummer. Another big bummer that kind of took an hour to figure out is whether or not the version on their website, play.witheve.com, is version two or three, because when you click Documentations, there's a button that says, you know, version two documentation and version three documentation.

And I didn't know which one it was. There was no sign anywhere on their website to help me figure out which version I was using. So I spent a long time reading until I figured out through inference and process elimination that version 2 is their website and version 3 you can only use if you download EVE and install it on your computer and use your own text editor. So I figured out that I was on version 2 on the website. Unfortunately, the docs for version 2 are very incomplete.

There are a lot of gaps. There are a lot of sections in their docs, like headings that don't have any text underneath them. So that's a bummer. And it's hard to stack overflow things, because there really isn't. I googled a few questions, but it was hard to find answers. They have a Slack group that I saw linked to in their blog. However, when I clicked on the button, it was broken. So I couldn't join their Slack group.

That button seems to be fixed now, but it was broken at the time. So I was kind of on my own. They don't really have error messages. When your code is syntactically okay, but things don't really make sense, it just kind of fails silently. So I have to do constant sanity checks. Every change I want to make, I have to make the smallest possible change and work up from there to see when things break so it's really slow going especially because their interface is buggy so I make a small change if it doesn't work I have to add random whitespace to the program unclick and reclick things refresh the page to make sure that things are really not working and it's not just their interface that's bugging. So that's probably the biggest thing that slowed me down.

I didn't like the set-based model of things. That was not intuitive to me, that you can't have more than one of the same value in a database. I kind of wanted every new thing to have an ID attached to it so everything was kind of unique implicitly. I guess I could have just made a random number ID on all my data if I wanted to, but I spent a lot of time with that particular bug. I wanted, when you click a button, to add something to the page.

And I set up that code to work, except the thing that I was adding to the page is the same. I wanted to add the same thing to the page every time. And Eve, you can't have two things that are the same in the same set. So nothing was happening. So that was super frustrating. It would solve the problem if they had some sort of interface to visualize queries a little bit more. But given that they didn't have that,

I had to print random divs to the screen, and that was a little annoying. There are other specific things i didn't like about how you have to like specify the database that you're searching in and there's like if you don't even put a database there's like an implicit database that it searches in i couldn't tell what that meant but they use that in their docs a bunch so that was confusing um i i basically spent the entire day trying to make a counter work. So you click on a button and the number inside the button increases the more you click on the counter. Pretty simple. It took me like five hours to do. And you can kind of read through my stream of

consciousness. I tried this, didn't work. I tried this, didn't work. Tried this, it didn't work. It goes all the way down. So you can kind of read through that if you want my whole experience with EVE. On the positive note, I really liked how everything has a uniform record syntax. It's pretty similar to Lisp in that way, which begs the question, why don't they just use a Lisp syntax? And it turns out there is a Clojure library

that's very similar in style to Eve that I found on the Eve Google group. I'll link to that in the notes. I really liked how the core data structure is a record and database. You know, everything it's very like SQL like, which I'm a big fan of. I really like how the API simplifies everything to either searching or adding things to a database. Now, there are two different ways to add to a database, you can either commit or bind. And, you know, I had a lot of trouble with the difference between

commit and bind. So if I had my druthers, I'd probably just do commit. And then if you want a bind like interface, you know, you have to monkey with things or vice versa. I think one should be built in the other. But I really like how they simplified it. It's just three things. You're either searching or you're adding. And then everything is a database. So it's similar to CycleJS, where it's either a source or a sync.

It's very, very straightforward. I really like how they make it simple that way. So one quote in the EVE Google group that I saw, it was a quote about an aspiration that the Eve team wrote. It says, in Excel, they show the data and hide the formulas. In programming, they show the formulas and hide the data. In Eve, we show both. And I really love that quote as a vision for programming. But Eve doesn't live up to that at this point. Right now, Eve is much, much closer to Python than it is to Excel. So that's Eve. I sent all this to the Eve team and got back some really nice emails. They responded to my individual

complaints. And like right away, basically the next day, they responded to almost every single one. And they said they're going to improve upon a lot of these things in version 4, which is exciting. And hopefully, yeah, they'll keep working on it and it'll get better and better. Those guys are really smart and clever and nice and whatnot, so I wish them the best. So then the next day, week I did a deep dive into Imp and so Imp is created by this guy Jamie I don't know his last name but he used to work at

Eve and he has a website called Scattered Thoughts and so I think I took about two hours to skim through or read his entire development journal for Imp, which happens to be the inspiration for my development journal. So even though I hadn't read his development journal until last week, the fact that it existed inspired me to create my own development journal, which has been super helpful for me. So thanks, Jamie. So I read through his development journal, which was really interesting. He spends a lot of time on optimizing his compiler and runtime and making things fast and work, which seems a little bit irrelevant to me for the problems that I care about, but I think that makes a little bit more sense

given what he's concerned about working on. And then I really loved his website. He had some amazing posts. He had a post about rock climbing I sent to my co-founder, and he really liked it. And he had a lot of books. We've read a lot of books in common, and I have similar ideas about them.

And there are a lot of books that he's read that I haven some similar ideas about them and there's a lot of books that he's read that i haven't so i bought like a dozen books after reading after digging through his website and um yeah i was really impressed by the stuff he was thinking about so i sent him a long email telling him uh about all this about you know how i stalked him on the internet for a few hours. And he wrote back the next day, and we're going to find time later this week to chat. Hopefully I'll get him on the podcast soon. He seems like a good guy.

Seems like, you know, maybe I'll make another friend, which is exciting. So then, Friday and today, I decided to do some research into spreadsheets and streams and start planning out stream sheets. First, I wanted to take Colonel Elliott's advice and do the denotational design thing. Think about the interface to streams and what you can do with them, what you can't do with them, that kind of thing. So I did that on Friday mostly. And it was semi-helpful. I'll link to the pages in my journal that I have those notes for on the page for this podcast. So you can check those out. It wasn't ultimately that helpful.

But, you know, it was a good place to start. So then this morning I whipped out some code, which is really fun and exciting. I'll link to the commit that I finished up this morning. So you can see it's pretty bare bones. It's a div on the left, like a little section of the screen with a button in it. And every element in the page, which is really just the div itself and the button, has an event listener attached. And every time any event happens, so it has every event listener attached, so like mouse move, mouse enter, mouse leave, focus, click, mouse down, mouse up. Anytime any of those things

happen i append it to a spreadsheet on the right side of the page so you can see any of the events that have ever happened so to recap stream sheets is a the idea is to model streams as spreadsheets so it lets you kind of be more concrete when you're visualizing the way streams make up your application. So to back up even more, where this idea came from is I want to help people visualize their whole application and how it kind of makes itself up. And then kind of peel back layer by layer and change things. So I'm like optimizing for understanding of an app and how different pieces work and being able to modify them quickly.

Those are like the things I'm optimizing for. And I went through a few different design ideas. And eventually what I came up with is a series of spreadsheets that link to each other where each spreadsheet represents a stream and what's cool about representing streams as spreadsheets is that you can kind of see each value for that stream which will help you realize whether or not the streams you're creating make any sense like You click on a button and you see that event flow through to the stream you care about.

So really, I'm basing this whole thing off of the Cycle.js architecture. So Cycle.js was created by Andre Saltz, someone I've been following for many years. I've talked about here before. I'm a big fan of his work and the way he thinks, and I'm a big fan of CycleJS in particular. I spent a bunch of time months ago reviewing CycleJS and Elm, because they happen to be pretty similar, and great in their own ways. So this tool, StreamSheets, is like a more user-friendly way

to develop in CycleJS. Because in my experience, working in CycleJS through code, through typing JavaScript code, was really hard. It took a while for me to get used to it, which Andre warns you of. It's a steep learning a while for me to get used to it, which Andre warns you of, you know, it's a steep learning curve,

but once you get started with it and get the hang of it and understand kind of what streams are, it becomes easier. And I agree with that and I could see how that would be true but it's still hard. And I wonder if you can make it better with the stream sheets like architecture. So I made this little prototype but i'm starting to feel like earlier today that i was feeling uh in the middle of today after i

like finished this first commit on on the prototype i was starting to feel like there's so many design choices to make and it's just so daunting to build all the functionality of the cycle js framework into a gui there's so many decisions to make and each decision would take like so many days to code that i don't want to commit to anything i want an easier way to iterate and prototype on this idea i almost want so for example i think at the end of the day be really neat if everything could be a spreadsheet so even the whole page is just a stream of html so so even the page itself the html is represented as a spreadsheet and then each html element each like you know it is a row and then you know you can click you can click, you can kind of, and it's like nested rows.

So you have like the body and then the body has children. You can like open the children element and they're rows and they have children rows. And I think that could all be really neat, but it would take, even just to build the spreadsheet, nested spreadsheet model thing,

would take a while. You know, maybe there's a library that does that out of the box, but just getting it all right will take a long, long time. So one idea to speed that up is that instead of putting HTML in a spreadsheet as well, I could just have like a template language, like a function that has all the other streams in scope

and returns like an HTML string, or it's like a JSX type of thing. And whenever you change html it just refreshes the page and hooks things up for you that seems like a simple way to go because i don't really care about right now i don't care about whether or not visualizing html in a nested spreadsheet is going to work that's kind of the core thing I'm trying to see is whether or not thinking about streams as spreadsheets with that visual metaphor will be a intuitive way to do stream algebra to make a user interface. So just extending that idea of like what's the minimum viable product to see if stream combinators is an intuitive way to build and understand your code.

So the minimum interface to that is CycleJS itself, which already exists, and it's pretty great in a lot of ways. I just think it could be made more intuitive. So I googled CycleJS, I was reading the documentation again, you know, re-familiarizing myself with it, and then I saw that the CycleJS dev tools exists. And I knew that exists, but I watched the video of how that works, and I was really kind of blown away. The CycleJS dev tools are amazing. So you write your app in CycleJS like you normally would,

and somehow, it seems magic to me, it picks apart all all the stream combinators that you use you know if you if you map a stream to another stream if you merge streams whatever you do and it represented as a as a data flow diagram with nodes and edges and and then as the event as the events happen in the streams it like animates through the graph it like so you can see your data flowing through which is pretty amazing that solves most of the things that i that stream sheets is trying to solve and andre makes it very clear in the talk where he introduces the chrome dev tools for cycle js this data flow diagram he

makes it clear that the point of cycle js is so that people can understand their code from like a macro perspective they don't have to piece it together puzzle piece by puzzle piece with a debugger they can just zoom out boom there's there's a structure of the code automatically generated for you and and he killed it i'm like so impressed um but one thing that this doesn't do is it doesn't let you create a user interface in this way it doesn't let you it doesn't make creating it easier it makes understanding it and debugging it easier and those are different things so i continue to do some research and i found that on andre's talk for cycleconf 2017 he has a bullet uh underneath the future of cycle cycle js uh the bullet says

cycle js dev tools as an ide i think that's exactly what stream sheets is trying to be cycle js dev tools as an ide and so that's what i started to think about like holy crap what if this was a two-way data binding to cycle js code like what if you could turn any cycle js code into stream sheets and any stream sheets in a cycle js code what if they compiled to each other that would be unbelievably cool so um because i have talked to andre. I shot him an email. I sent him an email, asked for help, and he says that he worked on CodeMentor,

like a platform where you can pay people to get help with different projects. And I think his rate is $120 an hour. But you could book at the 15-minute increment, I think. So we met for like a half an hour, maybe a year ago and had a great conversation and we emailed back and forth a few times since then he i just he's so great and to me it feels like magic that i can just send him an email and like buy his time for 120 an hour just it feels like such a bargain he's just such a genius and he's so nice and um and yeah i just feel so blessed that i could shoot him an email and he responds right away like he did today so i sent him an email about some of these questions and

said would you have time to talk and he said that he would like to talk but he has a bunch of conferences in the next few weeks so he he can't talk until not this friday but the following friday september 8th um which i understand you know he's a busy guy um so i send him two questions to see if you can answer them in the meanwhile over email if he has time and i'll pay him for his time um that he that he takes to write the write the responses over email but if not i'll just put this prototype on hold for the next two weeks and work on other things. But if he responds, that'd be amazing. The questions I asked are,

number one, what does he mean by CycleJS DevTools as an IDE? Does he mean a data flow point and click generator tool? Is it going to be similar to no flow? You know, does he have more thoughts on this that are articulated somewhere else on the web that he can point to?

Or can he like kind of write a paragraph or two or three for me to like articulate what he's thinking about there? Because I want to see how similar our thoughts are there. I would love to work with him. If Artharth line up, oh my goodness, that'd be such a dream to be able to collaborate with andre on this kind of an idea and then the second thing i asked him for was a little more technical i asked him if he could help walk me through the cycle js chrome dev tools code i spent like 20 minutes poking around seeing like how it figures out how it like inspects the

code to figure out the different nodes and edges of the graph. It's not entirely clear to me how that metadata is stored and kept track of. I have a semblance of an idea, but I wasn't feeling great about it. So if he could point me in the right direction that would save a lot of time so i'm excited to see if he can respond to that email but before i talk to him um on september 8th yeah assuming he doesn't get back to me i have other things to research they don't feel particularly pressing but they're things that i do want to spend time with so might as well do that now while i have the time jonathan edward sent me a long paper that i want to get get back to him on

there's like a bunch of alan k bread victor stuff that i i can just jump into a hole for yeah my list of links it just goes on for days at this point. So, so I have plenty of things to do if he doesn't get back to me. And if he does, I'll, you know, we'll see what he says. And we'll go from there. I have six days in this next cycle before my next research recap. I have Wednesday, Friday, Monday, Wednesday, Friday, and then Monday again. So six days um which I'm excited about I think I'll be productive the the one thing that could disrupt that is I want to spend some time refactoring the woof.js database so as I've talked about before woof.js is a programming language uh an IDE platform that my students use.

And there are a few thousand students around the world who use it. And it's pretty great. And I'm very proud of it. But I'm really excited to build a social network aspect to it so kids can share and like and comment on each other's games. I think that might give some real legs and help with the virality of it. But in order to make that happen, I have to kind of refactor the Firebase database so that things are more normalized. So so we can query for things and not pull like all the data just, you know, I have to separate the metadata from the data. So I'm going to I'm planning to

do that on Thursday of this week, but I might bleed over to Friday of this week, which would give me one less day of research. But that is an acceptable risk. So I'll get back to you on that. Looking forward into the future for this podcast, I got some two really exciting ones that I'm taping this week. Paul Chisano of Unison and Pete Hunt from React and now Smite. Those guys are really amazing. So Paul and I have become close friends.

I've actually decided to angel invest and be an advisor to his company, Unison. I think he's just so smart and such a great guy and he's got a great plan and i'm just so excited to be able to help him run at it and go change the world i'm really excited for paul and then and pete is really cool um he helped make react js a thing helped popularize it. Some of his early videos are what got me hooked on React. I think early 2014, maybe even late 2013, I was obsessed with React.js,

and I was telling everybody that it's going to be the future when Angular was the hotness. And I'm very, very proud to be ahead of my time on that one a little bit. So I'm excited to have Pete on the podcast to talk about user interfaces and the future of programming. That'll be a great bit. So I'm excited to have Pete on the podcast to talk about user interfaces and the future of programming. That'll be a great conversation. And then tomorrow, tomorrow morning, I am finally going to publish this podcast

and research project, futureofcoding.org, to the masses. Up until this point, I've kind of been talking into the void. I haven't really been sharing it with anybody. It's all been public, but nobody's really been looking at it because nobody knows that it exists. But starting tomorrow, at least some people will know that it exists, and we'll get to see if anyone cares or pays attention.

I am mostly doing this for myself, as you might be able to tell. Everything I do is very stream of consciousness. My journal is, this podcast is, everything's not very lightly produced. And mostly just for me to think my own thoughts, get them out there. So if you don't listen, that's fine too. But I do really like it when people reach out. It makes me really happy. I like making friends who are smart and hardworking and nice and care about these things. So if you read some of my stuff or listen to some of my podcasts and have ideas, please reach out.

I really feel good when people do that. So it would make me happy to hear from you. So that's it for this week. And I will, we'll talk to you guys all soon. Bye.