Future of Coding - Unveiling Dark: Ellen Chisa & Paul Biggar

Episode Date: September 25, 2019

Last Monday, Ellen Chisa and Paul Biggar unveiled Dark, a new web-based programming environment for creating backend web services. In these conversations, first with Ellen and then with Paul, we discuss how they met, conceived of the idea, iterated on the product, and what their long-term vision is for the product. Dark is a web-based, structured editor with a data store built-in. It's code has a functional programming feel to it, but it also embraces what they call "functional/imperative". For example, their "error rail" allows programmers to defer handling nil-cases, much like a dynamically-typed language, but still keeps track of their existence in a monadic structure, like a statically-typed language, but without users having to learn anything about monads! Paul often brings the discussion of Dark back to Fred Brook's distinction in _No Silver Bullet_ between essential and accidental complexity. I had fun in this interview diving into the Aristotelian roots of that distinction. We also debated the meaning of the terms "no-code" and "low-code", and whether either could be applied to Dark. Dark removes accidental complexity around infrastructure and deployment. There is no separate step to deploy code in Dark. It's "deployless". Every single change to a Dark codebase is instantly (in 50ms, the time it takes to get your incremental change to the server) deployed to production servers. Of course this doesn't mean that every change you make is instantly deployed to _users_, but simply put on production servers behind a feature flag _ready_ to be rolled out at your discretion. Deployment, getting code running locally to run in production, is eliminated because all code is running on Dark's platform at all times. What remains is simply choosing when to release that code to users. One of my favorite parts of Dark is how readable its editor makes functional programming, which I typically find intimidating and difficult to parse. The Dark editor saves all past HTTP requests to all routes, and then uses those values to provide "live data" for every intermediate expression in that route. A dense section of code becomes totally comprehensible by clicking through each expression and seeing actual past values that have inhabited that expression. It combines of the best parts of a debugger and sprinkled console.log statements, but without the downsides of either. I'm glad that we had the opportunity in this conversation to dwell on some of the trade-offs of using Dark. Paul and Ellen are well aware of the risks customers face by moving their applications onto the Dark platform, and hope to alleviate those risks as much as possible. For example, they are looking into creating a legal structure that will make Dark open-source in the event that Dark shuts down. Paul Biggar is best-known in the Valley for co-founding CircleCI, a tool for continuous integration and deployment. At heart, he's a compilers nerd: he got a PhD in compilers, worked on the JavaScript compiler at Mozilla, built CircleCI which is a compiler for deployment, and is now building Dark, a programming language, environment, and infrastructure compiler. Ellen Chisa is passionate about helping people make things. She worked at Microsoft on Office Mobile, at Kickstarter, and started a company that built tools for travel agents, Lola. The full transcript for this episode was sponsored by repl.it and can be found at: https://futureofcoding.org/episodes/043#full-transcriptSupport us on Patreon: https://www.patreon.com/futureofcodingSee omnystudio.com/listener for privacy information.

Welcome to the Future of Coding. This is Steve Krause. So there are a few exciting episodes to look forward to in the docket. I'm Jad Masad of Replit, the sponsor of this podcast. That episode has been recorded for many months now, but it's finally going to be released soon. I also, a couple of months ago, recorded an episode with Jennifer Jacobs, who used to be at the MIT Media Lab, now is an assistant professor at UCSB. We also have Jonathan Aldrich coming on the podcast. You may have heard of his name because he is the advisor or the advisor of the advisor of many of my other guests who have already been on the podcast.

So it is about time to have his voice represented here as well. So look forward to all those episodes. And you also have some solo podcasts of me talking about my own research to look forward to. I have been very delinquent over the past year about publishing those episodes, but I have plans over the next couple of weeks to release one, and then maybe a few months later to release another one. So hopefully that happens and you can hear more about what I've been up to. You can always find what I've been up to at futureofcoding.org slash log. But if you prefer to get it in audio form, you'll have to wait a bit. So today's episode is really exciting. It's been planned for a long time. And it's particularly topical right now because they

just launched this past week. So I'm interviewing Paul Bigger and Alan Chiesa of Dark today. So beyond their launch, which you probably saw because it was at the top of Hacker News and all over Twitter, you may have heard of them because I've been working with them over the past two years on a really long and exciting research project doing a survey of the space of future of coding tools, languages, and interfaces. And we, in the past couple of months, packaged up that research to be published. And we launched that two weeks ago

as the whole code catalog, which you also may have seen on Hacker News and Twitter. So I was really excited to get that out the door because that took a lot more time than you would have expected to put together all those videos and reviews of these tools I had to actually figure out how to get them to run and then spend time building things in them so I could report back on how they felt and what it was like to use them so I encourage you to check out this whole code catalog which I talk a bit about with Ellen in this episode. So a bit about how this episode is

structured. It's a bit different, I guess maybe a little awkward than the way episodes are normally structured. We were thinking about publishing this episode in a different way. But the way that we decided to go forward is there are two interviews, one with Ellen and then one with Paul in series following this. So this episode is kind of like two interviews smushed together. The one with Ellen is quite a bit shorter than the one with Paul. With Ellen, we talk about the whole catalog, we talk about her background and how she met Paul originally. And also we talk about her experience hiring their first customer at Dark, which is a non-intuitive but a really compelling idea when she explains it. And then with Paul, we talk more about the programming languages side of

things because he's a compiler expert. He studied compilers in college. He worked on the JavaScript compiler at Mozilla and then started CircleCI and now he's doing Dark. So just to whet your appetite for the conversation, we talk about Dark, which is this tool, this web-based tool for building web HTTP backends. It's known for being a functional programming structured editor that features live data that you can play with as you're editing your code. So Paul talks a lot about accidental versus essential complexity. And that's something that I've been thinking a lot about in the past couple of months, particularly because essential versus accidental is an Aristotelian idea. And so Paul and I get into that semantic debate a bit. We also get into the debate

about what the phrase no code means and if it would apply to something like dark. And then we get into some of the bigger themes with dark, including the fact that it's a holistic editor, they've combined many of the features, and many different tools and languages into a single holistic tool that does a lot less than I guess all those tools individually but because it's been combined into a single tool it doesn't need to do as much there's a whole lot of affordances that can be taken advantage of by this holistic nature but of course there are trade-offs and so we get into the positives and the downsides and the main downside that many people have been pointing out is the fear of lock-in,

because we've all seen what happens when you move all of your infrastructure onto a startup's platform, and then that startup goes away. So it's a scary thing, but they have some plans for how to deal with it. And yeah, we talk about a whole bunch of things. This is a pretty long podcast,

so I won't recap the whole thing here. You'll just have to listen to get to it. But it's a really fun conversation. And it's exciting to get a little bit more perspective about what they're trying to do, in addition to all the information they released on Monday by their launch. So now just a quick message from our sponsor. Repl.it is an online REPL for over 30 languages. It started out as a code playground, but now scales up to a full development environment where you can do everything from deploying web servers to training ML models, all driven by the REPL. They are a small startup in San Francisco, but they reach millions of programmers, students,

and teachers. They're looking for hackers interested in the future of coding and making software tools more enjoyable and accessible. Email jobs at repl.it if you're interested in learning more. Okay, now I'm going to introduce Ellen, and we will hear from her for about 30 minutes, and then I will introduce Paul, and the rest of the episode will be from him. And without any further ado, Ellen Chisa. Welcome, Ellen.

Thanks for having me. Yeah, I'm really excited to have this conversation on the podcast. Yeah, we've been talking for so long about these tools, but it's exciting to do it for the podcast. So before we start talking about Dark, I wanted to get a bit about your background. Before Dark, and then how you met Paul and got involved in this strange world of improving programming. Yeah, it actually makes a lot of sense. So for undergrad, I went to Franklin W. Olin College of Engineering. And the focus Olin had was always on dramatically improving engineering education. So this was a while back. So at the time, that was things like

having project based education, not necessarily just doing a problem set, and like doing a lot of the things we think of as being like really normal in the workplace. But for education, it was a different approach. And so I spent a bunch of time there and really enjoyed it. But one of the things that frustrated me was that I did find that a lot of the time I spent programming felt not as rewarding as I wished it would, either because there'd be some syntax error, two tools wouldn't work nicely together. And that was what led me to go into product management jobs instead of software development jobs was I just felt like I was making better use of my time.

And so from there, I did I did a bunch of different consumer companies. I worked at Microsoft for a while. And then it was a Kickstarter, which I loved. They really like helping other people to make things. I went and did the first half of an MBA at Harvard Business School, and then I was working at a travel company. And I realized that every time I pitched the travel company, I kept being like, and then people will travel more. So then they will make more things. And I realized I had to get closer to helping people make things again. That makes sense. What product did you work on at Microsoft? I worked on Office Mobile. and so i was there in 2010

which was in the era where the world smartphone market share went from being dominated by sybian to being completely iphone and android that's a really interesting time to be at a big tech company it was it was super interesting and the organization was set up such that we were we worked a lot between the different groups so we spent a lot of time with Office and with Windows Phone and with Windows Live, SkyDrive, which I guess is called OneDrive now, and with SharePoint. So we got to see a lot of the different pieces of Microsoft. So how did you get connected to Paul? It was funny. So originally, Thompson, who started FrameData, sent me an email and asked if I wanted to be introduced to Paul to work in this space.

And it just wasn't a particularly good time for me personally. I really, I liked my travel job aside from continually pitching it as being about helping people make things. And my husband and I just bought a house in Boston and we were, it was right before we got married. And so it didn't seem like the best timing. And then two weeks after that, Paul cold emailed me, I happened to be on a trip to San Francisco anyway, and I agreed to have coffee with him. Wow. That's quite a disruptive change. So is that why your MBA is only partially finished because you left to start Dark? I actually left when we were starting Lola. I'd been in EIR in Blade, which was a consumer tech incubator in Boston at the time, which then became Lola.

And I'd originally intended to go back at some point after Lola. And then I met Paul and was like, OK, I would much rather work on solving this problem. This is everything I've cared about solving for a long time. Got it. Yeah. OK. So it sounds like when you first met him, the message really resonated based on your experiences programming in college. You definitely saw the problem. Yeah, and not just programming in college. Also, all of the experiences I'd had in the workspace with Microsoft, even just getting the emulator set up for the tooling we were using to build for Symbian was an hour-long endeavor.

And then inevitably, you would have days worth of meetings. And then by the time you got back to your desk desk something else would be broken and you'd be spending another hour trying to get a simulator to work again and then like a kickstarter a bunch of the time at the time i was there they were using a different payment solution and it was really hard to test anything with the payments flow in your staging environment and so every time you deployed something related to that you were a little bit nervous um and there's this whole thing around we had like a campfire room this was before Slack to coordinate when we were going to deploy. There's a bunch of really interesting work going on there. But there were all these

things that were accidental complexity about software that were getting in the way and like similar problems at Lola, for sure. And so I, like right in the front of Dark's website is this claim that you're trying to make programming 100 times easier, which is a pretty big number. And I'm curious if in that first meeting, Paul said that's what he wanted to do. And if you believed him or you took a bunch of convincing or immediately you were like, yeah, yeah, it feels like this could be made 100 times easier. I was pretty much immediately on board. And Paul and I were actually talking about this last weekend because most people at the time weren't. And a lot of the things we've been working on with Dark for the last couple of years now seem a little bit more tractable.

But at the time, it seemed off the wall. It didn't seem like something you would be able to do. So before we keep talking around it, I wanted to paint a picture of what Dark looks like today. So people have this mental picture in their head of what it is that we're talking about. And I thought maybe you could take that one, like just describe the editor experience kind of roughly, like what it looks like and how you interact with it. Absolutely. So Dark, like we've said on our website and other places, is a way of trying to remove accidental complexity from software. And so sort of the underlying thing behind that has been with a lot of our developer tooling over time, we've been incrementally making things we've always used

better. So editors get a little bit better, CI gets a little bit better, version control gets a little bit better. And while they're getting better, they don't necessarily become easier to use or easier to connect to other parts of the tool chain. And so the original motivation with Dark was how can you pull out all of the things you don't want to be doing to be focusing on your code? And when we started thinking about that, we thought about, well, we don't necessarily want to think about setting up our infrastructure. Like the wow moment is you get something working locally and then you're like, oh wait, now I have to like go get this somewhere else. So like, how can that not be a problem? And then same thing, every time you make a change and then you wait and then you deploy again and then you see and you go get this somewhere else. So like, how can that not be a problem? And then same thing, every time you make a change, and then you wait, and then you deploy again, and then you see and

you go in this loop, but it's pretty slow, because you're waiting so long for your deployment. How can we make that not a thing? And then similarly, my old problem from when I was an undergrad, you make a syntax error, and then four hours later, you find your one syntax error, and you're very upset about having wasted four hours. So we started looking at how to make those not a thing. What we came to is you could make a holistic solution that had the language, the editor, and the infrastructure together. And by doing that, you would be able to have each of them informing each of the others and do a bunch of work at the boundaries between them that you wouldn't have been able to without integrating them into a holistic platform. But when you think about how Dark actually works, a lot of the pieces, the language itself, for instance,

feels a lot like a language, people always think it's going to be way out there completely different. But programming in dark, you use a lot of the same things, like there's still ints, there's still strings, you're still using list map as a function, like a lot of the core concepts for writing code itself feel the same. Our editor is structured. And so the easiest way to kind of think about what it is to have a structured editor is to think about tools like IntelliJ or like what Kite was doing, where you have this much more robust set of autocomplete tools. And so we definitely did not invent having autocomplete in an editor. We take a lot of inspiration from other tools that have solved these problems well. But so in Dark, when you're

adding something, you don't just have the autocomplete as a suggestion. You actually can't add something that's syntactically invalid. You can't type list MA instead of list map and then get an error for that because as soon as you type list MA, your only option is going to be using the list map functionality. One other third thing that's interesting about Dark,

which is probably the more different part, is that we are really focused on giving people the ability to look at the overall architecture of their code base and i'm sure when you talk to ball he'll get into this more but the idea being that when you start a new job someone takes you to the whiteboard and draws out how your system works and they don't just give you a list of files that you're editing they tell you about how your system is actually operating and so the way dark is organized we're moving towards yes you're editing, they tell you about how your system is actually operating. And so the way dark is organized, we're moving towards Yes, you're looking at what looks like code when you're writing an individual function, or when you're when you're writing some logic, but you still want to be able to see that high level view of how the different things within your code base

connect to each other. So how does that look? Yeah, in the editor? How do you connect things? We're only taking our first steps towards this. So one thing we have, it's actually pretty similar. You saw the work that GitHub came out recently with references to see how things call each other. But so when you're looking at, say, an API endpoint in dark, if it's writing to a database, you'll see what data store it is writing to. If it is emitting to a worker, you'll see what background worker that it's emitting to, and you'll be able to easily get more information about which different pieces of your code base you're connected to when you're editing that specific handler.

And another thing just to add on, I don't know, you may have said this already, but an important thing to note about Dark is that it's for creating backend web services. Is that my limiting scope of what Dark is used for? So right now, Dark is very focused on building backend web services. Is that my, or am I limiting the scope of what Dark is used for? So right now, Dark is very focused on building back web services. We think a lot of the concepts in Dark will apply to other types of programming as well.

But when you're already at the point of making a programming language as an editor and infrastructure, you have to narrow scope somewhere. And so we started with building back web services. I may forget a few,

but the sorts of things you can create at the top level in Dark, whereas you can create a HTTP handler, you can create a client job, you can create a data store. You can extract any code that you want to reuse into functions that you can rerun. So not like serverless functions, like more classic functions, as you would think about them when coding.

And then there's a couple of other interesting things. You're able to host static assets. So if you have compiled JavaScript that you want to host on the front end, or if you have images you're serving to your end users, you've got how to handle this. So I thought you got a lot of attention last year, I think it was, for hiring your first customer. So I'd be curious where you came up with that idea

and exactly what the deal was for the person who got that and how it went, what you got out of it. And yeah, if you'd recommend it, that sort of thing. Yeah. So like we were talking about earlier, you're making a pretty big bet when you're deciding to build on this new infrastructure that you also have to learn a new language for. And so at the very beginning, we also hadn't built that many things on top of it ourselves. And we knew that it was really painful and that we were going to spend basically all of our time having to support the first person. And so we were thinking about how can we incentivize someone to take this big risk with us? And how can we like find out if

that's something people are interested in and making sure we're building for real use cases rather than just our front side projects or our side projects. And so we decided that the classic way to incentivize people is that in addition to giving them cool new technology that would hopefully be better than what they were using, we would also pay them. And so we had Daniel from Dabblefox in for three months, and we paid him a stipend of $3,000 a month. As the idea was, this was someone who is working on their business. They weren't someone who needed a full time job, like $3,000 a month as the idea was this was someone who is working on their business they weren't someone who needed a full-time job like three thousand dollars a month is not going to support you for very long in the bay area um but it would give him the incentive to come in show up every day and help us like push through all the early problems that came up as he was building on

dark and was it worth three thousand dollars a month Did you learn a lot? Oh, yeah, it was great. And when you think about that, $3,000 a month actually compared to what some companies will do if they, say, hire someone or have one of their founders spend all of their time trying to land one enterprise proof of contract deal, it's a lot of money, but it's not as much money as it might sound like. So I think it was a really good investment compared to lots of things we've done over time. And have you repeated it or you just needed it that one time? So we definitely needed it the most the first time.

And definitely, it's funny, I was thinking about this recently. When Daniel first started, it was pretty much one person was on Daniel duty and one person was building all of the big projects that came up that Daniel needed. So it was sort of, we like either were immediately triaging or we were working on longer term investments. And so from that regard, a lot of those practices continue today. We still have all of our alpha users in a Slack channel and we try

to be as responsive to people as possible and get them bug fixes as soon as is necessary. And then we've actually continued with having people in longer term as well. So we no longer pay people, but now we treat it as more of a developer in residence. And so the idea being, we're still getting all of this real-time feedback. We get to have someone else from the community in and working with us and spending time with us. And they get a desk that they're able to sit at. We have some extra space in our office right now. So right now we have Costas in, who's a developer, who's working on building an application he calls Song Battles. So people can put up two songs while they're at a party

and the attendees of the party can vote on which song they want to hear next to the DJ. Cool. So you're in private alpha now. So you have how many people using the product? And is it for businesses, their businesses? Or is it more just for side products, you're not supposed to use it in the private alpha for a business? So what we focus on with the private alpha is doing a mix of both.

So we kind of think of them as having slightly different use cases. The advantage of having people building real businesses on top of dark is we start to get that feedback of what people need as they grow as they have more customers, as they like are trying to solve specific business problems. And so from that regard, we've had people build complete backends for their business on top of dark. So a great example of this is Chase Oliveri at Altitude, usealtitude.com. And so he builds an entire flight deal alerting system, which has by far the best flight deals I have ever seen. And so in dark like that has his integrations to stripe so people can pay him it has his integrations to mailgun so he's able to send out emails about the deals as they happen and things like that and that

full back end runs in dark that's been important to us and then it's very cool yeah and then we also recently had pixel keat um which is a design firm that does like a lot of landing page design. We're actually working with them on something else. And they had an internal project tracker that they were using with their clients. And what they did was they built that back end on dark so they'd be able to scale and open it up to other design firms to be able to use as well. So how many people are in the private alpha at this point? Yeah, I would say at this point we have about 100 developers who've let into the private alpha, the number can be a little bit misleading. So that's a mix. So those obviously, there's examples

of people who've built entire businesses. There are people who've done really interesting side projects, there was Tokameki unfollow, which was originally built on glitch. It's a really nice website that allows you to individually see everyone you follow's tweets and then decide if you want to keep following them or not based on if their tweets spark joy, kind of going off of the KonMari trend. And so all of the data store for that runs on Darknow. It got a little big for Glitch. And so Julia's moved over the data store.

And so we have that. And then we also just have friends that we let on super early. Like I remember Peter Van Hardenberg, who was early at Heroku. and so we have that and then we also just have friends that we let on super early like i remember um peter van hardenberg who was early at heroku at the beginning i had him come over and was like you have to try this new language it can finally write fizzbuzz it's like i'm hovering behind him forcing him to write fizzbuzz in this like absolutely terrible tooling and so he still has an account from that and sometimes plays with it but like so it's a mix of companies people with side projects that were really interesting and needed dark for a specific reason and they're just friends who helped us out

along the way like how you have one for instance yep um i in talking to people about dark which happens pretty often given that we work together um people are very eager to see it and play with it themselves so um maybe you can give people some sort of a timeline on when they'll be able to see it and play with it. Yeah, so we're excited. People are going to be able to see it starting on September 16th, with two exceptions. If you are coming to Strange Loop, we are having a party that Friday evening, the 13th, and we'll be showing it there as well as all of our work on the whole code catalog with you. And then we're also going to be showing it very briefly at full

stack fest in Barcelona, the first week in September. But after that, it'll be you'll be able to see the video on our website, be able to see everything on the 16th of September. Cool. And can people play with it themselves yet? Or we're very optimized around someone coming in and knowing what they need to build and then building it and so i know a lot of developers like you play the project by like kicking the tires being like oh i'm just gonna like make some random stuff dark definitely works better when you sit down you're like this is the api i need or this is the data store i need or this is like the api in the data store i need but when you come in and know what you need to build, the experience is much better. So like dark right now isn't very good for like explorative programming?

I would say, yeah, it's not as good for explorative programming yet. I think it's also by nature of the onboarding. While we've been in private alpha, I've been onboarding people by pairing with them, which is really enjoyable for me. I learn a ton about the product. People have like that face time

and can get to know us as a company a little bit in addition to just seeing the product. But we haven't really put that into something that is like an easy to learn onboarding tutorial, which you experienced once recently. And so I think that will help a lot with that too. I'm a little bit confused why you think dark wouldn't be good for explorative coding

because it's like all about, it has like a very tight feedback loop. Like you make a change, you see the result. It seems like that's one of the most necessary ingredients for exploratory coding. Yeah. I think actually that's a really good point. I think the reason for that tension is we kind of, we've been calling it the dark flow recently,

but dark works best if you either already have a front end you're passing data from, or if you're connecting to an external API that's going to be giving you a response. We very much go from what is the end user going to see to what is the interface the end user is looking at. And then you talk to Dark for building out your backend to support that, which is a little bit different than how people classically think about a backend where they're like, okay, I'm going to get all my models set up.

But in Dark, getting all of your data store set up, you're going to be like, okay, I built a bunch of data stores in five minutes and like cool like it's not it's not an exciting experience just because it's so easy which is great when you're trying to get it shipped but like as a fun thing to do it's not like super fun to type out a schema oh that's interesting um well i feel like then like the way to phrase it is that, yeah, back-end people, I guess won't have that much fun. I guess, or phrasing it the other way, front-end people would, I guess, have a lot of fun building like exploratory things with it. Yeah. And I think back-end people would too,

but they just have to flip like, and not all back-end people are this way, but if you start from the perspective that I'm going to start with my front-end, get my front-end going, and then use that to play with the dark, then you would have a really good exploratory time. That's fun. I guess that kind of makes, I feel like that's what your first tutorial will probably look like, right? You'll like supply people with the front end code and then like work with them through the tutorial to how to write the backend code. Yeah, exactly. That's how we do onboarding now too. So I wanted to, I guess, explain to the audience how you and I started working together.

I think it started when you reached out almost two years ago. I think one of the reasons we reached out to you though, was like, you seemed like the obvious person to talk to for the thing we were curious about, where we wanted to see like lots of interesting best-in-class things from lots of developer tools that were happening and specifically around how they felt and what felt good and what felt bad to serve as inspiration for what we were doing yeah so i think there are like two interesting uh things about that idea you had to survey the space. I guess the first one is, is the idea of survey, surveying the space at all. I, maybe I'm wrong, but I don't think many companies do that. I guess particularly from the developer tools perspective, I don't, I don't know any cause cause that's the space I know best.

I don't know any other tools that have done like a survey thoroughly the way, the way we have. So yeah, where'd you come up with that idea? Yeah. Yeah. Yeah. I think this is actually one of the ways I learn is I like to have lots of inputs to kind of build out a system and a mental model for how things work. And so I think that's partially like a personal learning choice. And so for me, that's like, okay, let's see all of the things that we've tried before, because we don't want to just go either make the same mistakes, or we don't want to do exactly the same thing. We just want to be well informed about what has happened and why. And that was important to me, especially having not been working in developer

tools recently. I think another piece of it is a lot of times, doing research or doing surveys like that gets a bad rap, because people think of it as, oh, you're just like making a competitive landscape diagram. Like that's such an MBA thing. And I think that's not the interesting part about looking at other tools at all. I think it's much more you want to look at them for that qualitative sense of what are the trends for where we're going? What are things that people are really enjoying? Why do they really enjoy these things? And much more from like this human understanding part of how we're interacting with our tools. I think in developer tools, it was also in particular because we see innovation across so many spaces. So you have academics, like Cyrus working on Hazel, who are thinking about it from that perspective. You have people

who are building very pragmatic things, or people who are building like a tool that was originally just intended for themselves, then it turns out that everyone uses it. You have people who are building like a tool that was originally just intended for themselves then it turns out that everyone uses it you have people who are trying to build companies around developer tools and you obviously like have the big cloud players doing a bunch of things as well and so it's it's much more disparate than you see in a lot of other spaces yeah that's a really good point i find it's kind of a beautiful a beautiful thing it's like hobbyists and academics and startups and big companies. Yeah, it's cool. I was also really interested in was like, one of the things that's different about developer tools is you're building tools for people who are experts at something already. And so I knew I'd already taken a bunch

of inspiration from what we did at Lola, because we were building tools for travel agents who are experts at booking travel. And so I wanted to also be looking at expert tools across the board, not just necessarily for developers, where when be looking at expert tools across the board, not just necessarily for developers, where when you think about like tools like origami, which is really focused on designers rather than developers, or things like Photoshop or Sketch or Figma or all of kind of those spaces as well, those can also inform what you do in a developer space. So I guess maybe if you had to pick one or two of your favorite tools that we reviewed that you'd recommend for people to look at when the whole code catalog is public, what would you recommend? This is so hard. I like them all. I would definitely recommend looking at Coda. I think it's really interesting to think about the contrasting approaches between trying to add more programming technologies into office-related tools compared to starting with a programming tool and then figuring out how to make it more accessible over time.

I think I'm also biased there just because I worked on Microsoft Office before. That space has always been a really interesting space for me. So I'd say Coda is definitely one in that regard. I guess since I've also already mentioned it, Hazel is a good one if you want to see more of an academic approach. And then I think I would also say Eve, just because that's sort of the classic one. I really admired how much they were willing to try and experiment, share everything they learned so publicly. Like pretty much the opposite approach of what we did at the very beginning was they were working all the time in public and showing everyone everything they learned as they went, as opposed to us kind of spending a bunch of time learning.

And now we're trying to share as much as we can. And so I think theirs is really interesting as well. They're also interesting for different reasons. Like I think if you want to get into like the history and like one of the things I originally loved, like we did HyperCard. And I loved HyperCard so much in elementary school. And I think that was like one of those things

where I think actually the reason I wanted to work on Microsoft Office was for me, PowerPoint was the next best thing to HyperCard when HyperCard went away. I'm not really sure why I went the PowerPoint direction instead of like a programming language direction, but I spent a lot of time in like the PowerPoint object model, like scripting how to move around between different slides with buttons, basically emulating HyperCard. So I think from the historical standpoint,

HyperCard is really interesting. I think from the community standpoint, glitch is really interesting. I think from like the structured editor perspective, Hazel is really interesting. I think from the like, how do you take something that feels a little bit more like Scratch and get it in a browser where people can try it, experience, Revolet is really interesting. I think that was your comment on it as well as like, it's like Scratch for people who are slightly older and don't want to be using like this tool that was designed for children uh which i think i mean i'm totally fine like i like lego robots i like many programming tools that were designed for

children i think they're all really cool but um i think rebel it does like up a level that a little bit um i guess again like if you're more on the academic side small talk is interesting they're all interesting for different reasons and so I guess I would say when you're looking at the whole code catalog and figuring out what to dig into, I would go for something that you personally like a space that you're interested in. I think that's like when I get the most out of it is when I start from like other concepts I'm already thinking about. So maybe talk about why you decided to make all this research public and put together the catalog. I was very excited when I heard that was what was going to happen. Actually, I think part of it was we kept doing it and I was excited.

And I would talk about our work with people who visited our office all the time or other people who are interested in the space and then it became like oh i'm like occasionally like copy pasting individual things and sending them to people or talking to people about individual tools that we've worked on um but we we assembled the research more as as background for ourselves and because we wanted to know state of the art but we want everyone else to have that as well and be able to work from that and kind of see where we're getting to as an entire industry of people who are trying to make the experience of coding better and so obviously our approach at dark for making coding better is having a full language and then figuring out how to make that not have as much accidental complexity when you're using it really getting down into that essence of what it is to code and other people are looking at it from how can we kind of sneak code in through

our office tools that we're already using um But yeah, I think all of us are trying to give more people the ability to, to compute things in the way that they want to and be able to build their own tooling. And I think having access to that is good. Before we sign off, are there places on the internet that we could link to you or anything. And if there's anything that you wanted people to reach out for, like if you maybe at this residency you mentioned, or if you're hiring or other things like that.

Yeah, absolutely. So we're, our dark website is darkling.com and we are at darkling on Twitter. And then I personally am Ellen Chisa on Twitter. I'm in Steve's Slack group. You're welcome to talk to me there. I feel like the conversation moves very quickly, but you can DM me and I will respond. And then I'm just Ellen at darklang.com. We are definitely hiring. Obviously, lots of the stuff, like Steve said, is an alpha. So there are many bugs to fix. There are many

features that are basically stubs that we want to keep working on. There are many exciting problems still to solve. So that's great. In particular, right now, we're looking for more engineers, a developer evangelist, possibly a VP of engineering. If anyone happens to be listening, he's excited about that. So yeah, definitely reach out to me for any of those. Great. Well, thanks for talking.

This was a lot of fun. Yeah. Thank you for doing all this work with us. Welcome, Paul. Thanks, Steve. This was a lot of fun. Yeah. Thank you for doing all this work with us. Welcome, Paul. Thanks, Steve. Happy to be here. Yeah. I'm really excited to finally have you on the podcast. Yeah. I've been meaning to do this podcast for ages.

I feel there's an email in my inbox about this for like a year now. Yeah. Well, you guys have been keeping things under wraps while you're working out some of the details. So I'm excited now to share some things about Dark. DAN GALPIN- Yeah, I'm excited to be sharing them as well. CHRIS CARTLAND- So right before we get into the details of Dark, I wanted to just get a bit about your background, because you've been in developer tools for a

while, but I want to start maybe before. Just tell us a bit about you background, because you've been in developer tools for a while, but I want to start maybe before, just tell us a bit about you and then what brought you to Circle and then Dark. Yeah, so my background, I did an undergrad in CS and I discovered that I really loved compilers. And so then I did a PhD in compilers and then went to work for Mozilla on their JavaScript

compiler team. And after I was there about a year, I got tired of a particular tool that we were using at Mozilla, which was essentially their CI tool. And so I started CircleCI in about 2011 to do CI. And I did that. I stayed there for about four years. And then I took a break and started thinking about what am I doing next? And eventually decided to work on fixing everything, which became Dark. Cool. Yeah, I didn't realize that the idea

from Circle came from the experiences at Mozilla, that's interesting. Yeah well the I had done a Y Combinator startup before between Mozilla and the PhD and one of the things that that was said to me at the time was like why don't you do compilers as a service? And I kind of had no idea what that was. And then after spending about a year, the fact that I was frustrated with this tool and I was like, oh, this compilers as a service thing kind of makes sense. Yeah, and I guess that's related to Dark. Dark is also a bit of a compiler as a service. Yeah. I mean, essentially, my career is getting people to give me millions of dollars to build

fun compiler tools. Hmm. Yeah, very interesting. Very sneaky. Mm-hmm. Yeah, they don't know I enjoy it. They think it's about the money. Mm-hmm.

You're just an academic at heart, just wanting to build compilers. Right, exactly. Tenure is a bad place to do it, but industry is a good place to do it. I think a lot of the people who listen to this podcast, a lot of programmers know what CircleCI is.

And based on a few of the essays you wrote while you were at CircleCI, like lambasting all the complexity in normal deployment. I feel like those were kind of where the seeds of Dark came from. Is that accurate? Yes, a lot of it. It's funny.

At the time, so the blog post that I think you're referring to is It's the Future. And at the time I was I was actually a believer like I was like I see why this why this docker thing is useful I understand orchestration like I I honestly you know was was in the camp that this was the future and I wrote a follow-up to it um and it wasn until a couple of years later that I realized oh shit you know that this is way too much there's way too much complexity and it combined with with a couple of other ideas that I that I'd had about how hard it was to build internal tools and flaws and sort of existing programming paradigms and how we're trying to build cloud things with

languages that are that is designed for single machines and sort of that that impedance mismatch and lack of abstraction so all of those combined into one and became dark cool Cool, yeah. So I didn't realize that the essay, it wasn't actually sarcastic originally? So no, it was sarcastic, but in a tongue-in-cheek, we're all friends here sort of tone. But I wrote a follow-up that's called It Really Is the Future,

where I talked about why containerization is important, and orchestration is important, and how I actually believe that this set of tools is the future, and that the problems that they solve are real. And the problems that they solve are real. It just turns out that the solution is too complex. OK, so I think that's a perfect transition to my next question. I want to talk about complexity, because it's something that a lot of people, a lot of programmers

talk about in general, but particularly us folks trying to make programming better, we talk a lot about this distinction between essential complexity and accidental complexity that comes from Fred Brooks' essay, which he in turn borrows all the way back from Aristotle, this distinction between essence and incident or accident. Interesting, I didn't know that connection. Yeah, yeah, yeah. And I didn't realize that connection either, but I was talking to a friend who's a philosopher

about this sort of work, and he was like, do you even know what essence is? Like Aristotelian essence? And I was like, that's not what he's talking about. But then I went back to the Fred Brooks essay. And there's the word Aristotle right there. He says, like, he specifically, yeah, Fred Brooks is a, I think part of the reason we still reference him is because he's so well read. And I can draw on all those big ideas from uh anyways philosophy so anyways um i i've become become to like wanting want to drill a little bit deeper into like

what is the difference between essential and accidental complexity because as you were just saying like uh the problems that docker and orchestration solved are real problems. So it's kind of hard to say that they're not part of essential complexity. But then I guess if you use a tool like dark, clearly they're not part of the essence of dark. So, yeah. So how do you- Yeah, I think one of the things that makes dark what it is,

is that we draw an extremely wide uh bound around around accidental complexity or maybe an extremely narrow one around essential complexity um so if you have someone that tells you that like you know architecture or infrastructure um is is complexity uh or sorry is essential complexity i'm extremely um i'm extremely skeptical And the thing we keep coming back to is that the thing that you want to do with a backend, so dark is for backends, you want to receive data, you want to send data,

you want to process data, and you want to store or query data. And there isn't a orchestrating the services that do that in that description of what you're trying to do. If you ask someone to describe their app, you know, it's going to be an app that talks to an iPhone and saves some data and, you know, sends out some notifications or something like that. And there's going to be no mention of Kafka or Postgres or something like that. And there's going to be no mention of Kafka or Postgres or something along that. Well, so I guess to push, to use your own definition against you, if someone was designing an iPhone app,

I don't think there'd be any mention of HTTP handlers or databases either. They'd talk more abstractly about saving data somewhere. Yeah, that's absolutely right and i think the um uh the uh there's a difference for dark between uh between what we're shipping today and and what we think of as of accidental complexity and i i think the the question sort of uh uh points out that you've seen Dark, whereas most of your listeners won't. But in Dark, we certainly have certain HTTP concepts and that kind of thing,

which as Dark expands over the next decade or so, may go away if we can. Oh, that's very exciting. Cool. So there's what we're launching and what Dark will be in the short term and then there's the vision of getting rid of accidental complexity. And I think getting rid of all accidental complexity, and we talk about making Dark,

or Dark making coding 100x better, it's very much a computer on every desktop kind of vision. It's not a claim that we have achieved this, but rather that we are going to do that. And our current work is in the service of it. Okay. Yeah, that's a very important distinction to make. Because I think I even saw people in Hacker News comments, you know comments saying things like, there's no way they've done this. So it's important to be upfront that you haven't.

And that is accurate. I would say we're currently in, depending on someone's level of expertise, we're maybe 2, 3, 4x better, which honestly is kind of phenomenal. People make companies that make things 10% better and they're successful. But then they also, or we are currently also bringing

our own complexity, which tampers that down a little bit. Oh, interesting. Yeah, yeah, yeah. You remove a lot of complexity from other tools, but there's dark, specific, new complexity that you're adding to the space. Right, right. And most of that is things that you can't achieve yet, things that need to be simplified more, you know, hacks and shortcuts to get to launch, etc.

So I think now's a good time before we get too much into the weeds to describe Dark, I guess conceptually, the things you can do in it now, what it's built for, and then visually just describe kind of what the screen looks like and how you can interact with it. So taking it from accidental complexity, our purpose with Dark was to make backends really easily. And the way that we determined that we can remove all this complexity is by having a holistic editor, programming language and infrastructure. So Dark fundamentally is an editor which you access via your browser in which you write the Dark programming language and which is instantly deployed to Dark infrastructure. So how that looks when you go to the editor, you'll go to

darklang.com slash your app and you'll be presented with an editor, which is just a black screen and a sidebar. Our current metaphor, which is not a long-term one, but is the one today, is a canvas. And so there's little bits of code on the screen, and you add new bits of code. And the bits of code are specifically handlers and databases. And the handlers are, we're describing it visually, so handlers, so let's say there's a bunch of HTTP handlers and they have a specification

for what they are, the URL and the method at the top and then code in the body. And it's visually, everything's very dark, obviously. Some shade of black for the background, some other shade of black for the code. Will there be a light mode?

We're joking about maybe we should make light mode for everything, and then whenever anyone asks about why is it called dark, and when it's actually light, we're just like feign ignorance. But yeah, there will of course be a light mode. So what you're writing in these handlers, like the code that you're writing looks a lot like code. There's the, like it looks a lot like modern code. You're typing let x equals 5, let y equals 10, x plus y.

Does it look most similar to like OCaml, Haskell? I don't know. So yeah, it looks a bit like both. A little bit of tidied up the OCaml syntax. It's definitely based on functional languages, but we're currently leaning in a functional imperative direction, which means that we're stepping a little bit back

from pure functional. And I mean, we're really trying to simplify how people code and there's a lot of things in functional languages that don't simplify how people code. So we're trying to separate the two. Got it. So I thought one way to kind of explore Dark is to talk about some of the big pieces or big ideas behind it. So maybe the word deploy list is a good place to start.

Yeah. So I always bring it back to the big picture. So the big picture with Dark is what specific accidental complexity is it that we're addressing today? And the two major parts of accidental complexity that we're, that dark in its current form deal with is the deployment complexity and the infrastructure complexity. There's two other major parts. One is API complexity and the other is sort of the tooling complexity that we're not solving as much today as the other two. So getting rid of that deployment complexity is what we're calling deployless. And it's deployless in the way that that's serverless. With serverless, there's still servers, but you don't think about servers the same way.

With deploy list, there's still a deploy step, but that deploy step is cut to the smallest possible thing. The blog post I wrote about this was how dark deploys in 50 milliseconds. And 50 milliseconds is literally the round-trip time to the server. So it's the smallest amount of time it could possibly be, and that's what we were going for. And I think that's where the name Deployless, which we didn't actually coin, came from.

Cool, yeah. I think that kind of leads us well into another big technology that enables the Deployless, the structured editor, and why you chose that piece of technology? So we chose a structured editor for two reasons. One, that deploy list was what we were going for, and we knew it couldn't be deploy list if we were taking the current way of writing code, or the way of deploying code. So today, you take a bunch of code, you push it to GitHub, it gets put in a package by your CI,

and that package is maybe a Docker container and it gets sent to a registry and then it gets pulled in by your orchestration. And you can deploy other ways. You can do a Heroku deploy or a DigitalOcean thing. There's a bunch of different stuff, but they all end up in the thing of take all of your code, put it in a box, replace the old box with the new box. And we said, all right, what is the smallest possible unit of deployment?

And the smallest possible unit is, you know, the thing that you just typed, the keystroke that you typed, and that's what we're deploying. And in order to do that, if you were to use today's technology, every keystroke you type, you'd still have to like package up the whole thing,

put it in a box, put the box on the server and stuff. So we're like, okay, how can we make it so that people can safely write code? And that's the tie of the structural editor. The structural editor makes it such that you never have your code in some sort of structurally invalid way. So there's no syntax.

It is not possible to have a syntax error, to have a parse error. And so you can't just take this working code and then mash the keys and end up in a bad place. If you mash the keys in dark, assuming you don't hit backspace very much, you'll probably end up with still a working program. That's funny.

But of course, the immediate question that people ask is, every programmer is deploying to production all the time. This must be chaos. Yeah, yeah. I've started trying to define what deploy list means. And I realized that this was an important thing to bring into the place because it's not it's not just like, it's not just deploy list to our sync to your server. A key part of it is that the tooling that you're using allows you to do this safely. And by safely, I mean that you don't take down the site. I was going to say frequently, but actually I mean at all. That there's a structure that prevents you from taking down the site.

And Dark mostly uses feature flags. We have a bunch of tools around this. Function versioning, feature flags, language versioning, database migrations, those are kind of the main things. But the point is that we're integrating the tooling that has the essential complexity of a deployment and that has almost none of the accidental complexity. Okay, so I think maybe I just want to spell that out a little bit in terms of the feature flags.

So you've taken a feature, sorry, you've taken a feature, I guess. Feature flags themselves are a feature of a programming tool chain, and you're using them in a, you're overloading them, you're using them as like almost like branches? Yes, so where we're kind of starting from here is the realization that over the last 10 years, for anyone who's doing proper continuous delivery or continuous deployment, whichever word you prefer, there's

a separation of getting the code onto the server from enabling that code for users. And the realization comes that once you have that full separation, which typically uses feature flags, then the actual getting the code onto the server is not important. It has to get there. It still has to be done. But it provides risk, and it's not an essential part of how people think of their interaction with their users. The way people think of their interactions with their users when they deploy things is, you know, is this going to break users? Is this change going to be okay for users?

Is it going to cause them downtime? How do we make sure that the right people see it at the start and that we have a slow rollout and that we're able to be sure that making this change does not take the site down and so on? Got it. So feature flags, this is sort of a glorified if statement. And it's an if statement that you can modify from outside the code in a certain sense.

And yeah, that's the tool. That's the main tool that we use. So if you have a piece of code in production, you wrap it in a feature flag, and doing so is atomic because of our structured editor and also because of our language. There's no stage that which you go through where the code doesn't work. So you add a feature flag, you add the conditions for the feature flag, you add the new code, maybe you just like cutting and pasting and changing a character or two, But you can do it fairly fine grained. And then, you know, you enable it maybe just for

yourself, maybe for your team, for your QA team, or maybe it's such low risk that you're enabling it for everyone at once. And then you mark it as done, and it is in production. Yeah, so I think there's a part about dark feature flags in production that I didn't quite get until you just explained it. So when you start using dark, the tool, every change you make, unless you put

it behind a feature flag, is in production, which is exactly what you want when you're starting a project and nobody's using it. But then, the confusing bit is that we all, and anyone who's worked on a production tool knows that once you have people using it, But then the confusing bit is that we all, and anyone who's worked on a production tool knows that once you have people using it, you want the reverse workflow.

Like any change you make should by default be behind a feature flag. But that isn't how Dark defaults today. So there's two phases. And any piece of code will go through those phases. Like they aren't company lifecycle phases. They're like almost like piece of code will go through those phases. They aren't company lifecycle phases. They're almost like piece of code phases. And the first phase is no one is using it.

It's a new feature. It's a new something. And so you can basically do whatever you like in that case because you're the only user. So you're starting from a blank canvas for this new page, let's say, or this new API. And you type it all out and iterate, and you can completely delete it.

No one cares because there's no users. The second phase is you've got this thing that hundreds or thousands or millions of people are using, and you have to be extremely careful with how you roll changes out to that. And this is the part of coding today that people have almost no support for, where we write all of our code today as if it was the first one.

In Dart, there's two separate modes for that. When you have users, all of your handlers lock. And by lock, I mean that you can't make changes to them. And then you must make changes by a feature flag. And then opening a feature flag gives you the first state again. You're in this fresh paradise because no one's using it. And then once people start using that, again, it goes into that lock state. Got it. Oh, okay. I guess I've never built a dark thing with users, so it never triggered

that for me. That's cool. I'm not being particularly careful here about dark division and dark the current state today. I see. So we don't automatically lock today, which is probably why you didn't see it. I see. So like your customer will tell you, I have users now, and then you'll switch the mode for them. So they can lock it themselves. Oh, okay. Yeah, and the issue today is that we're not 100% happy

with our feature flag implementation. And so we don't want to impose it on everyone. So people can use it, but they don't use it as much as we would like because there's still some product and tech that needs to be done to make that a really, really good experience. But that's the plan. Okay, that makes a lot of sense. So the thing that I have begun to realize is that the distinction,

like the incidental complexity you've removed, is that normally you would get the code working on your own physical computer, and then you would figure out a way to get it working in some container and then you have to like actually deploy it to some other service and orchestrate it so it was like this whole um rigmarole to get it working like from your local physical computer to some some other place and so the distinct the real distinction dark makes is that there's no physical computer involved. So it's always working on other people's computers. And then whenever it is that you want to turn the switch to get it shown to users or not shown to users, that's up to you. That's unrelated.

But the actual problem you solved is that we're always working in the cloud. Yes, yes. So we're always working and then these are uh tools that are built for the the problem that you actually want the the essential complexity of of building features is when do we is when do we give this to users not how do we get this to that machine over there totally yeah okay that that makes a lot of sense. That's very compelling. So then I guess a follow-up question is, if you're most of the time developing, so you explained earlier that part of the reason you wanted structured to be adding new features behind feature flags, couldn't you get rid of the structured editor and then just whatever little bit of text they put behind the feature flag, just send that little bit of text over? Like only if it parses, right? Yes and no.

I think that someone could do something roughly like that. It would have, it would take a great deal of work to make that effective. The main things that break with that. So one, you've got this parser and now you've got an AST. And in fact, you've got two ASTs. So AST is the abstract syntax tree, which is the representation of the program. So you've got the representation of the program

that's in production, and you've got the representation of the program on the user's thing that was just parsed. What is the distinction between these two things? Do they have a mapping to the other? I mean, they should, right? Because you've only made one change, but they might not. So if you've got this pure textual thing, it's hard to sync up the old text to the new text. If you've wrapped something in

an if statement, for example, might be a good example of that. And the textual mapping algorithms around that are challenging and not particularly advanced. So it's keeping track of the identifiers of each part of the AST. Yes, I mean, in Dark, everything has an identifier. So you're making a change to this identifier. And you don't see the identifiers there, you know, they're IDs, they're not displayed in the code. And I'm not saying that

you couldn't do it without that, but it certainly simplifies it. There's a couple of other benefits that it brings, notably the fact that the editor, you can't have a syntax error, that isn't possible to have a syntax error. But the thing that was interesting to us is I thought the structured editor by itself would provide this, but it actually turns out that we need a ton of language features as well. So feature flags are not really an editor feature so much as they are a language feature with editor support. So there's a semantics to every incomplete program, and that semantics of

incomplete programs, the fact that dark represents incomplete programs turns out to be really important for this sort of editing in the cloud. Because otherwise you have a program in an incomplete state and you're working with text to get it to a complete state and it might be hours between those two. With dark, the language and editor i guess um there is never there is an incomplete state but all of those are fully representable and and have full semantics i see so it's the automatic whole insertion from cyrus's work uh they're very similar to um uh to the hazel stuff yes cool um yeah because i think that's what he talked about. The reason you need a structured editor, if you want to solve what he called the gap problem of never having an incomplete

state, is that you have to automatically insert the hole. So if you like five plus, in our mind, there's a hole there, but the computer doesn't realize that there's a hole there. So you could type a question mark there or an under bar there but um a structure editor would do that for you automatically so yeah is that what you were saying or if there's something else uh so so so it's it's that but it uh there's a lot more than that so um let's say let's say that that you're in a text editor today and you're writing an if statement you're going to write if space you know some condition and you're in a text editor today and you're writing an if statement. You're going to write if space some condition. And you're going to have no tooling support while you're writing that condition because it hasn't filled out the then, the whole for the then,

and the else, and the whole for the else. So it's going to be in an incomplete parse state then the else and the whole for the else. So it's going to be in an incomplete parse state until you get the braces in place and the new lines in place and all that sort of thing. And in dark, when you type if, that creates a fully structured if statement, similar to IntelliJ or something like that. And the semantic analysis keeps running as you're writing that code,

as you're writing the condition. So we're literally running your code to show you the results of what that execution would be and to show you things like type errors and that kind of thing. And you need your engine to understand what it means to be in this incomplete state. And if you look at something like languages that have specs and they have this undefined behavior, where sometimes for even complete programs, they don't know what the meaning of a particular program is.

And in Dark, every program has a meaning. And I'm pretty sure that that's essential to having, at the very least, a good experience here. Okay, so I think I've been convinced why, in order to solve the problems you're trying to solve, you need a structured editor. But it's a hard thing to build so maybe because it hasn't really succeeded yet like there's no I guess maybe the most successful one that I could think of is scratch the blocks based one well excel well excel isn't in my, structured because you're typing the formulas all in text. Yeah, that's fair. Yeah, I guess it... A lot of the program structure is structured, but the final snippet of code is...

I mean, I guess you have text editing for an expression, and then you'd sell as an expression. Would you call Jupyteriter notebooks or observable observable notebooks um structured editors because they have cells that you type big blocks of code in i guess i wouldn't but i'm with you that like that yeah i i i see what you're saying there like if you were if you were to use jupiter notebooks to write very small blocks of code and and there was a little bit of text editing to use jupyter notebooks to write very small blocks of code and and there was a little bit of text editing in those blocks i'd be willing to call that a structured editor i see okay so i guess it's a bit of a spectrumy thing how how structured is the editor

right right right right i mean so the first uh structured editor that that i used and that i really loved was was par edit um which is which is a mode for Emacs for writing Lisp in. And that, I mean, partially this is because Lisp is so, you know, just doesn't have very much structure. But it amazed me how little it did and how amazing that was. Like how much it could do with so little. Essentially what it does is it, it, um, whenever you insert a parens, it inserts the closing parens and it puts your, your cursor in the right place. And then it has, uh, commands for like pull the next expression into the, um, into the,

into the parens. Uh, and it made sure that your program was always well-formed, which for Lisp is really important. It completely took away all the downsides of Lisp or all the parentheses-related downsides of Lisp. And I was really impressed with this. So question, how does it know where to put the closing paren in ambiguous cases where it could go in a few different spots? Well, so it's your choices,

we're talking about par edit here, right? Yeah. Yeah, so your choice is you insert an open parens and it will insert the close parens right here. And then your cursor is inside that and use the command to like pull more things into it. Got it, got it, cool. And, or the other thing is you can select a bunch of text and it will figure out what expression that text is in

and wrap the whole thing with it. That's another command that you have or you can unwrap as well. And I think that works because Lisp is so simple that there's no more syntax than that. But when you're building a structured editor for a language with, I guess, more complicated syntax, like Daruk or Haskell, it seems, based on the amount of effort I've seen people put

behind it, and yet how less than ideal the outcome is, it seems like it's a really hard thing to do. And when I tried myself building a JavaScript structured editor, it was like, wow, this is way harder than I thought. Yeah. So we're on probably the, maybe it's the second or the third iteration. And I think you haven't used our new fluid editor, right?

You've only used the existing editor. I spent maybe 20 minutes in the new editor. Okay. So our progression was that we started with this thing that was totally new, which was like a graph-based editing. And that sort of went nowhere, and we spent a couple of months on that. And then we had this regular-looking AST- based editor where you were filling in holes. And that's the default editor today.

And we realized that there's a whole bunch of problems with filling in holes. Mostly that you actually want to make larger scale changes to things and you don't necessarily want to go to an editor command to add a let above. Most common feature we had is, how do I insert code above the current line? And so we realized that we were doing it totally wrong. And we had this vision of, all right, what if it's still structured, but it feels like code?

And that's why we call it Fluid, feels fluid. So Fluid is still, it's not quite done uh it's currently about 4 000 lines so 4 000 lines of a camel um so in terms of of your question of like is it hard yes it's absolutely hard um there's there's so many edge cases um and there's there's so many like you know, if I'm here and I press this character, what is the thing that I'm trying to do? And the result for us has been really, really good. Like, it feels really good. And it's interesting that when we look at what some other people in the space are doing, I saw someone come up with this thing called TofuJS. And it was an experiment that this guy Gregor did. And it's really nice. At least the demo is really nice. I think it doesn't do all that much.

But it turns out it does basically the same as us and has a lot of the same design principles as us, and it's really cool. Oh, interesting. Is Tofu.js a projection editor for JavaScript or for his own language? I mean, I think it's a demo of how one could be. I don't think it's a full editor. Got it.

Because one question I wanted to ask is, given that it's so hard, do you see any way for it to be done in a language agnostic way? And then everyone could just use it kind of like Codemirror, for example. I think it would be hard. I mean, sort of in the same way that language server protocol is hard. Doing it for one language is one thing, doing it for as a generalized thing. So what we have is like a giant state machine of, you know, when you're in this state and you press this input, what do you do? And most of those are probably not generalizable. The default case of I press a character and it gets inserted where the cursor is,

is probably generalizable. And the 500 edge cases, probably less so. And they differ for every language and all the languages have different syntaxes. I would not enjoy that project. This sounds like a not fun project. The compiler guy is uh calling quits dangerous yeah well so one of the things i always hate about compilers is parsers and i mean essentially what i just described and i feel uh a lot of what this fluid editor is is

sort of a reverse parser um you're you're you're generating the ast uh so you're generating text from the ast and keeping enough state uh to to know what uh what what state you're in i feel it feels it feels very like parsing uh and i feel that that we're going to get into like the generalizable world here we're going to be in in some sort of like parser generators and that kind of thing. So when I was using Dark, especially the fluid editor, I guess both of them, they're both in very early stages where things are morphing, but there were like a bunch of bugs, like trying to get into states and whatnot. And I've used other structure editors and they also

are either very restrictive and not fluid, or they're buggy. And so part of me wonders if there's a way to ever be confident in having caught all the edge cases. And in such a way that instead of 4,000, maybe it'll be 100,000 lines of code. And if someone wants to change a part of it, how will they be sure that it doesn't all break? I guess this is a question about, I guess, all programming. Yeah, I mean, it sounds like you're asking the question, like how does one software engineer? Okay, I guess that's fair.

In particular, structured editors is the kind of software that people listening to this podcast, I think, are disproportionately likely to build themselves. Yeah. That exact same piece of software. So if you have any software engineering advice specifically for Structured Editors, that's kind of where I'm getting at. So I think the thing that we started doing very early was unit tests. So far, far sooner than I would typically write tests. And we're using a statically typed functional language. So testing is sort of a thing we do when we find bugs,

as opposed to when we write code initially. But with the fluid editor, so we have hundreds of tests now. And there's more tests in fluid than in the entire rest of the application. And so the first thing we did was we made a test harness. And we made a test harness that made it super, super cheap to write this sort of thing. I mean, that's a general good testing practice.

Thing that we're gonna add soon is fuzzing. I think fuzzing is an essential part of finding all those edge cases. And I think once you run the fuzzer for long enough, and in a certain sense that this is similar to parsing as well, if you run your fuzzer in your parser for overnight or days, you're eventually going to realize that it's caught all the bugs. So yeah, we have a pretty immense test suite. One of the things, and this isn't a great answer, for most people is that we invest a lot into it. So, you know, there's multiple people on the team here who are building and solving those edge cases.

We're spending quite a lot of money on this. And then there's users who are trying it out and finding the bugs, which is not a wonderful answer for people who are maybe solo projecting it, but that's how we got there. Yeah, that makes sense um uh let's talk about the dark data store um could you compare it to well i guess maybe first tell me like how proud of you of it you are because there are like a number of things that are like in progress decision so if it's something that like you're not proud of yet we don't have to talk about it but if you're proud so it i I would describe ourselves as not proud of it yet. I think we're fine with it right now.

And it's one of those things where we're on step two of six, and I think we'll start getting proud of it around four. Okay. So the way it works at the moment is that when you have a data store, data stores are key value stores. So you can store a value in it and store it by a key, look it up by a key, and there's also, you can query it. So you can query it by things that aren't the key, like by other values?

By exact matches of field names. Okay. So if you've used Mongo or Redis, I know it's not the greatest, Mongo especially isn't the greatest tool to be comparing to, but you save values in a database, so it feels a little bit like document stores. And what we're unhappy with is that we don't use

real types for them, so that's kind of our major thing. And then the next one is database migrations, so handling data migrations, not is database migrations. So handling data migrations, not just schema migrations between when you're going from one type to the other. And all of this is specced, but not implemented. And then I think we'll be relatively happy at that point. I think the next thing that it will take for me to be really happy with this is having more than just storing records in it, but being able to store anything. And then after

that, we'll have more persistent data structures. So not just key value stores, but like values, lists, et cetera. And at some point, we want to add back relational aspects to it. So the initial version of dark was relational, and we realized that the data model didn't match a relation. So data model of the language didn't match a data model of the code.

Sorry, of a relational database model. So we tore that out, went to persistent key value stores. And the thing that, like, you know, I sound a little bit negative on my own product here. I guess the important thing is, you know, it works. Where we're going with it, one of the particular challenges is when you look at something like Mongo and how it moved its query language,

what Mongo and Redis and a bunch of others said was, SQL is bad, we're gonna come up with something different. And the thing that they came up with wasn't great. It's this sort of like language that's sort of hacked into key names and that kind of thing and ends up being really weird. So we're not happy with that direction.

What we want is to be able to do a query that has any arbitrary code in it, and for us to compile that to a really fast indexed query. And that's where we're going with this. So rather than looking things up by specific fields in the record, you'd look things up probably by a function or a lambda that says if this field is this and has the prefix of this and all this other stuff and all this complex logic. Yeah, that's really cool that you can unify the query language and the programming language that way. Right, and people have done this before. So there's a link in the C Sharp world.

That's actually the only one I'm really familiar with that does this well. But one of the problems that lead to all this accidental complexity is that different domains you can't use the same language for. So if you're SQL, like so if you're writing something in Rails, then there comes a point where you have to drop down to SQL and now you're in a completely different world. And a world that you probably have no familiarity with. So we want you to write the exact query that you want

in the dark language and this is part of the accidental complexity that we remove is figuring out what your infrastructure is. So one of the major things aside from the solving deployment is solving infrastructure and we want you to just write the application and we will compile that to the right distributed system for your service. And part of that is this query optimizer. I liked your post about getting the benefits of both static and dynamic typing. Because that's something that I find myself

saying a lot. But I think it's kind of rare. I think you and I, we both have this conviction that there is a way to get the best of both worlds with the editor support. Right, right. If you add tooling in, like current languages do everything in the language because that's the domain that they have. And that's what they're able to work on. But some things are better done in an editor than they are in the language. Yeah, that's well said. So I feel like all the flame wars between static versus dynamic,

we can literally end those wars because you can have something that feels as dynamic as you want, but then on the back end, somehow it could warn you that certain choices you made while prototyping under certain conditions that it could satisfy for you will lead to errors. Yeah.

I mean, there's kind of two aspects to this static dynamic war. And so one of them is uh you know how how much can you guarantee is is not going to um uh how many bugs can you guarantee cannot happen um and the uh then the other side of it is like what what constraints are you going to apply to make that the case? And one of the major constraints that is applied by all these statically typed languages is you can only have one type of a thing in a certain place. So in Python, a list can be anything, which has benefits and has downsides.

And in Haskell, a list is of a particular thing. Now, it might be of a particular thing that's quite general with type classes, something like that, but it's of a particular thing. And I think a lot of the backlash to C and C++ came because the cost was high and the benefit wasn't all that. And I think when you look at the statically typed functional languages, the things that they move into the type system, in particular exceptions and null, or the absence of null, I think makes a really, really big difference and provides a really solid move forward from the other static languages.

And this other movement that you have at the moment is gradual typing and bringing in things like TypeScript and MyPy and static typing into all these dynamic languages. And my impression from that is that they're bringing in this very classic Java slash C++ model into languages that aren't that at all. So when I was trying to MyPpy up the initial version of Dark, which was written in Python, and I discovered I did all this dynamic things, and they weren't represented in the type system at all.

So I think there's a reason that people are adding these types to these systems, because it's kind of hard. But I think the other way might be the way to go rather than starting something that's super dynamic and start to introduce all these constraints. What if you started with the super constraints thing and then relaxed a bunch of them via tooling? Yeah. And that's essentially what we're doing with Dark. We're starting with something

that's roughly like a Camel or Haskell and saying, all right, what can we relax here? In particular, what can we relax that makes for a better coding experience during different phases of it? Yeah, I personally think of it as a scaffolding. So I think, I guess to highlight, a thing you do particularly well in dark is automatic unwrapping of maybe or option types. Because when you're prototyping or when you're building just for yourself, you don't want to deal with error states

because you're doing something, prototyping or in your building just for yourself, you don't want to deal with error states because like that just, you know, you're like doing something like you just don't want to, you know, it's like not part of the thing you're building. You just want to deal with the success cases. And then maybe if this code survives more than five minutes, you'll deal with the error cases. And so, yeah, you do a great job of that in Darktip. Right, exactly. Explain it. Yeah. And that's the really important thing. And the real downside that I feel that you get in static-type functional languages is that you pay the cost for code that might not survive.

And you don't do this in Python, right? In Python, if you're prototyping something, maybe if you write unit tests, if you're a tester in development, maybe you pay the cost up front. But certainly when I write Python, I prototype it, I get it roughly in shape, and then I add the stuff to make it work. And that option isn't really available in OCaml, in Elm. You are forced to pay the cost upfront and then discover that actually your algorithm was wrong. Well, you've paid all this like static typing cost

and it would have been much better to not pay the cost to discover that the algorithm was wrong. And then when you get the algorithm into the right shape, now is the time to like, to add that cost back in. And we also have um so we have like the ability to tell you did it you did the thing that you're worried about actually happen what is the priority of this um so you can ship the thing that like you know hasn't had all the edge cases taken care of and we'll tell you which ones actually happen in practice. Yeah, so let's talk about how that looks in the editor. I think you use the term

railway programming? Yeah, so railway oriented programming is this concept by Scott Latchin, I think is how you say his name, who is an F-sharp consultant, and he has a web page all about F-sharp and functional programming, which we should link to in the show notes, and whose name I can't remember. I think it's something like F-sharp for fun and profit. So he has this concept as a way of teaching monads. So as a way of teaching this sort of like, you know, dual system of real values and error values.

And he teaches it with the concept of a railway. And so a friend pointed this out to me a couple of years ago. And I went through all the stuff and I was like this is a really good model can we can we do can we actually put something like this in the editor and so what how this appears is on the right of of the editor so you know on the right margin that's that's where the rail is and values can be on the rail or off the rail. Being off the rail means that you're handling them with a match statement or with some of the monadic constructs like maps and option dot and then, that kind of thing.

But if it's on the rail, we just unwrap it automatically. And then this is where we envision putting our built-in sort of exception tracker. So the thing that tells you in production all this went wrong. And we'll bring you back to that particular trace. And then you can see in this trace, there was this data and it led to this thing being wrong here. Yeah, it's really cool. And I don't think this is in the product today.

Well, actually, it's really cool. And I don't think this is in the product today. Well, actually, it might be. If I do a search operation in a list, I guess. I guess it's called find maybe in JavaScript. Yeah. Will that automatically go on the rail? So it would assume that I found the thing? Yeah, okay. Yeah, so anything that returns an option or a result automatically goes on the rail.

And you can pull it off if you choose to. So if you think in this case it's always going to be there and it's essentially an assertion violation that it's not, then leave it on the rail. And that's a case where you want to be notified if it's actually on the rail because you misunderstood something. And so if a user hits a rail,

a condition that's on the rail is false for some reason, what happens to the request? So it will either 500 or 404. So for options, it will 404. For results, it will 500 and show the error. So we're going to add more structured things. So it's like, oh, this particular error shouldn't be a 500. Or this particular error should have this error message or something like that.

I see. I think we hadn't defined the terms uh in option is kind of like uh what in haskell and results it's a maybe in haskell okay option is maybe and results are uh i think i think people in Haskell use the term result, but if they don't, they use either. Got it. So a result is an either which uses the constructor's okay and error instead of left and right. Got it. Cool. All right, yeah, maybe it does have result. I haven't used Haskell for web programming since college, so I don't know the state of the art of the type

classes so um you mentioned briefly that you used to have a what I call a node and wire editor data flow diagram um yeah I I um I maybe I shouldn't but I I call it a bit a phase that all of us uh kind of have to go through. Right, right, right. It's like grunge music. Yeah, exactly. It's like we all, you know, like start with regular programming. Then we're like, we see the first one or, you know, somehow we think about visual programming and that's kind of how we think of it.

And so we're seduced by it and then we start making it or using existing ones. And then we're like, wait a second, this isn't all that I thought it was going to be and then you you switch back maybe to a structured editor or maybe switch back to text you're just done with done with it would you um and so i normally advise people to just either just go for it if they really are into it or just skip that phase but people don't really listen to me would you also advise people to skip that phase or do you think that there's something there that could be useful and that maybe if someone really worked on the node and wire thing,

they could figure something out? I was reading something over the weekend that talked about the act of software engineering as the act of learning the problem space. So the artifact is partially the code, but it's partially what the team has learned from doing it. So I'm not sure I'd advise people to skip it. I think we certainly gained a lot from doing it.

So our graph phase was around three months. And I think now we could actually go back and achieve a lot more than we did because we figured out how to solve a lot of the problems that we had with the graphs that we could probably backport. But what we did at the end of the graphs was we said, okay, what things are really good about this and what things are really bad about this?

And are the things that are really good about this, are they achievable in another way? And the things that are really bad about this, are they like inherent flaws in how one does graphs? Or are they like things that we could like figure out new features and a problem that we had the whole way through the graph thing is that we kept we kept putting off what was what we said was a we said was a bug was actually in some cases like a core usability problem that couldn't be

solved so one of the things the really annoying things about the graph editor was that we actually had to make perfect outcomes to discover that the perfect outcome still had bad usability. Sorry, what do you mean? So layout, as an example. So we would layout out the graph editor, and sometimes something would end up in a really bad place or it would be confusing.

And we were like, oh, that's a bug. Sorry about that, user who's in for user testing. It's just a bug. And there's a lot of things where we're just like, oh, that's a bug. And eventually we realized, maybe this isn't a bug. Maybe this is a fundamental flaw in how we do it. So we wrote, we fixed all the bugs, and then we had users do the user testing,

and they still came up with these programs that they were trying to write where the graph was unusable for it, even though it was doing its job perfectly. And we realized that we actually just didn't know how to solve some of these things. So there's a simple example. This sort of relates back to the Excel thing that we were

discussing earlier. So we had this, how do you write if statements with complex conditions? And when I say complex conditions, we were just trying to get people to write fizzbuzz, so not even that complex. So in fizzbuzz, you've got if value mod 5 equals 0. And so it's two bin ops with a specific precedence.

And we discovered that it was just really, really hard to write that because each bin op would form its own node in the graph. And so we were experimenting, well, maybe for conditional expressions, we can have a text-based thing. We actually wrote a whole parser for single conditional expressions. And so we eventually got it to the point where users could write it, but then when they went back to edit it, we would expand it out to the whole thing and then they were like, what the fuck is this?

So we just, it's just like it wasn't... Not only was it not easy, but like new problems kept coming up all the time and we didn't have really any answers to a lot of them because the model was just like so different and people people i mean fundamentally people were confused all the time and we were confused as well like there's a lot of times that we would write things and we're like yeah i kind of don't know what i would expect to do here so what we got from the phase was like that there's some specific things that we're trying to do so we're're trying to represent what we call the architectural view. So we're trying to represent how data flows

between different components in your programs. But that doesn't have to be done via a graph. Yeah, I don't know about other listeners, but to me, it sounds like a pretty strong case to not do the node and wire phase. But I guess i think a lot of people come from tools like pure data or other um artistic tools that have a data flow graph that they had a lot of success with and they didn't even use a keyboard you know they were like

you know clicking and dragging and um and so that they they want to translate that same experience to full coding but um my hunch is that the reason it's never really worked is because full coding is arbitrarily complex in a way that, I guess, I don't know. Songs are pretty complicated, but I don't know if it has the same kind of Boolean logic and if statements. Yeah, I mean, the things that we were trying to represent were like lambdas and requests that have data coming in and data coming out. And they sort of are in the same place.

The input and the output are sort of in the same place. And we were trying to show what we call live values, which are sort of light table-esque data to the side of what data is actually flowing through this particular part of the program. And I think there's that thing that you said earlier. The initial case is like, it looks extremely promising. And the initial the initial demo I made for dark before before we raise money this was this was all we had

but it was a two-week demo of like can you write code with this and I was like yes you know I understand what this is each node in the graph is a function each each line is an input very similar to Luna if you've seen Luna. But yeah, we couldn't make it feel good. Okay. I want to spend some time on the full vision for Dark. I don't know how much you can approximate it, but one day, how far will you go? Will it one day be for writing um the front

end and the back end will will you use it to write your apps for mobile devices um and then i guess like maybe on the outer edge of what your vision might be will you one day have something that looks kind of like coda in the sense that it'll blur the line between using software and making software or you're going to stop stop well before that and it'll still be someone writes software and other people use that software? So I guess I should talk about them separately, but the answer to both is yes. So let me start with the Coda one.

We don't plan to go into the low-code, no-code side of things. We don't really believe in that, except for a couple of things like layout and design and that kind of thing, where we feel that no-code is actually a good metaphor for it. But our intent is definitely that you don't necessarily need to make a tool to make the transformations to your state.

So what I mean by that, we write handlers in dark that don't take inputs. We call them repls, which is a slight cheat on what a REPL actually is, but where you can just write code and execute it. And then we save them and just leave them in the code. So an example of one of these might be adding a user. So a way to add a user in Dark is to open up your Dark program, call the add user function,

and then that triggers all the send them the email and put them in the account and all that sort of thing. And that's a way to do it. And we want to build up more of that. So when you think of the admin dashboards that people build like in Django or something like that, that's the sort of thing we're talking about. And that may eventually become, you know, something where,

you know, where perhaps an individual, this is an example that we use in that sort of distant future where a recruiter might have built their applicant tracking system entirely in dark and where they have a bunch of tools that they've written in dark to do modifications to that. So it's not going to feel like the documenty metaphor, uh, I think. Um, but there's, we definitely want to blur the line between, you know, how you write software and how you use software. Cool.

Um, so to, to the other question, you know, does, does dark, uh, go into front end and mobile apps? So the, to a certain extent, I don't think that we're replacing Swift or React. We don't intend to. In fact, when we look at other tools in this space and Meteor is one example, one of the things that led to huge problems for Meteor

was that they were trying to be both front-end and back-end at a time where there was a huge explosion and excitement in front-end stuff and that you couldn't use it in the initial version. They later let you allow support React in Meteor. So we don't intend to have dark compiled to JavaScript that replaces React. What we would like is this GraphQL stuff

that you're writing, maybe we can make that easier. Maybe we can add an SDK to React or to Swift or whatever, which does a lot of maybe the transport layer. Maybe we can automatically optimize transport, maybe keep the types the same in your backend or your front end, that kind of thing. And beyond that, really no idea where that goes. I don't imagine people writing single page apps in Dark

though, at least not into some long distance in the future. And if that does happen, it will likely come from the community rather than something that we're planning to build. Just to be clear, I think what you mean is that people will be able to build a single-page app with Dark as the back-end, but you won't be able to build it. Sorry. Yes. Yes. I, I meant, I meant, uh,

we don't expect that people will replace writing react, uh, or writing view or, or Svelte or whatever with writing dark. Yeah. Uh, but they will use dark for the backend. That, that, that is what dark is really good for. It's really good for being the backend to a single page app. Yeah, yeah, exactly. I was pretty sure that you page app. Yeah, yeah, exactly. I was pretty sure that you weren't. Yeah, yeah.

Yeah, you just push your static assets to dark or some other static place, and then they just talk to dark, yeah. Yeah, yeah, exactly. So I want to push back on your claim that you aren't into low code or no code because um i i think that dark is no code in the same way that salesforce is anti-software not software you you take that back well in the same way that salesforce is not software because salesforce clearly is software it's just not the annoying parts of software that people used to think associate with software it's like when a business person thought of software 20 years ago they thought of like these massive contracts and it's like two-year install period and like training isn't that what salesforce is today

no salesforce today well i guess it is kind of yeah that's actually true there is a lot of that today but the promise is that you um you just like you know go to a browser and you sign up and then you just start using it. And the software itself kind of disappears because you're just dealing with your customer data. That's like the dream. You don't realize you're using software. You're just dealing with your customers. And so I feel like dark is no code in that way. What they had as, you know, what they were talking about when Salesforce said no software is like, you know, it's in the cloud. Yeah.

There's no installation. And, you know, that is sort of the same. And I think that's where the similarities end. I think no software was a very bad metaphor. I think it was very good for their customers. With developers, developers largely do not want... They like code. They want to write code, and they want tools that make coding better

as opposed to tools that take away uh that take away that coding and so when you think about so coding is kind of like you know symbolic reasoning um no code is write software without symbolic reasoning uh so without the thing that we have spent years getting good at and without the thing that is super powerful. Low-code is writing code with different tools that don't feel codey. We very much intend to build simpler ways of writing code that feel like code and that contain the standard metaphors of code. Well, I think this is obviously a semantic discussion we're having. It is, absolutely. But I think it's a fun one because when I think about the essence of code,

I agree that it's like symbolic reasoning of some sort. You know, like the idea of encoding your ideas into Boolean algebra seems pretty core to coding. And if you're not doing that, then you're doing something else. But if you look at Webflow, which has put itself at the center of the no-code or low-code movement with this no-code conference,

they're like a CSS compiler. And they have animations that you can code in there. And they have a whole CRM. It's very clear to me that Webflow is programming just with a direct manipulation. It was just a nicer interface. So from that description, I think the word no code is becoming to mean, even though it's phrased no code, it doesn't actually mean no code. It means none of the annoying parts of coding.

I think it means no syntax errors and things like that. Because I imagine if I put someone who's never programmed... That isn't how I perceive the semantic, the meaning of no code. I know that that's not how the word sounds, no code, but I just imagine putting my brother or my girlfriend, people who could definitely use Dark, even though they could definitely not use any other backend tool, like not even

close. I feel like, yeah, so many of my friends could use Dark. And I think they might not realize that they're programming. I feel like they might be like, oh, this isn't code. Because they don't know what code is. They just know that coding is a lot harder and there are a lot more error states. They've had some negative experiences with coding

that they're not having here. This is like using a regular piece of software. Well, I mean, it's got a black background, so it's coding. Okay. No, but the... When I think of what's hard about coding, you're totally right. This setup, the git, and setting up npm, and bash, and terminals, and vim.

There's a lot of things that add to this huge complexity that you have to get over in order to write your first line, your first working application. And you can go through a tutorial and you can end up at the other end of the tutorial with an app that does something and you have no idea what you did. That was just the stuff that you had to do to get there. But I think, sorry to interrupt,

but just quickly i'll interject i think that what sticks with people and like the medium is the message sense is like all that annoying stuff they had to do that was the coding and like a little bit of code they wrote like to get the hello world to show up that was like just for fun like you know like they don't really i think i think it's right right but but that that's what the coding is like the you know when we talk about like symbolic symbolic reasoning, it is the print statements. And everything that's around that print statement is crap. It's accidental complexity.

So I feel like what I'm getting at is if that's your stance, then the no-code movement, there's something wrong with it because the no-code movement in theory is getting rid of all the cruft. And also it's getting rid of the code, too? It is. I mean, I think that's fundamentally it, that no-code is about... They came to the conclusion that coding is hard.

Oh, I see. Because they saw that the other stuff was hard. They didn't realize you could remove that stuff. Right, exactly. And so if you remove that stuff but kept the coding bit, and you kept the ability to write during complete programs, that can do anything. And you can see this. I saw when you did Coda for what I guess is in the the no code catalog but

when we saw that the preliminary version that there was a bunch of stuff that you couldn't do a lot of programs you couldn't write a lot of a lot of you know sort of logic that couldn't be represented and like I'm willing to stipulate that Excel is coding. That's true. Excel, I'd sort of call low code. But underlying Excel is like there's a lot of code. And there's a lot of formulas that you write. And it's stripped away all of the NPM install, the Git init part of coding.

Most of the no-code tools I've looked at, and I do not believe that they fundamentally allow coding under it, which clues in the name. So where would you put Webflow in a spectrum? So Webflow, I think, is in the category of things where a visual editor is really a good idea. So I think what no code is really good for is problems that are design-related

and problems that maybe we shouldn't have been doing in code in the first place. So I remember when we were in college, we had this project and we were creating a Windows app. And so someone wrote all the code to do it, and it was buggy, and it was hard to get right. And then instead, we used the Visual Studio built-in drag and drop thing, and it created a ton of code. All the code was unreadable and so on, but it worked really well. You see this in Swift as well.

It's hard to write that code. There are problems that actually maybe aren't really intended to be solved by code anyway, that a visual editor is ideal for things like, how do you lay things out? So I think of Webflow as a great solution for that particular problem. So the particular problem of how do we build websites, I don't think of it as a particularly great problem for how do we architect front-end applications, which is a problem that it isn't

trying to solve. But when you've been writing React or single or single page apps for a while you you start to understand that the architecture of this application is like super important and it's hard to do um and you know that there isn't something about uh inherent to to low code or to a specific visual paradigm of editing um that that helps with that problem. All right. That makes sense. I wanted to ask you to talk about some of the downsides of dark.

I liked how you listed that that was one of the blog posts you're going to write, like explicitly calling out all the downsides. And you listed open source, self-hosting, and potential lock-in. I think those are some of the biggest ones. Yeah. So the biggest thing is probably that one. So let me start with the smaller one first. So people are going to have to do things a little bit differently in Dark.

And, you. And specifically, they're going to have to leave their favorite language and they're going to have to leave their favorite editor. And I don't think anyone really has a favorite architecture. People like GitHub. Yeah, and they're not using the GitFlow workflow or the GitHub pull request workflow. They're not writing code on their machine. And coding in dark feels a little bit different.

On your first day, you're going to feel a little bit uneasy because everything's live and you don't have this thing where you don't have a commit that does the, this thing is done kind of moment. So yeah, things feel a little bit different and some of that you'll get over. And some of that is, if you've got this super customized Vim or Emacs or VS Code or whatever setup, Dark isn't going to support that at the start. We may support similar things at the end, but we're definitely not going to do it at the start. Especially if you're a Vim user and you're used to all the Vim-y things.

We may eventually have a Vim user and you're used to all the Vim-y things, we may eventually have a Vim mode. We certainly will not on day one. Maybe you'll start with the Emacs mode on day one? Actually, if you're used to Emacs, things like Ctrl-A, Ctrl-E work nicely. And the read line shortcuts. And there's things like alt X to bring up the refactoring commands. So it does feel a little Emaxy. Oh, wow, I was joking.

Oh, yeah. And the intent actually is for Dark, for you to be able to write editor extensions in Dark, which is also very Emaxy. So, yeah. the right editor extensions in Dark, which is also very Emaxy. So that's going to be one thing. People are going to have to leave some tools. And that's a thing that people on an individual basis can decide to do or not to do.

The more major one is, of course, like are you locked into dark? And what are the options around that? And fundamentally for us, it's not a good thing for you to be locked into dark. Like your product should stand on their own merits and if people want to leave for whatever reason, and we are aware that we are definitely going to churn customers.

That just happens. At Circle, I remember that our first customer who got to spending a million a year churned and our second customer who got to spending a million a year churned. And then we finally had figured out and solved all the problems. And now customers who get to a million a year don't churned and then we we finally had figured out and solved all the problems and now customers

who get to a million a year don't turn uh but you know that that that is a that is a life cycle that that most companies go through where where um they you know the first the first people who need a certain thing uh if they really need it um that you may not be able to solve their that you may not be able to solve their problem and they may need to leave for that and one of those is scale. So I have no doubt that people will churn. And we want people to be able to churn, mostly because it lowers the risk of signing up to Dark.

We don't want Dark to be only for people who are backed into a corner, who don't have the resources, and dark is the only way to achieve what they want to achieve. We want it to be a conscious choice. You're looking at dark for its advantages and disadvantages, and you're comparing it to Go or Node or whatever, on whatever stacks they come with. So we have looked at various different ways of solving this. The one that people ask specifically about is open sourcing dark. When we look at the advantages of Dark and kind of what Dark is, the main thing about Dark is that we run it ourselves.

That we host it, that we run the infrastructure. And we've talked a bit about compiling queries and sort of optimizing that infrastructure stuff. That's kind of like a very intentional part of Dark. And there's many tools today that don't do that, that you can run yourself. So the other factor with this is the current movement in open source around,

especially with open source databases, around like who gets the value out of the ecosystem. And increasingly the answer is Amazon. And it's super important to us that, especially from the perspective of this sort of, you know, lock in question, dark needs to be sustainable and the, it has to be something that builds on itself and that the,

you know, the company or the people behind building Dark needs to be a sustainable ecosystem for this to be usable for people. And we saw what happened with Parse as an example of something that wasn't sustainable. It got acquired by someone who didn't have a strategic value of it. And for people who build on dark, having dark be sustainable is existential risk. So we are not planning to open source dark at the moment, which would be one way of dealing with that lock-in. The way that we intend to deal with that lock-in in the short term is sort of dealing with... So there's two kinds of cases. There's dark dies. Dark isn't able to raise money. There's some cataclysmic event and dark has to shut down. So what do people do then? So in that case, we're looking at creating some sort of legal framework

that says that we will open source dark if it shuts down and maybe even setting aside money or creating a fund to support us during that phase to give people the ability to start a hosting thing or to, you know, run it themselves or just, you know, have some smoothness as it as they move off it. I like the idea of like Social Security, like, when you pay for dark, like 10% of what you're paying for is just in case dark shuts down, like getting off of dark. Yeah, so absolutely not 10%. But yeah, we're looking at it more as a,

how many people could we keep this in maintenance mode with? And we need to pay their salaries, and I guess I'd be doing it for free. But yeah, there's other people whose salaries we'll need to pay. And at the start, when users signed up, and we had our first user, our second user, our sort of handshake agreement with them was, like, if this doesn't work out,

we will rewrite the code for you in Node or whatever, and we'll help you get it onto Lambda or something like that. And we want a variation of that. We can't hand write it, but we do want something that if you decide to move off dark, there's a way to get it onto Lambda or some other framework like that. So we're looking at, so this is the situation, not so much dark shots down situation, but dark succeeds, but you can't use it for whatever reason.

Got it. And so like compiling it out into a node app or a go app is one option that we've looked at, compiling it out maybe into a binary. So you have to start the code again, but you have the ability to keep running the current thing. All these solve a subset of problems. They're not a complete solution.

So we're interested in hearing from people which of these solutions are actually interesting and from people that would consider them, I know there's a lot of people who just wouldn't consider them. But I guess that's interesting information as well. Cool. Yeah, I really appreciate how honest up front you are with this choice and the trade-offs. Fundamentally, lock-in sucks for us because it prevents people from adopting.

So having a solution to it matters to us. It's funny because that goes against the common sense or common wisdom, like lock-in is good for companies. I don't think that lock-in is good for companies um i i don't think that lock-in is good for companies uh i think people start to resent the companies and it spends whatever consumer goodwill when they have uh when they have lock-in like you want lock-in from from being such a good product uh that people just can't use anything else. You don't want lock-in from, I mean, my philosophy of businesses is that lock-in from a monopolist standpoint sucks.

So many people hate Facebook and use it every day because there's no alternative. Or look at people's feelings about Verizon or Comcast. There's a certain point at which companies turn to rent-seeking and lock-in is used for rent-seeking. And that's the point at which innovation has long died at the company. And yeah, we absolutely don't want that to be what dark is. But at the same time, you fear that if you give like a lay all lock in fears and go open source, you won't be able to be a sustainable business because people will just

not pay for it. They'll just self-host the open source version. I mean, the fear that we're looking at is that Amazon will host it. I see. And, you know, this happened with Elastic, this happened with Mongo. And, you know, it's not even that, like, you know, Amazon is bad. They're, you know, no doubt they're doing the things that their customers want them to do. But we have a responsibility to our customers to make this sustainable. So open sourcing it, ironically, isn't necessarily the most sustainable approach.

I'm wondering if an open source license, like a particular open source license might, you'll just tell me why it doesn't work, might work. Could you open source it in such a way that if I'm using it as an application, like if I built something on Dark and I want to self-host it, I can do that. But if I want to sell it as a platform

in the way that Amazon Web Services would, that's not allowed. Yeah. So this is what a lot of these database companies are experimenting with. So they're source available licenses. They're not technically open source because open source has a specific meaning. And adding that provision, I think it is not possible to be technically open source and adding that provision, but I think that we're going to see a shift to this sort of source available thing. There are certain expectations that people have for something being open source, like their ability to contribute to the ecosystem and that may or may not exist in source available licenses. There's a certain thing about open source that it's much

more than like the legal definition and it's how you treat your community and how you build software and that sort of thing. And there is an element to that that will exist in dark already. So there'll be a package manager in dark and there'll be the ability to create and share libraries. So I believe that we will have many of the advantages that people expect to see from open source, but that the running the platform themselves is

not necessarily... I don't think there's anyone who wants to run the platform themselves, including people who get too close or get too big for the ecosystem, or people who get too big for our hosting thing. If we can't make it work, it is unlikely that just hosting it themselves is going to solve this problem, except for things like, you know, kind of on-prem and that kind of thing. And there's other ways to handle those. Oh, yeah, how is the on-prem, like,

when, for legal reasons, you need your servers to be on-prem, do you have a solution for that? I guess Salesforce doesn't. On-prem has had many changes in definition over the last decade or so. So it used to be on-prem meant physically on the premise. Nowadays on-prem is like running in an Amazon VPC that you control so that when you

shut it down, you get to shut it down and the company can no longer access within it. I think we're probably going to stick with more of the public cloud thing, but perhaps the lines will become so blurred that there's a thing where we can address your issues in a way that is sort of manageable within the model that we're looking to build. Got it. So one day, is the future of Dark, you're going to compete with AWS directly and run your own data centers, or you're going to always be on top of some other cloud service? I think that we will always have some amount of things running on the cloud.

So in terms of business model, we're absolutely competing with AWS. We think AWS and its 200 services are, not all of them, but a large majority are accidental complexity and that people shouldn't have to do that. In order to write code or in order to receive data and process it and store it and send it, should be much, much simpler than what exists. And what we're building is essentially a competitor to AWS. The question of when and how we build our own data centers, there's so much that are in these public clouds that people kind of need.

So, you know, building all the data centers that currently exist is hard. So, but even if you were to build them, like a lot of what people need is I need this app to like sit next to my existing app, which is in, you know, US East 2 or something like that. So it isn't just a case of you can build data centers and replace it.

I think there's a hybrid world. I think that we will start looking at optimizations, including cost optimizations at some point that will move a certain amount of traffic or of compute or storage off of AWS. I mean, we're not using AWS, we're actually using Google Cloud, but move it off those things or move it between those things

or allow you to specify constraints where, you know, perhaps you don't care about this service at all, but you'd like it to be cheap. It doesn't matter, You know how available it is Well as long as it's you know somewhat available, but like you you can tolerate a certain amount of downtime If you know if you can be run for a third of the price and in that case, you know Why wouldn't we run it on you know Hertzner or you know one of the one of the budgets? Infrastructures rather than rather than paying for the full availability

that you get from AWS or GCP instead. So we think that lots of these are options in the future, but how we think about Dark is a compiler for your infrastructure or compiling your app to the ideal distributed system for what you want. And some of those constraints are cost and some of those will be eventually, yeah, we're going to move off. And I think that we're also going to get a couple of really nice economies of scale from

that optimization. Even staying on the cloud, we might know that the certain kind of constraints you're looking for here mean that we should use this service rather than this service, and this one is a lot cheaper. I think that at some point we'll get to individually compiling databases for your workload. And rather than running it on Postgres, which is what we do now. And that will realize, you know, orders of magnitude of performance improvements and cost savings from doing that. Cool.

Yeah, it sounds pretty exciting. And now I'm wondering if someone really buys into this vision, which I think a lot of your early customers might, have you thought about running some sort of customer fundraise where they could somehow get a piece of the upside if you were able to win? I would be interested in doing that. We'll be fundraising in the near future. It is very challenging to raise money from non-accredited investors. Legally.

Uh, legally, like it creates a lot of, a lot of challenges for, for, for your company. Um, and so, so I'm not sure whether, whether we're going to do that. We, we kind of have a lot of people who are interested in giving us money. So it's not, it's not a thing that we need to do, but it's a thing that I like doing from that upside thing, but it might be legally too challenging. Yeah, yeah. That makes sense.

I don't expect you're going to have trouble raising funds from very smart investors, but I feel like you could get some amount of customer loyalty, extra points. Right, right. And I believe in that. I want to do that with Circle. To a certain extent,

I kind of want to do that for Circle customers today. That, you know, the invest in Paul's next thing. It's unfortunate that the legalities are annoying there. Okay. Two more questions. Hopefully these ones will be quick.

I wanted to quickly talk about the tech stack because as a compiler, programming languages person, I imagine you've put a ton of thought into it. Before you talk about this tech stack, I want to talk about the tech stack of Circle simply because that was how you and I originally met. I don't know if you remember, but I did some sort of a search on

Hacker News for just the word Clojure and to see if there were any companies that were hiring for Clojure developers. And that's how I found you initially. And so I'd be curious to hear about why you chose Clojure and why you didn't choose Clojure and Clojure Script and why you kind of switched to the OCaml ecosystem, all your and yet why your experiences with Elm and moving off of Elm all that that whole story I guess it's not not it's not a quick question. So initially when Alan and I started Circle that's Alan noten who's my dark co-founder um the uh i i was working on it uh nights and weekends and he was working on it full-time and so we wrote it in closure which was his his favorite language um and i i was initially against that but after about six weeks

uh of of you know getting to know Clojure and Emacs, which I sort of learned together, I came to realize that Clojure is this beautiful, elegant language, and that Clojure is great. And so our initial front end was written in Backbone and JavaScript, and we converted that into ClojureScript using a model that's very similar to the Elm architecture. That was a couple of years later in 2014. And yeah, we were, you know, Dark at a certain point was a Clojure monolith. And at some point it, you know, forked various things off.

And the problem that I found with Closure, aside from things like the REPL or having to use a REPL because the spin-up time is annoying for the JVM and that kind of thing, that's all kind of tolerable, but if irritating. The major problem that I had with Closure is the lack of static typing. And this guy Ambrose did a lot of work on typed closure, core typed it was called, which Circle sponsored, or Circle sponsored an amount of, and I think he published his PhD on it late last year, I think, or maybe earlier this year.

And it's adding gradual types to closure never ended up being good enough in the experiments that we did and we ended up with a lot of problems where there was just just a nil somewhere and you know the nil propagated for miles and miles and miles, because that's what Clojure does to nil. If you do things with nil, you end up with nil, and eventually you end up with nil, and where did this nil come from?

And yeah, that was really annoying. And I had feelings that static typing was the solution to this, and so I played around with Haskell for a while. And I'd written a bunch of Haskell years before, during my PhD, and I really liked this feeling that once it compiles, it doesn't necessarily work, but it often works or is pretty close to working you you never have to deal with a thing in enclosure or python where like uh you know you got the arguments wrong and like you you made a change to something and now this other path just like doesn't work at all and so

you have to write tons and tons of unit tests but i never was really satisfied with with haskell as a as a language i felt that Haskell brought you know a ton of its own accidental complexity um and uh and so I I wasn't I wasn't happy with it and I'd written uh I'd written a bunch of Elm uh and Elm I designed. And looking at Elm and how productive I could be with it, looking at Haskell, I was like, there's really something to this statically type functional language. And so the one that we hadn't tried was OCaml. So when we started Dark, Dark was about 800 lines of Python, and we tried to add mypy to it and just didn't get good results out of that. So we wrote it in OCaml.

And there was a very high learning curve for OCaml. There's a lot of problems with OCaml. But we approached the tech stack for Dark with the idea of none of this is going to survive. So we are solving today's problems and tomorrow we will solve tomorrow's problems. Because that was the phase, that was the early phase of company that we were at. We're now switching into a different phase of company where we think a little more into the future, possibly a lot more in the future, but at the time it's like we have no idea how much of this code is even going to survive at all. So let's write it in a tool that allows us to write

things as quickly as possible. And a particular advantage of OCaml for early stage projects is is how easily it lets you refactor things. So it's not at all unusual that I'd write, you know 2000 lines of refactor and that I had the compiler working with me to get it working. And that at the other end, you know, it mostly worked or, you know, 99% worked. Or, you know,

99% worked and there was a couple of bugs still left over. So OCaml has been great for that stuff. It has not been great for operationalizing. In particular, its concurrency story um it's it's very hard to write concurrent code in what you kind of think of a standard OCaml um you have to write this cooperative um it's called LWT um this you use this like cooperative async um libraries uh that that completely change what your code looks like and how it feels to write code. So, yeah, I don't like where we are with OCaml. Our current view is that we'll be rewriting it in Rust in a relatively near future. And there's a couple of different plans that are some amount of OCaml left to no OCaml left at all. But we haven't made that decision yet. And what about on the front end?

So the front end, we started with Elm, and then we switched to BuckleScript, which is a kind of OCaml that compiles to JavaScript. We're quite happy with that. It has solved a lot of the problems that we had with Elm, in particular the lack of native code. I think that there's a possibility that we move out of OCaml at some point to something along the lines of TypeScript. But we haven't felt that need yet at all.

Got it. That makes sense. And then in terms of where you're rendering your dark canvas, is it on the HTML canvas or is it HTML elements? Right now it's HTML elements. Talking to people who have built large-scale editors, it feels like there's a future that's WebGL. But we haven't done any experiments with that. PAUL LEWISOHNKOWICZ- Cool.

Right now, it's just pure HTML with a virtual DOM that handles most of the performance. PAUL LEWISOHNKOWICZ- Great. And I guess that just leaves me with the final question. Maybe list some of the places that you could be reached online and then what sorts of things that you want people to reach out to you for if you're hiring, if there are other things you want to let people know about. Yeah, so we're hiring. We're hiring a lot of roles at the moment, not all of which are online, but they will be by the time this podcast comes out. We are looking for users,

so in particular people who have a deadline for an app that they need to write. So rather than kicking the tires, we have lots of people offering to kick the tires and we're really focusing on people who have an app they need to build right now and that they need to ship soon. So places that we can be reached online, darklang.com is the obvious one and signing up for our mailing list, signing up for our beta and applying for jobs, a good place to do that.

By the time this comes out, we should have a lot more information on that on the website about Dark, but we have a bunch of blog posts that are currently on medium.com.com.com And then on Twitter, we are at darklang. I'm at Paul Bigger and Ellen is at Ellen Chisa. Those are the ones I can think of. Great. Well, this was really wonderful.

I'm glad we finally found the time to do it. Same, same, same. Lovely having this chat with with you i really enjoyed it great uh talk to you later if you enjoy these conversations i bet you'd fit right into our online community you can join our slack group at futureofcoding.org slash community where we chat about these topics share links feedback and organize in-person meetup groups in various cities new york london san francisco boston and others if you'd like to support these efforts there are a organize in-person meetup groups in various cities, New York, London, San Francisco, Boston, and others. If you'd like to support these efforts, there are a few ways that you can be helpful.

I would really appreciate any reviews you could leave for the podcast, wherever it is that you listen to it. If any episode in particular speaks to you, I would encourage you to share it with your friends or on social media. Please tag me if you're sharing it on Twitter at Steve Crafts, so I can join the conversation. I also accept support directly on Patreon at patreon.com slash stevekrause. And as always, I'm so thankful for your constructive criticism and feedback, so please don't be shy with your advice. Thank you so much for listening, and I will catch you next time.