The Changelog: Software Development, Open Source - Darklang Diaries (Interview)

Episode Date: February 26, 2021

This week Jerod is joined by Paul Biggar the creator of Dark, a new way to build serverless backends. Paul shares all the details about this all-in-one language, editor, and infrastructure, why he decided to make Dark in the first place, his view on programming language design, the advantages Dark has as an integrated solution, and also why it's source available, but NOT open source.

This week on the ChangeLog, Jared is joined by Paul Bigger, the creator of Dark, a new way to build serverless backends. Paul shares all the details about this all-in-one language, editor, and infrastructure, why he decided to make Dark in the first place, his view on programming language design, the advantages Dark has as an integrated solution, and also why it's source-available but not open-source. Big shout-out to our partners Linode, Fastly, and LaunchDarkly. Linode is our cloud of choice. Check them out at linode.com slash changelog. Our bandwidth is provided by Fastly. Learn more at fastly.com.

And our feature flags are powered by LaunchDarkly. Check them out at launchdarkly.com. Linode is simple, affordable, and accessible cloud computing the developers trust. Linode is our cloud of choice. We trust them, and we think you should build anything you're working on, a fun side project, or that next big info move at work with Linode. The best part, you can get started on Linode with $100 in free credit. Get all the details at linode.com slash changelog,

or text changelog to 474747 and get instant access to that $100 in free credit. Again, leno.com slash changelog. I'm joined by Paul Bigger. You're a founder of CircleCI, and now you're working on Dark, which is a combined language editor and infrastructure to make it easy to build backends. Paul, welcome to the ChangeLog.

Thanks so much for having me. Happy to have you. Well, tell us the story. Start us off from CircleCI to Dark. Yeah. Thanks so much for having me. and how difficult it is to add cloud into them. Looking at how teams build things and looking at how hard it is to build your own internal tools because every internal tool that you build is infrastructure that you now need to support and maintain. All these ideas were fermenting my head

around 2013, 14, 15. Then I left Circle in late 2015 and I was massively burnt out. Startups are hard. And made lots of mistakes as a first-time founder that led to more incredible burnout. So I took 18 months off. And then towards the end of that,

I started thinking, what is it that I'm going to do next? A lot of the things that I thought might be cool to do, a lot of startups that I thought might be fun to build, that kind of thing, they all had this infrastructure component. If I'm going to build a small company, a one-person company, a two-person, five-person company,

I don't want to spend all of my time dealing with AWS. That kind of thing. And so all these ideas just kept pointing me back to this other idea I had, which was Dark. And so it kind of got to the point where I just couldn't stop thinking about it. And so that's what I decided to do next. So you have these three aspects of Dark

that you say in the tagline, right? It's a language, it's an editor, and it's infrastructure as well. So building a language is hard, making editors an editor, and it's infrastructure as well. Building a language is hard, making editors is hard, it's also hard, but even conceptually, what's the win of combining these, usually disconnected aspects of coding?

It's kind of funny, we find it really hard to describe what Dark is. As you say, it's a program language, it's an editor, it's infrastructure. Soon it'll be a package manager as well. A lot of people have the why. And the why is what dark is not. So the thing that dark is not is it's not setting up Kubernetes. It's not setting up serverless.

It's not CICD pipelines. There's in fact a zero second deploy in dark. There's, in fact, a zero-second deploy in dark. There's no waiting for 15 minutes or whatever. There's no AWS. I'm just trying to think of all the other things that we have to do. There's no YAML files. Databases, queues, all that sort of thing.

It's all just built in, and so you don't have to think about, oh, where am I going to spin up this thing? How am I going to get the code from here to here? It's all just handled for you. The way that we're able to do this is by integrating all the things. Because if you say to someone who's using a standard AWS setup, it's like, we would like deploys to be zero seconds.

They're going to say, well, that's not possible. That makes no sense. We have to do tests. Maybe they do staging, or maybe there's an integration test, or there's a QA. There's something in the way to sanity check. You actually have to build in all this additional stuff to make sure that this thing actually works when you deploy it.

You can't do that with languages that aren't designed for it. I don't just mean languages here, but I mean the language, the editor, the infrastructure. It's all sort of designed for this purpose, and other languages and editors and infrastructure are not designed for this purpose, and so they don't have these features. So can you tease it apart? I know it's tightly integrated, but can you use Dark, the language, without using Dark, the editor? No, absolutely not.

It's all the whole kit and caboodle. Yeah, and that's the advantage. To give you a couple of the reasons that people like Dark, one of them is this thing that we call deployless. It's an instant deploy of any code that you write. And obviously that sounds dangerous. We perhaps have memories of writing PHP in 2001, where it just didn't work to edit and Vim

on these shared VPSs that we had at the time. So Dark is designed around this. It's like, okay, well, edits have to have atomic units. It has to be impossible to have syntax errors. And that's where the editor comes in. So the editor is the thing that enables that. But the editor, in most editors, you can have syntax errors. You can type whatever the hell you like.

So Dark's editor is a structured editor, which doesn't allow invalid syntax. And it significantly reduces the amount of times that you can introduce problems by typing. And so that's a place where you have an editor and you have a language and they enable the playlist thing and that just wouldn't be possible without having the editor, the infrastructure,

and the language being all part of the same thing. I was going to say, no syntax errors. You obviously never see me write code because I can find a way. But apparently the editor just says, no, that's just not going to... No syntax errors. You obviously never see me write code, because I can find a way. I can find a way. But apparently the editor just says, no, that's just not going to, well, you just can't hit the key, or how does it work?

It isn't text, right? Most programming languages, they're text files, and then there's a parser that parses it. In Dark, when you type in the editor, everything that you type is a valid part of the program. And so you might think, oh, what happens when, it's sort of easy enough, I put in double quotes and that's a string and then I type in the string and it's fine and you can't type outside the double quotes or whatever.

But when you think about a function call or something like that, what happens if I press backspace on the name of a function or something like that? We have this concept called partials, where while things are invalid, the old behavior remains so that you get an atomic switch from the old behavior to the new behavior. And then the major thing that we have to enable that sort of change is called, well, they're feature flags, so very common things. They're specifically integrated into the language to give you a new place to write new code

without affecting the old code. So basically, just lots of safety rails built into the language itself. So what is Dark's sweet spot? Is it web apps? Is it CLIs? Is it games? What typically would you build with Dark? I think the truth is today that there is no sweet spot. And this is something that I've come to realize over the last year that dark was not quite ready for anyone and

i think that that is still the the case today so really the you know kind of people who are using it today are people who are experimenting or they're building small tools or something where there's you know people often build slack bots um or or they build something that that integrates a couple of services. Where we intend to get to is that you will build everything in Dark. Everything can be built 10x easier with these things, but we still lack a lot of things that you actually need in order to be able to do it, such as things like user management and package managers and that sort of thing being built in the way that the Rails ecosystem has device and OmniAuth and all that sort of thing.

So you began it 2016-ish. It's 2021. It's in private beta. It's a long road and it sounds like you said it's still somewhat experimental. You're still trying to flesh things out. When you started Dark, did you know the long and windy road that was in front of you, or did you think it would be an easier thing to tackle? I didn't.

We officially started in about 2017, late 2017. It's been a little over three years now. We expected it to be much, much quicker. But what Dark was in the early days has changed significantly. In the early days, Dark was sort of a nodes and edges sort of thing, where you dragged things and nodes were functions and the edges were the results of those functions. And we were experimenting with lots of different ways to be able to write code.

And all that got discarded and went to things that looked a lot more like what people expect coding to look like. But we managed to keep the advantages of what we were looking for. That deploylessness, and I'll talk probably later about liveness or live values, which are this other amazing thing that you get from integrating those three things. There's been a couple of major inflection points. When we got our first user really building something,

when we got the first person to build their company on Dark, when we built a new editor and suddenly people were going from, I can't use this thing at all, to, oh yeah, I understand how to build in this. I think that the next major inflection point is when there's a package manager and people can instantly use third-party vendors and Stripe and Twilio and all that sort of thing directly from it.

But yeah, it's been a long road for sure. It's a platform that this journey will never be done, I suppose. Right. Yeah, but think about this concept concept recently because motivation is so perishable. And that old Thomas Edison saw invention is 1% inspiration, 99% perspiration. I just thought about how hard it is to persist and to persevere on an idea over time. The moment of conception is amazing. That's why so many people have domain names that they never use.

We tell our friends that sometimes that is enough to satisfy the idea and we never follow it. Sometimes we go a little further down the road and we hit troubles. Watching somebody persist through on a vision, we were just talking with

the founder of the Open Source Ecology Foundation on the last episode. And I mean, he's got this eight-year vision and he's 10 years into it. So like an 18-year vision. And I wonder with you, it sounds like these inflection points maybe help,

but how do you renew motivation and when it's like days go by, weeks, months, years, and you're still toiling away towards this end. Does it sound like maybe you celebrate some small wins along the way, or what? I think the main thing is that I take the motivation from users. We have about 7,000 people who have tried Dark,

and it's a small community that have stuck around on our Slack. There's 40 or 50 people whose names I know. There's a couple of people I talk to regularly. And there's new people coming in all the time and they have problems and things that they can't get working or ideas that they have. And those are the things that lead to motivating.

And often there's this sort of idea in software development that you should do the most important thing. You should always be doing the most important thing. What I've been finding over the last few months in particular has been that you should do the most motivating thing instead. I have a bunch of Post-it notes on my desk and it says, do work that's exciting

as opposed to necessarily doing work that's the most important thing. When a user tells me, oh, I really need this fix, and I'm like, that's a priority 12 bug that will never get done, sometimes I'll just fix it right there because it brings that sort of, the endorphins of doing a thing for another person, even if it's not the most important thing. I like that. In order to stay motivated, especially on a project like this,

do things that excite you because those by definition are motivating or you already are motivated. Work on what you're motivated about. It's almost like intelligent procrastination. It's like picking the right things in order not to burn out because you've been through the burnout before.

Are you familiar with that structured procrastination manifesto? No, I haven't heard of this. Tell me about it. It's a thing written by some college professor. I'll look it up so you can put it in the show notes. The idea is basically that whenever you have a really important thing, everything else looks a lot more exciting. So you use that most important thing to focus on things that you would not have done if they themselves were the most important thing.

So you have a deadline coming up, that's an excellent time to grade student papers. Or whatever else is. That's a thing that I have used since I was in grad school to actually get things done. It's like, I don't feel like that, but there's 12 other things that actually have value. And if I ignore the value and just sort of look at the excitement, I can probably get through them. friends at LaunchDarkly, feature management for the modern enterprise, power testing in production at any scale. Here's how it works. LaunchDarkly enables development teams and

operation teams to deploy code at any time, even if a feature isn't ready to be released to users. Wrapping code with feature flags gives you the safety to test new features and infrastructure in your production environments without impacting the wrong end users. When you're ready to release more widely, update the flag status and the changes are made instantaneously by the real time streaming architecture, eliminate risk, deliver value. Get started for free today at launch darkly.com again, launch darkly.com. so language design is something that always has fascinated me and here you are designing a language i like the way you said it's like more about what it's not than what it is and i feel

like programmers over time as we experience different languages, we know what we don't want, right? And we know what we don't want to handle and deal with, such as deployments and backups, perhaps, and all these things. But it has to have some paradigms for itself. So how would you describe Dark? Is it like an FP thing?

Is it object-oriented? How does the typing work? Give us the lay of the land, what design choices you made with Dark, the language. Yeah, well, I think the first thing I'll say is that Dark is incomplete. And so lots of what I'm about to describe are not necessarily fully there. Often there's some of it, but not all of it. So I'll describe the aspirational version. But it's an ML. So the specific subset of functioning languages that includes Elm, OCaml, Fsharp. Technically not Haskell, but actually it's similar enough.

And I guess Scala has some influences from that as well. If you're familiar with Elm, it's sort of Elm. And the reason that we chose that is, one, the liveness feature that I alluded to earlier, the idea that you can always see a value for the code that you're editing, works really well in immutable programming languages. So languages where the values are not updatable in the way that they are in object-oriented languages. And this is a common

feature in functional programming languages. And people have taken it other places as well. So there's a lot of frameworks for JavaScript that are immutable for example. And React and functional reactive programming, all sort of related to that world as well. So that was one reason to do it, because it fits in well with this liveness feature that we wanted. But the other reason to do it is that just immutability helps you think about programs. Because things never happen over there. They always have to happen in a place that's very obvious that it's happening. So that kind of is the major thing that I love about Dark.

And in every language that I've really used over the last decade, I used Clojure a lot at CircleCI. Dark is written in OCaml, it was written in Elm, it's now being rewritten in Fsharp. But all of those are immutable functional languages. And it's really, you know, I find it just the easiest way to actually write programs. Go ahead and unpack that live feature for everyone, because I think I'm picturing it, but I'm surely you can describe it with words better than I can. Yeah, how it works in my head. So in your editor, you've got this line that says,

let my variable equal name. When your cursor is over the word name in the text, at the left of the text, it shows you the string Paul. Because that was the request that was made a couple of minutes ago. We call them traces. When a request was made, a trace was recorded, and every value throughout the program is either recalculated or is stored. And so whenever you put your cursor over something, we show you the actual value that was there

recently. So if you ever open a bit of code that you haven't looked at in quite some time, or that someone else wrote or you're unfamiliar with. And you have no idea how it works. It's adding a couple of things, it's concatenating them, it's doing some substrings. It's like, why is it doing this?

And it's like, well, you put your cursor in it, you see a value. It's like, oh, I see exactly what they're doing. This only works on strings that has hyphens and it's removing the first four characters of it because that's what exists before the hyphen. It's like, in all these traces, I can see this. That kind of thing.

So you can think of it as a debugger where you never have to actually turn on a debugger, or you can think of it as printf debugging where you never have to add in the printf. It's just any time, at any point that you want to see a historical value, a current value, a test value, whatever, you can see it anywhere in the program with no instrumentation.

So does that allow a similar to, I think that Redux made popular, kind of the rewind debugging where you can step through different states? Or is it always just like the current? You said at any time. How do you manipulate that time? Well, because it's immutable,

there isn't really a time component. So in Redux and in the Redux debugger, really what they're showing you is the old state and the new state. Yeah, they're modifying the state. Those are actually independent values. It's actually a list of states and it's just showing you where they've been historically

because the new state never changed the old state. Those are different values. And that's the immutability of Redux. And it's exactly the same thing in Dark. If you've got an old value and you update it, I just did air quotes, which you can see in the Zoom, but your listeners won't be able to hear. Yeah, you had to verbalize your air quotes around here, please.

So yeah, if you have an old value and a new value and there's an update in between, you'll be able to see the value of the old value, you'll be able to see the value of the new value, you'll be able to see some intermediate values as well. So yeah, very similar to the Redux thing. If you have used Redux's debuggers and that sort of thing, and you understand why they are awesome, then you will understand why Duck is awesome. Okay, very well said. One thing I caught on

to there is you mentioned that it's had a few rewrites along the way. Elm, OCaml and now Fsharp. Can you share some of the insights into why the language switches over time? Yeah, so in the early days, we wrote it in technologies that made sense at the time.

So for the front end, it made sense to write it in Elm because I was familiar with Elm. And also because a lot of what we were doing in Dark was we were building our own thing. We weren't building a SaaS app. So reusing a lot of existing ecosystems wasn't as valuable as it was to have the Elm-like language. And after a couple of years,

Elm made a couple of changes. They removed some of the things we were using, and we realized that this is a little bit of a dead end. I wrote about that at the time. And I wrote a compiler to switch from Elm to what was then called BuckleScript,

which is now called Rescript. Oh, wow. Talk about working on what's exciting. You must have been like, I'm going to write a compiler today. Oh, yeah. I mean, Ian Connolly, who worked with Dark, you sort of made this joke that, you know,

the answer to everything is not another compiler. But honestly, for me, there's a bunch of... In this case, it was different. We've written so many compilers in... There you go. Stick to what you're was different. We've written so many compilers. There you go. Stick to what you're good at. One or two compilers a year basically.

And that's actually completely true. I just wrote a That's actually completely true. Yeah. Anyway. What was I talking

about? Oh yeah. Switching from Elm to Rescript or Buckle Script which I've never heard of either of these things so please do tell.

Have you heard of ReasonML? Yes I you heard of ReasonML? Yes, I have heard of ReasonML. Okay, so those are all the same thing. And they have a branding problem over there. But they've attempted to solve the branding problem by calling it Rescript. But somehow ReasonML remains a new thing

that is not the same thing as it was. It's very confusing. It's all Rescript now. So that was the front end. A new thing that is not the same thing as it was, it's very confusing. It's all re-script now. So that was the front end. And then the back end was originally written in OCaml. And OCaml, it's a really wonderful language with a lot of bad things in there that people just sort of ignore.

There's an entire object-oriented thing in there that people just don't talk about. Don't use that. Yeah. OCaml, the good parts. Yeah, yeah, for sure. And we've been using OCaml for a couple of years and there's very good reasons for using OCaml at the start. One is that it's super performance,

very high performance. Another is that it's immutable. So instead of having to write all these immutable libraries and so on, we just use the built-in stuff directly in the dark implementation. But the ecosystem isn't very large, and we often struggled with looking for encryption libraries or using vendor SDKs.

They just didn't exist in OCaml. So a couple of months ago, weighing up, what should I be doing next? Where is the focus? And recognizing that the states that Dark is at is not one where we have a lot of resources. And so it made a bit of sense at the time to actually make deep technical investments like that, like doing a rewrite. I thought we were going to switch to Rust. I was sure. We've been talking for years about the Rust rewrite. The tractor beam is strong.

Oh yeah. So I did I wrote a series of blog posts on it that were big on Hacker News for a couple of days. And I experimented with Rust and I did not like it. This does not map to how my brain works.

That's how to go to the front page of Hacker News. Yeah, exactly. And so we didn't pick Rust. Not because anything is wrong with Rust. Rust is very good for what it is good for and it turns out that dark is not in that sweet spot. You know what's an amazing truth that we often ignore is not every language is for every person.

Right, right, yeah, exactly. You were asking me earlier what dark is good for and I said, right. Yeah, exactly. I mean, you were asking me earlier what dark is good for, and I said, oh, people use dark for everything. But you're never going to use dark for embedded systems. You're never going to use it for the high performance, writing a decompressor or something like that. For everything means in the niche of general purpose,

cloud backend sort of things. Yeah, fair enough. And then, yeah, eventually settled on F Sharp. So we're about two months into the F Sharp rewrite and most of it is done. All the difficult questions have been answered. There's just a lot of fleshing out to be done. Did you also write about that choice?

I did write about it. Short version is, F Sharp is OCaml backed by.NET. So a huge ecosystem and the ability to reuse C Sharp things and high quality tooling by Microsoft and so on. But it's the same. If the drawback's for you with OCaml was the ecosystem

and F Sharp solved that problem plus it's a lot like OCaml then it sounds like a win. It's not even a lot like OCaml. These languages are twins. They are pretty much the same thing. Very cool. Well, we'll definitely link up to those deep dives

as I'm sure people who are interested in language choice, as we all are at certain times in our career, like when we're deciding to switch to a language, would love to hear the deep dive on your insights there. But keeping the focus on Darklang and not so much other langs, I think the one seminal moment for many languages is when they're written in themselves.

And it seems like dark isn't quite there yet. Is that a thing that you want to do eventually, dark written in dark? Or do you think it actually doesn't make a fit for itself? I've thought about that over time. What would it mean for dark to be written in dark? There's sort of an obvious one of the bootstrapping where in another language it might be the compiler can finally compile the compiler

and you can stop using the other compiler. And I thought about, okay, so for Dark that would be there's one service and it's hosting this other service and that actually doesn't make that much sense for Dark. It's just sort of like introduces risk and that kind of thing. And dark is fundamentally a hosted platform. And so introducing that sort of infrastructure risk

doesn't make that sense. What we're doing instead, we talked a lot about dark in dark and we do it in quite a lot of places. So the user flow is in dark and emails that are sent out are in dark. A lot of the collaboration stuff is written in dark. Some of the tracing is written in dark. I'm about to rewrite some of our static out are in dark. A lot of the collaboration stuff is written in dark. Some of the tracing is written in dark.

I'm about to rewrite some of our static asset stuff in dark. So anything that's like the core language itself and the core of the system is in dark. And if this breaks, we would not be able to recreate it is not written in dark. Sorry, I think I said this backwards. I think I said the core is written in dark and the core is not written in dark. Sorry, I think I said this backwards. I think I said the core is written in dark and the core is written in not dark.

My brain parsed it the correct way. I'm with you. I think that's wise because any language or ecosystem that wants to be general purpose and relied upon by many people, it has to live in the real world, I think, at a certain point somewhat early on in its life so that it's rooted in reality because we can you know tend towards what the maybe

more critical than myself would call you know architecture astronauts this idea of just like there's a lot of languages that have a purism and an idealism, which is beautiful, but they're not going to be general purpose because the real world is corner cases and yucky and things need to work. And I think having the team working on Dark, using Dark for its purpose, which is like these kinds of projects,

maybe it's not written to write languages, but it's written to do these things and you're actually using it for those things. I think it gives you an insight which you may otherwise lack. I remember at the start, before Dark was really available, a lot of the people who were interested in Dark were the people who were really into programming languages. So I fielded this question of,

can you compile dark? Can you write dark in itself? Why aren't you writing dark in itself? And there's almost dogma. But fundamentally, languages are built for a purpose, and the thing that our language is doing is not building programming languages. Right.

So a complete sidebar, but I'm curious. How did you learn to write programming languages and compilers and these low-level things? Where did you acquire that skill? Third and fourth year compiler class in college. Okay, like straight up university. Straight up, yeah. I went to Trinity College Dublin,

and I did an undergrad in computer science, and then I did a PhD in compilers. Okay. So I kind of joke about this because i hate parsing i think parsing is like the worst thing so the first thing that you that you do when anyone writes uh writes a language is write to parser so i rarely write new languages or anything like that because i hate parsing so now you got me wondering i'm not going to ask this maybe we'll bring you on the founders talk and Adam can talk to you about your CircleCI days. How does a guy who's into compilers and languages

and studies these things ends up founding a CI company? For a different day, let's focus on Dark for now. Another aspect of the language, I guess the integrated aspect of the language, is that it has built-in support for things that normally you go to a community for. Background workers, scheduled jobs, there's data storage and stuff like that. Are these language-level primitives? Are they part of some sort of dark standard library?

Is there such a thing? How does that work? They are fully integrated. I'm not really sure what they are exactly. Certainly databases are language primitives. The important thing is that what you store in a database is actual dark data. There is no ORM,

there's no conversion layer between the runtime representation and conversion into SQL or Postgres types or something along those lines. Now you have me interested because one of the banes of my existence is marshaling and unmarshaling data

in between a data store and my language I'm writing it in. I just want it to be there when I come back. I just want to have this, I just want to keep it, and I just want to know it's going to be there safe. And I actually couldn't be bothered with the details of the ins and outs of the database. And so that's something that's very attractive to me as a developer. Yeah, I mean, it was a goal that it would be fully integrated

into the language, or at least into the dark system, or platform, whatever we call that. And so it had to be that these values could go straight into it. But also, the way that you query things in dark is by writing dark code. You don't write SQL. You write something like a query function where you pass into a Lambda

and then the Lambda is actually compiled into Dark. That was last year's compiler project. So what if you do want to bring your own things to Dark? It sounds like it's probably still too early on. No, you can't use Postgres with Dark. In the future, someone might write a Postgres adapter, driver, whatever. Yeah, so all you would need is some sort of low-level networking

Dark thing, and then you could just write a network adapter. Yeah, and having direct TCP or something is something that will come. But the goal of Dark is not to let you use whatever you're using today. The goal of Dark is, let's assume that that's all legacy tech and we want to get rid of a ton today, unconstrained by legacy.

Is there a day in the future where enterprises will use the Dark consultants to hook up their Redshift to dark to be able to do queries in it. Maybe, but it's never going to be the core of what dark is about and bringing your own stuff to it. It's always going to be about this idea of we can build stuff faster if we have it fully integrated in our system. to sell a global CDN, private networks, and auto-deploys from Git. They handle everything from simple static sites to complex applications with dozens

of microservices. If you're a developer or a founder that's frustrated with AWS's complexity or Heroku's high costs, you owe it to yourself to use the $100 in free credits they're giving our listeners to give Render a try. Render is built for modern applications and offers everything you need out of the box. One-click scaling, zero downtime deploys, built-in SSL, private networking, managed databases, secrets and configuration management,

persistent block storage, and infrastructure as code. The Roku customers running production and staging workloads typically see cost reductions of over 50% after switching to Render. Here's the best part. We work closely with the team at Render to ensure you have zero risk. By giving you $100 in free credits, plus they're going to assign a world-class engineer to your account to offer guidance and answer any questions you have. When you're ready to transition your infrastructure, they'll be there to help you with that too. Automate your cloud hosting with Render at

render.com slash changelog. Get $100 in free credits to try the Render platform, plus a I'll see you next time. so one thing i haven't mentioned yet is that i have you on today paul because a listener requested that we do this show oh cool yeah so shout out to john Stoddle, and he requested that we do a show on Darkling and your open source strategy. Well, thank you so much, John. Yeah, absolutely. We love listener requests, because now that we know we're doing a show that at least one person absolutely wants to hear.

Someone's going to be happy. It's going to be a non-zero happy listener count. And one thing that John said, he says you have an interesting take on an SPA slash PWA with functions as a backend. And then you also have a special take on open source, in the air quotes. Yeah, air quotes are much needed. Because it's source available licensing, yeah, but not open source.

So that kind of opens us up into Darkling's ecosystem. There's a business around it. It's not open source, but you can view the source code. Talk to us about that whole situation. Yeah, so where to start with it? I hate fake open source. I'm really into business models. I love thinking about business models.

And the worst business models are the open source business models. And we're a couple of days after AWS just forked. Yes, Elastic Relicense. That's right, that's right. Elasticsearch, and AWS just forked. Yes, Elastic Relicense. That's right, that's right. Elasticsearch and then AWS forked. Right. Previously, yes. It's a big conversation piece.

There's all these companies that are really struggling with the fact that they built open source businesses. And for a lot of things, especially for databases, you kind of have to be open source. It doesn't make sense to be, and Elasticsearch is a database, really. But you kind of have to be open source. It doesn't make sense to be, and Elasticsearch is a database, really. But you kind of have to be, because people want to run their own data and that sort of thing. And when you think about what is dark, what is the purpose of dark?

The purpose of dark is that you don't have to do anything. And so the first thing that you do with an open source thing is that you install it. You find a system, or maybe you put it on your first computer. And that's not what Dark is. That's not what we're here for. We're kind of in this world where you want a business model that matches to how you want people to use the system. And so for us, open source was not it.

It is not a thing that made direct sense because we have that AWS problem. If Dark manages to succeed, do we want to just be run by AWS? No. But also, we've been talking for 45 minutes now about this integrated ecosystem. And part of that integrated ecosystem is,

yeah, we're the package manager, we're the hosting. It doesn't make sense to have another instance of dark over there. There's a bunch of things I'm planning to build where I wouldn't call it social features, but something along the lines of you want to make a bug report to the package manager, maintainer, whatever. That's a thing that you do in dark and you can take your request and upload that or send or something along those lines. It hasn't been designed yet.

All things that are part of this singular ecosystem. So it doesn't make sense for someone else to have another installation of Dark or having their own installation of Dark. And when we look at people asking us for open source, there's a couple of things that they want. Some people are into open source for status. They want to build their reputation in order to do consulting or something along those lines.

So that's a thing that we wanted to enable. People want to install it themselves to have on-prem and that's not a thing we wanted to enable. We recognize that there are people who cannot use cloud things and that's not a thing we wanted to enable. We recognize that there are people who cannot use cloud things, and that's okay. Not every service is for every person. Then there's people who are worried about business continuity.

What happens if Dark shuts down? That is a thing that we wanted to take care of. Then there's people who just want to be able to extend their programs. They're users of Dark, and they want to be able to make it better. And so of those four things, three of them we wanted to enable. So there's a couple of things that we did. So one is, we've not done this yet, but intend to.

It's sort of like an open source guarantee, where if Dark shuts down, we'll open source it. That's for the business continuity side of things. We have not actually done the legal side of that. What you have today is, well, it's source available. If there's no business left, no one is going to sue you for installing your own thing, but we should do better than that. With the source available, you get the people

who are trying to build up a reputation for being experts in dark, can contribute to the community there. The people who want to extend the ecosystem and who are unable to do a certain thing in dark and so they want to make it so that they're able to do a thing in dark. That is really what we were looking for. And you cannot run your company on this thing,

or on your own instance of this. So we got a lawyer called Heather Meeker, who's a famous open source lawyer, and she wrote the Polyform licenses and a bunch of other things. So we got her to write a license for us, and the license basically says, this is source available, it is not open source, and you can use this for the sole purpose of developing

and contributing back to Dark. And then we have a CLA as well for contributions. I like how clear you are about that and straightforward. And you said you hate fake open source, and you're not trying to be open source, you're source available, you say it right there up front. I think that's admirable. The but is coming. Well, I mean, the but is not a big but.

Do you think that's going to hamper it? Oh yeah, absolutely. Because there's so many people that just see that and they're like, I'm out. I think that the problem with having nuanced opinions about anything is that looking for some middle ground is that a lot of people just aren't interested in the middle ground. I'm not saying that's bad. I think having an open source first philosophy isn't a bad thing.

But people who build stuff in the cloud are not using open source the whole way down. They're relying on AWS, none of which is open source. Even AWS's Elasticsearch, the fork of it. There's AWS proprietary stuff all over that. The people who are looking to build the sort of thing that we want you to build on Dark, I think for the most part are going to be very familiar with this idea that actually they don't have a license to everything

top to bottom. Fair enough. I think in light of that, languages have communities. Languages are not, if you think of a language that's owned by a single person or a single entity, even Microsoft has released the death grip on a lot of their proprietary software and languages.

Fsharp is open source. Right, of Apple famously open sourced Swift and they said they were going to do it the entire time and they just didn't do it right away. I think that's probably true or they wanted to do it the entire time. At least, I can't remember the language designer's name offhand but I know the core team definitely wanted Swift

to be open sourced. As Apple knew they needed to open source it, I'm now, this is my thoughts, because they wanted mass adoption and education and all these places on the server side where it wasn't inside of Xcode. How do you overcome the community problem? Because you have some people hanging out,

you got people extending, but do you have a strategy or are you just hoping that it works out? Just curious your thoughts on, since you are just source available, how do you get the real, I don't want to call them zealots, but the fans, the dark people? Yeah, so the thing that we recognized up front is that we want the dark fans and we don't necessarily want the open source fans. So if dark to you is a cool thing that I can hack on, it's not really that interesting because you're going to hack on the wrong things. We get a lot of feature requests from people on their first day

where they would not make the same feature request a month later once they actually understand the stuff a lot better. The purpose of going this source available thing was for people within the community to contribute back and to be able to contribute back to solve their own problems. It's sort of like a multifaceted approach because there's the dark code base, but there's also going to be a package manager

that people can contribute to. And that's also going to be in a non-open source license. That has a lot of the same sort of available features. For a bunch of reasons, we don't actually want anyone who can't get past the zealotry side of things. I don't mean to, like, zealotry sounds so negative, and I think that there's so many positive things about open source, but there's so many negative things about open source. People are talking about the licensing thing, but there's also like, you know, a bit of a toxic community

that has existed in open source for a while that, you know, people are, you talk about a little bit at the edges, there's that, you know, but you can see on certain GitHub issues where people are demanding things from maintainers and et cetera, et cetera. And so we're a little bit distancing ourselves from that and being like, this is a service. The people in the community abide by a code of conduct and you've got to be, amongst other things, you've got to be nice and inclusive.

And if you're in the community, we're going to have that conversation. You can participate in building Dark and extending it in ways that you want. But we kind of want to be a little bit further away from the things that are happening in open source. What's interesting is that your initial audience, I'm just thinking of the diagram

between language nerds, and I say that in the kindest way, and I consider myself somewhat one of them and then like open source nerds and then like the cross-section of those two and i feel like they're like 80 overlap well our target is not language nerds oh i know i'm saying your initial like you build your community out of your your early adopters right and you said that the people that are coming on early are like, hey, here's a cool new language. I'm interested in this.

There were, but most of our community now is people who build web stuff. The language nerds, they don't use Dark. They're interested in Dark in an abstract sense. Right, like they're watching you build it maybe. Yeah, but the people who use dark are people who build SaaS and web backends and web frontends and don't build backends.

Use dark for that. I'm not going to say that there hasn't been people who have dropped out of contributing because it isn't fully open source. I think that's fine. Everyone makes the trade-offs according to their values and we want to have a sustainable ecosystem. And a sustainable thing is focusing on what dark is

and focusing on the long term where the value of dark gets reinvested into this ecosystem. Yeah. Well, like I said, I applaud you for staking your claim. I'm with you. The only thing that I like open source, I'm also okay with proprietary,

and I just want the lines to be drawn and clear. And what I do not like is when you have people who are trying to ride the coattails of open source and benefit from the great goodwill that the term and the community has. There is some toxicity, but there's a lot of value there. There's tons of marketing value,

just to call yourself that, without actually being that. I think we're on the same page there. Let's talk about what it feels like then. Let's put me in your core user demographic. I'm building my startup on Dark. And I have a SaaS startup that I'm going to build. What should I expect in terms of business model?

How much am I going to pay you? How does it scale? I have no problem paying for things that bring me value. Where I have a problem is usually when their model doesn't scale alongside the way that I use it, for example. GitHub used to famously have this mismatch with agencies where I was a freelancer and I would like add new projects all the time because that's the way that I would move on. Oh yeah, I remember that. But they would charge based on project count and it's like I'm still just a one person company.

It's not scaling with my company. And so I'm curious, I know you're not there yet, you're in private beta, but you love thinking about business models. Surely you have some sort of a game plan, like what does Dark, the business model, look like for its users down the road? Yes, I mean, the belief that I have about developer tools is that people used to not pay for developer tools.

This was a truism up until about 2015 or so. And before that, those of us who monetized developer tools, such as CircleCI, did it by reselling infrastructure. If you think of AWS as a developer tool, it's actually we're selling you computers, and you're paying for the computers. And CircleCI is the same, you're paying for the computer. That makes sense to a certain extent in Dark as well.

You'll pay for the infrastructure that you use. We have described Dark, and Dark isn't there, but conceptually Dark is an infrastructure compiler. It's a thing which takes the code that you write and figures out how to run it. And we will charge you a reasonable upsell on the actual cloud costs of the thing there. However, a thing that I've been thinking about recently

is that the vast, vast majority of our users today and for the next few years are people who are not using it very much. They're not using it an amount that really costs that much money. So if you think about a typical SaaS, a typical SaaS is a workflow engine that has few users and not a lot of data.

And it's not Twitter, it's not Snapchat. And so if you're charging people based on the costs, there's not a lot of money for the size of businesses that Dark is going to be able to support in the next few years. So I think that long term, that is a good business model. And short term, I think we're going to need something else. And the thing that I'm thinking is something like the per user thing for companies.

Because if you're spending $10,000 a month on an engineer, you're not going to balk at $25 a month to make that engineer three times as productive. But we have to actually have that productivity. We have to have people actually using it and we have to have, there will always be a freemium component because if there isn't an open source component

to it, we need people to be able to get in. You've got to be able to try before you buy. Exactly. You can't just be like, hey, by the way, we're going to start or switch on this new language we never used, on a new editor we never used, on an infrastructure we never tested. Where do I put my credit card in?

I think that most people who like TriDark will end up at a free forever sort of thing. They won't be using enough traffic to hit costs. They will be writing in public and in the open and packages and that sort of thing and that will be free for the amount that they're using. And then at some point corporations

are going to pay something. And you said you don't mind paying for tools. Most companies have no problem at all paying for tools and actually prefer to pay for tools so that that will definitely be where the pricing goes. I remember James Lindenbaum,

who's the Heroku founder, has this concept of don't put anything that's developer-facing behind a paywall. So you can put the business-facing things behind a paywall, but not the developer-facing things. And that's an approach that I really like and that I intend to stick to. I like that idea.

So what does success look like for Dark? If you teleport yourself into the indefinite future and I did it, Dark wins or Dark is successful, what does that future look like? That's kind of interesting because sort of at an inflection point now, we haven't mentioned it yet on this show, but there was a layoff in July and Dark is just me right now. And so I'm having a think about what is the future of Dark?

What is the future of this company? And obviously the original thing that we sold to investors was this is going to be a billion dollar company. It's going to be amazing. All cloud will be built in Dark. And when you combine the fact that it's been three years and we're not yet a product market fit with the world that we live in these days,

a lot of what I'm thinking is maybe the billion dollar business is not the most interesting or valuable thing here. And raising the huge round to do the next thing, it's like, eh. I think I don't know what the future looks like. I think I need to sort out my own head for where I want the future to be. But something that I still love the idea, or the main thing that I love is

what you can do with Dark. And whether Dark is going to be a thing where you build apps for tens of thousands of people or for tens of millions of people, I think those are definite choices to be made. But we will definitely be progressing towards this hosted system where you can build small to medium apps at the very least. I have to think about whether I want to support more than that.

There you go. Well, I appreciate the honesty around the size of the team and all that. I didn't realize it was just down to you, but hey, the pandemic hit a lot of us in different ways. And like you said, we are living in difficult times. So happy to see it continue with you. Do you have like a, I know there's a public roadmap,

but like you said, you have some thinking to do, but do you have like a, I'm going to work on it for this long, or like I said, the vision, or how long are you in the game with Dark? My thinking is I might be in it forever. I'm going to work on it for this long, or like I said, the vision, or how long are you in the game with dark? My thinking is, is I might be in it forever. I'm not a hundred percent sure. Yeah.

Because startups, you tend to be in like, you know, I put in, you know, everything I have for four years or 10 years or whatever. Right. And then you like burn out and, you know, have a midlife crisis. You've already played that game though. I've played that game. Yeah. And, and, and that game went well.

And so now I'm in sort of the post game. Yeah. And Dark is my, you know, to a certain extent, my post game. Yeah. So, you know, Dark is a, you know, it's a code base I know very well. I love coding. There's a lot of, you know, there's something in me that's like, you know, I can build Dark forever and that would be wonderful.

Yeah. And, you know, there's another part of me that's like, build big startup success, et cetera. And those two are a little bit in conflict, right? It's like the devil on each shoulder. And that's why I was saying earlier, I don't know which one I'm going to go. Which devil is going to win? Well, let me just sit on one of your shoulders

and just reiterate what you said to me, is work on what's most exciting to you. That's where you are, you're in a good spot, you can do that, you can afford to do that. So if you focus on that versus having to decide go big or go home, it's like, well, wake up each day, work on what's exciting. And that will be at least exciting, at the very least enjoyable. The situation that we're in as a company, the company's separate from the product

or the code. We haven't got to product market fit yet. We have a million in the bank. We spent about four million to get here. And we basically have to get to product market fit on that million. And we're spending nothing at the moment.

It'll last a very long time. So then there's a question of what are we going to do? What is the best way to get to a place where there is optionality, where we can start to think about, when we get to product market fit, does Dark start to take off? Are we excited by that takeoff?

Then that kind of points in one direction. On the other hand, it's possible that when we get to product market fit, Dark will still be a thing for small people to build things with a small team, and that'll be another reality that we have to deal with. And when we have that information, I think it'll be clearer. That sort of information about what options you have

tends to lend a lot of influence to the option that you take. Well, for the tinkers out there, the experimental folk, the ones who do not like to marshal and unmarshall databases, want to try out Dark. It looks like it's in private beta at the moment. It's in public beta at the moment.

It's in public beta now. You can just go to the website, darklang.com, sign up, try it out. Give it a shot, hop in the, it looks like you have a community that you can hop into, ask questions, source available, so you can do your habitual GitHub complaining slash requesting of things, like you would with an open source project. There are GitHub issues, and pull requests accepted.

There you go. Paul, anything else I didn't ask you or you've just been waiting to say this or ask that before we call it a show? Nothing comes to mind. Okay. Thanks again to John for requesting this episode. Yeah, thank you, John.

This was wonderful. This was lots of fun. I'm fascinated. I would love to see you succeed and we'll definitely be keeping our eye on Dark. And we encourage all the listeners to go out there, give it a look it has a lot of

interesting properties and something at the very least you want to try out and see if you like having a fully integrated solution to coding up things

so Paul thanks for coming on the show thanks so much Jared we'll talk to everybody next time it was so cool to finally get Paul Bigger

on a full length episode you may have heard Paul in previous ads we've done for Rollbar love those ads It was so cool to finally get Paul Bigger on a full-length episode. You may have heard Paul in previous ads we've done for Rollbar. Love those ads. Love Rollbar. And that was fun. But a full-length episode with Paul, even better. If you haven't heard yet, we have a membership. It's called ChangeLog++ because, hey, why not increment things?

It is better, as they say. You can subscribe at ChangeLog.com slash plus plus. Get closer to the metal. Make the ads disappear. And, of course, support all of our podcasts. Again, changelog.com slash plus plus. And, of course, huge thanks to our partners, Linode, Fastly, and LaunchDarkly. Also, thanks to Breakmaster Cylinder for making all of our awesome beats.

And, of course, thanks to you for listening. We appreciate your attention. We appreciate you listening. And one more step you could take is to join the community. changelog.com slash community. It's free to join. Come hang with us in Slack. Call this place your home. changelog.com slash community. That's it for this week. We'll see you next week.