The Changelog: Software Development, Open Source - From Docker to Dagger (Interview)

Episode Date: July 28, 2023

This week we're joined by Solomon Hykes, the creator of Docker. Now he's back with his next big thing called Dagger — CI/CD as code that runs anywhere. We're users of Dagger so check out our codebase if you want to see how it works. On today's show Solomon takes us back to the days of Docker, what it was like on that 10 year journey, his transition from Docker to Dagger, Dagger's community-led growth model, their focus on open source and community, how it works, and even a cameo from Kelsey Hightower to explain how Dagger works.

this week on the changelog we're joined by solomon hikes solomon was the original creator of docker and now he's back with his next big thing called dagger by the way we are users of dagger we use it in production so check out our code base if you want to see how it works on today's show. Solomon takes us back to the Docker days, what it was like to be on that 10 year journey, his transition from Docker to his next big thing, Dagger. He also shares his path to get to today with Dagger, this community led growth model they're running, their extreme focus on open source and community. Of course, we talk about how Dagger works, and we also bring in Kelsey Hightower to help us explain things.

A massive thank you to our friends and our partners at Fastly and Fly. This podcast got to you fast because Fastly is super fast all over the world. Check them out at fastly.com and our good friends at Fly help us put our app and our database globally close to our users. And they'll help you too with no ops. Check them out at Fly.io.

Well, I'm here with Richard Moot, the API design lead for all of Square. And we're talking about the GraphQL API that is now in OpenAlpha looking for feedback. So Richard, what's the story with this API? So we've announced this at Unbox last year, and we've been just incrementally adding parts to our GraphQL API. It's been a big ask from developers within our community because it makes using Square's platform so much easier for particular things. You're no longer having to, let's say, call like three or four different APIs to like pull together, you know, a bunch of different data. And so we've just been trying to learn more and more like how developers are planning on using this and making sure that we get this right

before we actually transition to the next phase and its release. So you have the orders API out there, the catalog API, the customers API, the merchants API, the payments API, the refunds API, and the inventory API out there. And you also have the GraphQL Explorer out there tell me what are you expecting from developers

what feedback do you want what are your expectations? I think our expectations is to find out all the different ways that you're using it and that we can make it better for you I mean right now you know we've gotten really good feedback we have I mean as soon as I announced

the update to our docs that we recently did, the very first question that I got on Twitter from someone was like, when is this going out of alpha? And so we're really happy to see that. But we also are still wanting to hear from developers. Like, you know, you're implementing this, you're trying to build something. What is causing you angst? Like, what is, why are you, is it issues with like constraints around query depths

or a number of queries? Is it fast enough for you? Are you trying to use it in a particular mobile app, electron app or something? And like, you know, what what issues are you kind of coming across? And like, what, how can we make it better? And I would definitely say that, like, anything that you come across when you come in, you try it out, whether it's in the GraphQL Explorer, in your command line, in your app. We want you to reach out to us on our Slack or our forums. Those would be great. You can also tweet at us.

I will definitely be keeping an eye on that. But I will probably still always say like, hey, like the forums are a great resource because we have a lot of questions that are already asked there. And we really just want to like funnel all that feedback to the team so that we can get this into there in time to make this ready for the next phase. Very cool. Okay, so if you want to check this API out yourself, go to developer.squareup.com Again, developer.squareup.com It is an open alpha. They're looking for feedback. Hit them up on Slack. Head to the

forums. Whatever works for you. Once again, developer.squareup.com. How many game-changing ideas can one person actually have? Do you know that number? Is it two? Is it number? Is it two? Is it four? Is it eight? Where should we begin? I don't know.

I have no idea. I ask myself that question a lot. You know? Yeah. Yeah. I don't know. No idea. You think you got another one on your hands here?

I mean, this is your second act. Oh, yeah. Yeah, totally. So at least two, right? Well, I feel like, you know, actually, I feel like groundbreaking ideas, you can have a lot of, the question is, what can you do with it? You know, how much impact can you have with that idea? Like, is the world ready for more than one of your ideas, however groundbreaking you think they are?

Yeah. You know, so I got ideas. There's no problem problem there what to do with it we'll see we'll see the last time we talked to you at length was in your docker days I believe right I mean well you've been on ship but you've shared some of your dagger story there we're gonna act

as if some of that didn't happen not all of it so definitely go listen that we'll link it up in the show notes but Jared and I want to spin our own version of that here and hear the story arc essentially from changing the way DevOps works with Docker to now attempting to do it

again or a version of it again with Dagger. I mean, how do you feel about Docker these days? What's your stance on it? Are you still rooting for them? Where are you at? Yeah, totally rooting for Docker all the way. I mean, you know, there was a period of time where I think everyone was worried that it might not make it as a company uh i was i mean i and of course i left but now i mean it's doing well again and uh i think for the right reasons like it's now

finally docker is finally focusing on the stuff that matters and now you know kind of rebuilding from there which is exactly what what they should be doing. Have you ever shared why you left or versions of why you left or the public version of why you left? Well, yeah, there's a public, I've read a blog post. It's not that different from the private version. I mean, it's complicated, but the short version that goes, I think, closest to the heart of the matter is I was very tired and it, it was 10 years in for me and a lot happened in those 10 years. I went through a lot, you know, personally and professionally and, um, yeah, you know, whatever. There was a, I'm sure that something happened, a sequence of events happened

that were the last straw at that particular time. But really it was just, I was just tired, ready to take a break, you know? Yeah. Can you speak to the struggle of that, like the 10-year journey and just the game-changing, like I said, how many game-changing ideas can one person actually have? That's actually a line from Silicon Valley, which I love. Which I did not catch. But it's true, you know? And so that was a game-changing idea. Docker very much changed pretty much everything for deploying applications.

You know, it's the way, you know it's it's the way you know and it still is the way despite the company you know struggles and now non-struggle from a technology standpoint product market fit immediately with developers right for sure and then only since then it's just only been iterations on you know better developer experience you know, better developer experience, you know, et cetera, CPU changes across different development environments, all that good stuff. But can you speak to the struggle of that 10 year journey? Yeah, well, you know, it's funny, because I mean, I started out just a complete outsider, it's hard to hard to describe just how much of an outsider I was. I mean, I graduated from, you know, programming school, engineering school

in France in 2006. And I was obsessed with these kinds of things. I mean, I was already on a track to containers, you know, I think 2007 is when I started really messing with actual container tech and patching the kernel and just kind of getting excited about the possibilities in my young, naive way, you know, not knowing that it was impossible, therefore doing it, you know, that kind of thing. But it was just really, that was the beginning of the sprint that little did I know would last 10 years nonstop, you know, first five years of just crossing the desert, like obsession with containers and the possibilities and the ramifications and literally nobody cares. No one cared, you know, and then immediately,

like you said like something changed and everybody cared too much almost and then it was another five years of just keeping up with that i mean that the common thread was just obsessive focus on that tech that uh you know just had a gut feeling was important but couldn't articulate exactly why or how because part of it because it was very new but also part of it again because it was very new, but also part of it, again, such an outsider, both me and the founding team that I worked with, that we didn't have the tools to even express in what way exactly it was different or exciting. For us, it was just all new, all unknown. We didn't know which parts were new for everyone and which parts were new only to us because we were noobs. It was all just a big blur. That came out of.cloud, didn't it?

I'm trying to get my ancient history back out again. So when you said, you know, great ideas, wasn't like, I thought maybe you're talking about app.net. Wasn't app.net ADN? Was that a thing? Is that a.cloud thing? No, app.net was not me. Okay.

.cloud was me. App.net was Dalton Caldwell. Yeah, Dalton. Yeah, exactly. Yeah. He's now, I believe, a partner at YC? Yes. In fact, he was our partner for the YC batch in 2019 that Dagger went through.

Okay, so why am I associating those two things together? Just the dot, maybe. I thought that was all one big thing. Well, the dot and dot cloud, well, that name was inspired by.NET. The general aspiration was, dot cloud basically was the name of the company

before we renamed it Docker. And, you know, we went through a bunch of different products, but it was all container stuff. But the aspiration behind the name, the potential was.NET at the time, for me, it symbolized sort of a successful new standard, you know, a new layer in the stack that abstracted away something that

it was useful to abstract away. In the case of.NET, it was like all these different architectures, I guess, you know, you just, you wrote your code out of.NET APIs, and then your thing could just, you know, was floating on this portable platform, you know, it was abstracted away from the messy details, you know, and it was a new, more powerful programming model, yada, yada, yada. So in my mind, with containers, there was the potential to do the same thing, but for the cloud, you know, like abstract away all the messiness, insert a new layer that actually helps, you know, a real abstraction layer, not a terrible one, which is hard. You know, it's rare that there's a lot of terrible abstraction layers. Every once in a while, there's a terrible one, which is hard. You know, it's rare that there's a lot of terrible abstraction layers.

Every once in a while, there's a useful one. And it felt like, okay, containers could be that, you know. And that's really the core of when people say containers already existed before Docker, blah, blah, blah. Yeah, so that's the thing that did not exist. No one was thinking of them as an additional layer. It was a spectrum of virtualization. You had the heavyweight VMs,

and then you had the lightweight VMs. But everyone thought of them as you choose one. It's a trade-off, but it's still like a machine, virtualized versus admins, and then the application people do things on top, and that's it. And now we can't imagine the world otherwise, right? I couldn't imagine running an application on a disparate version of Linux Picker Dist, and having multiple applications run at the same time and trying to manage those two environments simultaneously. It's just like, why would you do that? You don't have to, obviously.

And for a while there, we did. We would actually literally either have a VM and then place our application on that, like a virtualized machine, and have different machines with different IP addresses and then working together. You still have networking, but then you have it at the Docker layer, a shared mesh network. And you can even do that in Docker Compose and create networks. It's like, you know, that's how it should have been. So thank you for those five long, hard years and really the 10-year journey.

Because, I mean, if you didn't have that curiosity and you weren't willing to chase it, we wouldn't have Docker today. So thank you. Well, I appreciate it. I feel like these good ideas, when they're ripe, you know, someone would have figured it out in a different way. You know, we were getting there.

We were really close. In fact, you know what? The core Docker community, when that, I think one of the sparks when we open sourced it and we started shopping this idea of open sourcing it, you know, because that's what the pivot was. .cloud was basically a Heroku competitor. It was same container tech, but bundled it to this big monolithic platform they had to sign up for.

And we had customers, but, you know, there was a lot of hoops to jump through. And then we started talking to a few hardcore container people, people who got it, you know, they were already developing their own container-based thing. So that was kind of a really tight-knit group of people, people who worked at Heroku and other places, started showing them this prototype of just spinning out the core container tech in a way that you could use it yourself and build stuff yourself. And that core community, that was, it was all people who were following some sort of a parallel track, you know? So for sure, let's say I'm not there, you know, the dot cloud crew is not there, then one of them would have figured it out. But what happened is we figured it out together anyway,

because they joined the Docker community and they, we built Docker together. Some of them actually shut down their own kind of early prototypes of something that looked kind of similar. Others were thinking about it and never got around to it. And then there was the repo, you know, I'm going to go join this crew. So it was like a motley crew of open source people. And that was really when it started becoming an emerging standard, right? Because the whole point of Docker is that you've got to standardize it.

You've got to get everyone to agree to use it. Otherwise, it's pointless. It could be technically brilliant. It doesn't matter. So getting a lot of people to adopt it is part of the design process. And so when those first, what, 20 people joined that repo and that little IRC channel at the time, you know, that was really, that was the key feature that was missing.

I think, I mean, one of the key features. There's a phenomenon I think is called the multiple discovery, I think, or simultaneous invention. Right. And you're kind of speaking to it and that's not necessarily what happened in your case, but you're saying like somebody else probably would have discovered this, you know? It was definitely, you could tell it was in the air. You could tell. It was definitely, you could tell it was in the air. You could tell.

It was bubbling. Yeah. Yeah. It's kind of a cool thing where like independent groups at the same sort of general time are all kind of in the same thought space and tend to land on either side of a discovery somewhat close to each other. So yes, we may have had something like Docker had you not done what you did, but nonetheless, you did what you did and now it's here and we're all much better off for it, right?

So that's why we say thank you. He just took all that credit right back, Solomon. Jeez. I just gave you all this credit and he's like just taking it right back. No, no, I'll take the credit. But it's really, it's an interesting topic because it's tied to why Docker actually succeeded.

We identified that and captured it too. You know that, okay, there's a community and the word community is really important for Docker. And what you just said, that phenomenon, that led to the kernel of a community, basically. Right. And isn't that kind of like lightning in a bottle to a certain extent?

I mean, I guess you're discovering if that's true or not now with Dagger because you kind of want to do the same thing, at least with the community again. You want to do that again, but in a different area. Yeah. Can you just repeat? Can you put it on repeat? Like you can your favorite song. That's something I think about a lot. And we talk about a lot with my co-founders, right. Who, by the way, were part of the founding team. You know, they were,

they were the earliest employees at dot cloud and lived through the pivot to Docker with me. And then, you know, so we shared a a bit of that journey. Yeah. So we talked about that a lot, you know. A lot of time together. I mean, all things being equal, would we love to go through the same magical moment where the whole world decides our thing is the thing? Yeah. Who wouldn't? Do we want to build, you know, if you're building a new thing, do you want to make that a condition of your success? No. Right. You can't rely on it. So you don't want it to be a condition. Can't rely on it. Yeah. I mean, there's a whole topic. I mean, I'm happy to go into it, but our process at Dagger of sort of managing that and setting expectations and just finding our

own path, but also not discarding all these useful lessons from, you know, just the most defining professional experience of our lifetime. You know. It just kind of defined us professionally. There's so much to learn, so you want to mine that, but you also want to forge your own path. So that's actually, we've started out going in a very different direction, and now we're actually, ironically, getting pulled into a community-led path more and more. It's just at some point some point you know when someone like

when you hear a real story and you know the joke would be well if you pitch that as a script it would just you know it would never fly right you know if that's what happened you know you can't argue with reality this is kind of the same thing where it feels like with dagger we're increasingly getting pulled into a path to basically finish what we started with docker you know and it's almost too on the nose but it's just literally through years of incremental user discovery and just spending time with users and shipping things and just working on the product and just iterating we just kind of get pulled into that so what exactly is dagger can you explain can you give us the 30 000 foot view of Dagger and what it is today

or how it began, where do you want to begin? Well, I'll start with just the big problem that we're seeing and that we think we can contribute at least to solving. Basically, I mean, a lot of things changed in the cloud landscape, let's say. Lots of great developer tools, lots of great infrastructure. But what has not changed that much is how terrible the delivery process in between is. So you've got great developer tools, you're shipping great code. It's right there. And then you got this really scalable cloud infrastructure ready to, you know, run anything. But in between, the pipelines are just still the

same mess of shell scripts and YAML and just a million artisanal scripts. And those things break. They waste the team's time. That's just kind of the problem we got pulled into because whatever else we pitched and showed and demoed, people are like, oh, that's great. That's great. Yeah, yeah. Can you help me with this, my deployment? Can you help me with my build and test and deployment pipelines? And there's just a long list of terrible, terrible things about them. And it's universal to small teams, what needs to be done to solve it. But the big picture

is, you know, delivery of applications to the cloud. If you zoom out and you remember what it was supposed to be, you realize we've been sort of settling for something that's not that, something that's really quite bad and we shouldn't sell. So what are the things that you are running from at Docker that you're now kind of running towards? I guess you've talked about it, but what like if you can get specific on those things, what is it? Well, one was open source. Okay. And the other was what I would call community led growth, where you focus on growing a community of people that share a common bond, a common excitement for a vision, a cause, a movement. And only then do you plug

products, adoption and growth into that from the point of view of a business, right? And I mean, everyone says, yeah, we have a community and yeah, community is important, but to actually say the top of my funnel, like the, you know, for my funnel to grow as a business, people using my, my product, people paying for it, people buying more of it over time, for that to require that individuals on their spare time are going to wear your swag and log into your Discord and write a blog post on their personal time about how great you are and just like annoy the hell out of all their friends talking about, you how great your thing is and not just a product just whatever the idea is behind it i mean that's kind of a big bet to make right and we're now

doing both of these things we're shipping a product that's not 100 open source it's two parts but the engine is open source and you can go i mean you can do a lot there's a whole community of people second second part, built around contributing to it, running it, programming it, writing code on top of it. And yeah, so we're now 100% community-led and the core engine in our product is open source. So the opposite of what we were hoping to do. The first thing we shipped was completely closed source.

And it was just another SaaS product that told you why you should use it. And you could sign up and if you liked it, use it more, blah, blah, blah. Turns out, I guess, I don't know. We're not meant to build a normal product with a normal funnel. I kind of wish we were because it sounds simpler, but I just, I don't know. It just never seems to happen. There's so much, so many more variables

in this direction, right? Like if you could just go make a good product, be proprietary, and have value and say, here's my price for this value. And you use the software and you scale your user base

and you get the value. That's somewhat the easy button. With this path, there's so many more variables because you can go down roads community-wise, have an amazing time. But like you said at the end, it may not be that valuable of a thing. You may chase the wrong rabbit hole, so to speak. Yeah, exactly. Which is exactly why we didn't start there. Right.

You know, and yet we're being pulled into it, you know, over years. I mean, we started working on this four years ago now and you know at the beginning it was just tinkering prototyping we were in no rush but i mean we've been working quite seriously on it for several years do you have investors we do how do they feel about this change in direction because this is pretty big this is drastically drastically different than proprietary SaaS to this direction. I mean, if I give you money at the proprietary SaaS stage, I might not be like, hmm, do I still like that decision?

No, it hasn't been an issue because well, we raised a seed round and then we raised a series A. And our seed round, we raised I mean, 100% on hey, it's the Docker team's next thing. I mean, a hundred percent on, Hey, that's the Docker teams next thing. Right.

You know, I mean, I would invest in that, you know, as a, as a, I mean, you, there's a premium for experienced teams with a track record of success. For sure. And the earlier stage it is, the more important that is because everything's unknown anyway. Right. So that's never been an issue. I mean, the new wave VC team, just to name them. And, you know, we have a bunch of really cool angel investors.

They're just, you know, you're the team we want to fund, go, you know. And then Erica Brescia at Redpoint led our series A and she's also awesome. And had she invested before we switched to open source, I'm confident she would have said the same thing, but she didn't have to because we launched our open source thing and raised our series eight pretty much at the same time so we had already made that switch you know gotcha yeah so she was on board with that from day one what was the pull then so you try to resist it you obviously i mean open source was lightning in our community at least docker community was lighting a bottle open source made you know productizing docker and stuff more difficult, etc.

You said, I'm going to start a new thing, different direction, closed source, SaaS. And yet here we are. You said you had kind of just been pulled that way. You just kind of end up that way. Was there like clear indicators? Was your community like, you know, were people banging down your door, open source it? Or was it just like an internal, your team just had to let it free? What were the indicators that made you change your mind? I think it's fundamentally a design problem. You know, we just spend enough time focused on the problem we want to solve, which again is application delivery is terrible. And then we really just spend enough time focused

on why and how to fix it. Like what's the fundamental thing that has to change? How come no one's fixed it? I mean, how many hundreds of startups and big tech products are there that aspire to improve application delivery? There's so many. And many of them are awesome. You know, and I'm casting a really wide net here, you know, but Heroku remains awesome.

All the HashiCorp tools, all these CI platforms, you know what I mean. There's a lot of tools. And so, why is it still terrible? And so we kind of honed in on that problem and looked for, okay, what's the fundamental thing that still not

solved and that we can solve? And we got an answer. And from that answer, then everything else flows you know okay what's the required solution how do you design it what components does it need and just to summarize the reason it's terrible is because it's fragmented because there's no unified platform underlying it where you can actually connect all the pieces there's no such thing right so you basically need i mean we call it it the DevOps operating system internally.

I mean, there's a million slide decks, okay, that have this aspiration. It's easy to say, oh, we're going to roll all that stuff up and we're just going to unify it under one platform. Of course, everyone thinks they're going to do that. But you two know, I'm sure, just how many have actually tried and they've all failed. Right.

Because they're doing it wrong and we're so smart, we're going to do it better. I mean, you know of course i'm going to say that but i think it boils down to even with dot cloud we tried also the the paths that's what paths was about you know the platform that you just run all the stuff on and it failed because it just became one more frag you know one more silo in uh in notion of silos. I think really the core of the problem is you got to think of it as a software problem. You know, think of your whole software supply chain as a really complicated application.

You know, it's a factory that ships your stuff that's made of code. Someone's going to run it and improve it. And that's a programming problem. So if you want to be a platform, you got to be an operating system. That's the only real platform there is something you can actually write code for. Windows is a platform. iOS is a platform. Heroku is not a platform. And neither was.cloud because you can't write code for it. So you need to be able to program all of that. So from there, that means you need some sort of an API

that can somehow cover that breadth of functionality, all these pipelines. And from there, you need SDKs, developer resources, you need developer communities, you need an ecosystem of code for your platform and the means to exchange it. You basically need a whole, like it's an operating system play.

Those are really hard. And back to your question, if this is really an operating system play, we're trying to take that whole software supply chain, all of it, and somehow make it programmable and give you the OS to program it and run it, which is an ambitious thing to do. The key to an operating system's success is the number of developers writing original code for it you know that's the only thing that matters so you need an ecosystem of code and in there there's code that could do amazing things that no one else has because it only works on your platform right and then you have a then you have possibly a hit on your hands you know and so how do you

bootstrap that well you got to do a lot of things, right? Like, A, it's got to actually work and do the thing better than everyone else, et cetera, et cetera. But also, none of that matters if you don't have that developer ecosystem. Early adopters need to love it. Right. Right. So we just kind of reached the conclusion that in this day and age for this audience of a developer community, because we want to take basically what you would call to the DevOps community and turn them into an actual developer community, you know, instead of scripting and half scripting, half administrating, half configuring. I guess those are three halves, but you know, these, these tools, all of a sudden you want them to be programmers. I'm, I'm programming my supply chain here. You know, that's the journey we want to take the DevOps community on, but we can't do that. If the thing you're programming doesn't have an open source the sdks the engine

the you know all that's got to be open source it's just not possible otherwise not for us maybe for github it's possible to try you know hey here's github actions they can build a developer community around github actions because they're huge. You know, CloudFormation is another example. It's kind of programmable. I would put in that bucket, you know? So sure, there's a developer ecosystem around CloudFormation. But even in that scale, you notice that in spite of CloudFormation being quite powerful, and now you have all these layers of CDKs and all of that, it's really, I feel like you can tell that the fact that it's fundamentally a proprietary silo is limiting

the growth potential of the developer ecosystem behind it. It's a massive product, you know, massive. But it will, I think you can build a developer community in that space that's orders of magnitude larger, but it's gotta be not tied to the proprietary silo in that way.

So that led us to open source. And from there, as we quickly realized, if you're doing your job well, building and supporting an open source platform, and you want an ecosystem of software, well then between your platform and that ecosystem of software for it you need a community of developers or just pumped right to write code for it and hence the community play like so one thing led to another basically gotcha so it comes down to the design very long answer yeah i followed though i think it was good yeah very good a lot of nuance in there yeah but it was design driven it was because of the problem you're trying to solve inform the the design, which is like an operating system,

which has to be a platform, which has to be open source, or else you're not going to get buy-in, and you're not going to get the software that runs on it or runs against it or is programmed for it. And so you were kind of barking up the wrong tree. You had a solution to the wrong problem. Now you've identified the problem, and you feel at least you have a strategy,

which turns out is the exact similar strategy of the last great idea. Yeah, because Docker really started out trying to solve the exact same problem. Right. You know. So this is like your second at bat then. Yeah, I think so. I'm now at peace with that. How long did it take you to get there?

It's really hard to prove that that's not what I was doing. Okay. But, you know, if... Why? I don't know. Well, first of all, I mean, it took a long time to figure out if what the problems were that were left to solve. I didn't leave Docker assuming it had failed to solve the problem. Right.

I left Docker assuming it was really tired. It wanted to take a break, you know. And then later, I caught up with Sam and Andrea, my co-found break, you know? And then later I caught up with Sam and Andrea, my co-founders, you know, and we got excited about doing something together. And that was the focus, just getting back, building something together, you know, being out there,

talking to people about their problems. And so you just, you set out to sail with a new ship, you know, we didn't start with a destination. We started with, you know, the ship and the crew. Let's go, let's go sailing and let's figure out where we go, know so that that's a process you got to right go through emotions and over time you figure it out this episode is brought to you by passport It is time for a new password manager.

This time, security first, open source, and with radical privacy. Passport is trusted by 15,000 worldwide companies, universities, startups, and many others. You can use it on-prem for total control or use their cloud service. What makes Passport different? The TLDR is more security and better collaboration. They are radically secure. They believe that any honest discussion about password managers must be heavily focused on security. And while most password managers focus primarily on individuals, Passport goes a step further, developing a platform that meets the needs of organizations as well as teams.

And privacy is in its DNA. They're headquarters in the eu specifically in luxembourg privacy is not only a top priority it is required by the law check them out at passport.com again passport.com I guess to a certain degree you have to kind of admit that you failed the first time around too right like you don't take a second swing if you hit the home run the first time. Now, maybe personally, financially, you hit the home run. But like in terms of solving the problem, you know, second at bat means I struck out to a certain, or maybe I hit a blooper to left field, they caught it. Yeah, I think, well, there's two things there. Me personally, I mean, there's definitely a lot

of things I would do differently. And I think Docker as a company, I'm so happy that it gets a second chance. And this is a real second chance. I mean, I've seen the numbers think Docker as a company, I'm so happy that it gets a second chance. Yeah. And this is a real second chance. I mean, I've seen the numbers. Docker is a really successful business now. So now it's, you know, up to them to keep going. But I'm really glad because boy, was that a missed opportunity. I mean, I think, I think Docker at least was a new red hat of VMware and that's pretty big already. And I say at least because I think it could have been more, but we made a bunch of very avoidable mistakes and I won't go into that.

But so there's a sense of failure for sure in trying to learn from them and do better the next time. Me personally, separately, I do think in terms of the opportunity for Dagger, those two things are separate in my mind. I think Docker is wildly successful in adding a layer of standardizing something and creating out of thin air a whole ecosystem

that's building tons of amazing things on top of that standard. And now, because Docker and the whole ecosystem around it, including, of course, the communities universe, but it goes way beyond communities in my mind. There's just a, it's so massive. Now, all these opportunities appear to solve the next stage of the problem. That's kind of how I think about it. And Dagger is entirely built on containers. So it's 100% impossible that Dagger would exist if Docker didn't pave the way first.

So really, for for me it's like finishing what we started you know yeah yeah yeah so but would would docker in an alternate universe where docker just just crushes it as a company the first time and just grabs the opportunity and just like this just does an amazing job at executing after the standard takes off right would it be docker doing this i mean maybe right sometimes i'm like maybe docker should be doing some of what i'm doing i don't know but they're busy with other things so so to fit my baseball analogy then it's not like you struck out it's like you hit a double and you're coming back up to bat you know you're gonna knock yourself home you know like you get a second at bat but it's not like you're out the first guy's he's on

second base he had a great run he's a building block towards scoring at bat, but it's not like you're out. The first guy, he's on second base. He had a great run. He's a building block towards scoring a run, and now it's time to score the run. Yeah, I think that's right. We love baseball around here, by the way. Always trying to hit home runs. It's all about the base hits. Get on base. That's key. Lots of singles add up. You mentioned GitHub Actions earlier.

Is GitHub Actions, is it a competitor to Dagger? Do they work together? I mean, because I know that we are using Dagger because our friend Gerhard and our president SRE and the host of ShipIt and on our Kaizens, he's a Dagger guy.

He works for you, Solomon, and he's very excited about that. And he has us running Dagger. I've peeked at it. I know it does some things. But we're also using GitHub Actions. And I'm like, I think a lot of us try to understand when you're creating something new, what can I proxy it to that I do

understand? Is GitHub Actions a good proxy for Dagger, or are they two different things entirely? They are different things, but they are related. They're complementary. You're not the only one who is not 100% sure. It was similar with containers in the early days. It was, it was so new that you kind of,

there was sort of a problem of, it doesn't fit neatly in any specific box. So most of the time when I'm, you know, advocating for Dagger in whatever way, explaining, pitching, whatever, I'm usually spending more time on the category and where it fits in the stack than whether it's better than X, Y, I'm usually spending more time on the category and where it fits in the stack than whether it's better than X, Y, or Z. Sure. So GitHub actions plus dagger is a very common

combination. And the best way to summarize what we do just stepping back is we basically give you a way to standardize all your pipelines and connect them all into a DAG and have that DAG be the new layer at which you do a whole lot of important things that have to do with delivering the application. You know, like orchestrating, imagining your builds and your tests and your deployments and all the other workflows that are involved that usually happen across several different environments. The developer's local environment, the ML team's local environment, the CI environment,

the production environment. Those environments are like silos. And the problem is your DAG really needs to connect all of them because there are dependencies, right? Step back, demystify DAG. DAG 101, just for the uninitiated. Yeah, a DAG is a data structure, really.

It's a graph of interconnected nodes where data flows in one direction. So the best analogy is a factory, you know, like picture the workstations and the conveyor belts of the factory, you know. That structure is a DAG. Turns out a DAG is a great way to model lots of things. It's a great way to model the layout of a factory.

It's also a great way to model the layout of basically your software factory and zooming out the whole supply chain, interconnecting your factories between them, you know? So it's just a construct. It's a way to model, but turns out is a really powerful one. And so dagger basically we use it as a verb. Actually, you take an environment, say your dev environment where these pipelines run, where you automate these tasks and you daggerize it. Then you take your CI environments and you daggerize it. And you've basically, what you do is you package all the pieces up into containers and then you interconnect those containers into a DAG. So you describe how they're related, which container sends data to what container,

and you end up with a full model of, you know, all the possible paths through running a build, deploying, you know, so now you look at that and you can see, oh, I see what are the possible things to do in this environment. Oh, I can build these, I can build these three different artifacts and I can run these four different kinds of test suites and I can deploy to these two different things. And each of those steps are, you know, they can be configured in this way or that way. And here's the flow of data between all of them. And all of that now is, can be run on top of a bunch of containers, which means it's portable, which means now you can unify those environments. So in very practical terms, what

we solve, the problem we solve is things are happening differently in CI and in dev, and it's just a pain in the butt. And now we give you a way to actually converge those environments. So it's the same thing. So any test suite or useful automation that's usually locked in prison in CI, now devs can use much earlier in the process they can shift it left you know so that means they can make sure it works more thoroughly before committing and pushing it you know so we make pre-push capabilities richer and vice versa you know we just kind of reduce drift okay so this is where you need a community because the whole thing is

i realize this is the opposite of a community because the whole thing is i realize this is the opposite of a neat product pitch so you can just put in a landing page and say yeah click buy it this is not something you can just in 10 seconds decide right oh this is what i want i'm going to buy it so you need a community of people like gear art who at some point they've ran into the problem there and they figure it out on their own. Okay. This is really in the weeds, but this is exactly what I was going to do anyway. So let me go ahead and try it out. And also let me go and join the discord because this is something I'm passionate about. And I want to just hang out with other people who are also, and maybe I can help them. We can help each other. So that's, that's what leads you down the path to community led because it is a niche problem that we solve, you know, for a pretty niche audience of people,

DevOps engineers. Oh, and all this runs on GitHub actions, right? It's compatible with things. So the connection, like GitHub actions would like trigger dagger to do its thing, right? Exactly. Yeah. Cause what, yeah, because right now your CI environments, what versions of what tools to run and what sequence, all of that, it's configured in its traps in a GitHub Action specific proprietary configuration. Right.

And so we basically, when you daggerize that environment, you free it from that prison. It's now declared in a completely portable way. Right. It's declared as code, so you can script it and do whatever you want with it. And GitHub Action is still there, but its job is now to run pipelines in that portable environment. So you basically shrink, you go from a 500 line YAML file, gradually you eat away at it, or all at once, your call. But eventually you end up, ideally, with a 10 line YAML that says run Dagger. I'm caricaturing a little bit but that's the idea it's effectively like a post commit hook kind of a thing where it's just like okay you know when it

when i push to this branch go ahead and tell dagger to do its thing and it becomes this just like trigger functionality at the end of the day okay so it's really that it's running on containers so it can run anywhere which people like and it's also scriptable like there's an api so all the behavior is expressed in code in the language of your choice we have a bunch of sdks that's the other big draw you know you can replace all that yaml that becomes pseudocode over time which that's another new direction as well because when we first adopted dagger we're 0.1 adopters around here yeah when we first adopted dagger it was q it was configuration something language

what's q called i forget what it stands for yeah it's an acronym it's a configuration language it's like yeah better than yaml because it's typed or something so people like it but it's a yaml-esque configuration thing and now it's not now it's like uh have at it you can write golang you can write typescript yeah that's exactly right so we launched a year ago well a little bit over a year ago, and you had this DAG engine, you know, so that an engine that enables what I described,

you know, you can DAG-erize your environment, package it up in containers, describe the layout between the containers, and then it's scriptable and portable, et cetera. Great.

But you have to write all that new code, that's those scripts in this obscure, new configuration language. That's like HCL or like a much better YAML. I mean, the Q community will hate me for saying that. But, you know, it's in that category of languages.

It's declarative. You know, it's just a very powerful declarative language. It's a next generation configuration language. And honestly, it's awesome. It's very powerful. The only problem is that nobody, I mean, it's new. It's very powerful. The only problem is that nobody, I mean, it's new and people generally don't love new. Learning a new language is just a high barrier of entry, specifically in the context of these pipelines and the environments where

the pipelines run. If our pitch is basically, hey, get rid of all that terrible yaml that only works in one place now it's scriptable and it's containers and it's scriptable and then we add oh and it's scriptable this other thing that's kind of like yaml except it's newer and you don't know it right you know it's kind of a bait and switch is so that's the feedback we got kind of side grade you know and people wanted it so bad i mean we had a pretty massive cohort of people run through it and use it, join the discord, ask questions. We had a solid six months process here where a lot of people flocked to this product when we launched it. And then just very slowly we realized, not all, not at once, but eventually we realized they're all going to leave. And some of them would stay

for months and just keep trying, but they would just hit one little, it was just death by a thousand cuts. And we didn't realize it, but we kind of got dragged into a tarp that would just slow us down more and more. We were kind of getting dragged down by all this friction. And some of it was just how we used Q. Some of it was just some fundamental issues with Q because it's new, like memory leaks, et cetera. It got better, but basically it didn't matter. It couldn't be perfect. The problem was that it was different. So eventually we figured out a way to script our engine with a language that was not declarative, even though the way the engine works, the API for it is fundamentally declarative because you're describing a graph. So you can't, like it's a 3D construct, right? You can't go through a sequence of things to do. It's not a to-do list though. I mean, that's what imperative

languages are, right? They're to-do lists for the computer and you just, you're missing a dimension. So that's kind of, that's why we went to Q in the first place. How do you express this three-dimensional construct without a specialized language? So we found a way. And so we eventually shipped all these SDKs. Yeah. So now there's Python, Go, JavaScript, TypeScript, you know, you can, you can create your own SDK. There's a bunch of new SDKs that are experimental that the community is developing. Rust, Elixir,.NET. So that really unblocked a bunch of things for us. App.NET also runs on App.NET. I'm going to go pitch Dalton on it.

Try to work that back in there somewhere. Yeah, so that's cool. I mean, ours is, our stuff is written in Go, but we're an Elixir app, so it'd be cool to be able to unify, you know, on a language like that. Does that mean, so I mean, obviously those languages are all Turing complete, but are they like generating, they're not generating the Qlang,

they're generating something else at the end of the day. No, but we consider that, we explore that path, but Qlang doesn't like being generated, basically. It's really designed to be the thing that you write and it generates everything else. Gotcha. And so it just felt like we were trying to use Q for something it was just not designed for.

Okay. And so eventually we stumble upon GraphQL. And so the Dagger engine has an API. Okay. That API is a GraphQL API. It turns out it's a perfect match for describing a graph, a DAG, you know? Yeah.

And then on top of that, we got bindings for these different languages and they're actually generated from the GraphQL schema. So when we ship a new feature on the API, boom boom all the SDKs get regenerated and you get native support in them it's pretty neat and of course the pipeline to do that isn't is a dagger pipeline of course it has to be yeah yeah yeah very cool well I'm not sure if you heard it or not Solomon but we had Kelsey Hightower on ChangeLogging Friends a couple weeks back and we asked him about some things that are going on did Did you hear him describe Dagger?

Yeah, I did. And I remembered exactly the conversation we had at that VMware conference. Well, no, the conversation. Oh, the one that he, yeah, yeah. Because you guys spoke at VMware conference. Yeah, yeah, yeah. Yes, yes, yes. Well, the way you described was very gracious.

You know, I was there showing this demo and like a little stage on the side. And, you know, he was very gracious and came in and saw the demo. And then we talked about it afterwards. And I mean, you know, Kelsey and I go way back. Yeah. Summarize as this like customs robotic arm. Yeah. That's like part of your factory.

That's a good analogy. Yeah. Yeah. Do you think that's accurate? Was there any like, did he miss the point on any aspects of it? No. I mean, the only aspect that, well, I think it's part also the timing. I mean, we started out being very focused on the deployment use case, you know, the last mile. Right.

And over time, we got pulled more and more towards the CI side, you know, build, test. Ultimately, it's all in the DAG. I mean, ultimately, you know, we're going to DAG-erize all of it. But there's definitely a sequence. I mean, there's some pipelines are a better fit for DAG-er today. And it's really build, test, deploy. And the deploy usually is we just hand off to a specialized deployment tool. Like we hand off to kubectl or kubectl, or we hand off to, you know, the Netlify clients to upload the Netlify or to an AWS Lambda tooling to just send the thing up there. You see what I mean? So it's kind of like a handoff.

So for us, that's just one node of the DAG. So really, the integration of all these different parts is really where Dagger shines. But that fits perfectly in the factory analogy and swappable robots that Kelsey uses. I love that. I love that analogy. And I mean, I talked, you know, you need a standardized box to ship things around. We're solving the manufacturing problem. You know, how do you standardize how the things are assembled and the interdependencies between all the different steps of doing that, you know, within the organization, but also between organizations. You know, there's your factory, how it's organized, and then there's the factories upstream from you and what they ship to you, et cetera.

So that's kind of the factory problem of software, and that's what we're tackling. I think that's a good metaphor. Yeah, I think so too. It's not quite the containers, like the boat that we all see. We assimilate Docker to be this, I don't even know what you call those container boats. If they're...

Oh, yeah. Cargo ship? I don't know what you call it. Yeah, that's the foundational analogy, right? The shipping container. But standardizing boxes to me makes slightly more sense because I, probably like anybody, get a lot of shipments to my home from Amazon and the like.

And so, you know, any given week, I'm like, side story here, just tearing down boxes to recycle, essentially. I mean, it's just a massive amount if you order a lot of... I just moved into a new home, so we're buying things and getting things and whatever and, like, obviously tearing down boxes we have things stuck in.

But, you know, you think about the size of the box, and this is something I think FedEx and UPS and other large-scale shippers obsess over is like if we could just have a standard box we can we can be more efficient in how we actually transport things to and fro maybe even standardized prices because the size doesn't change but the weight might and as long as we're under a certain amount and the size of the box stays the same you have predictability in the model and so deliverability moving those things moving what was manufactured in the box you know gets a lot more easier so the the analogy i think

is really better in that case than just simply this large-scale cargo shit that we don't really see too often you know i think it's a better analogy well you know you actually the the weakness uh the limitations of the the physical shipping container story, well, the limitations of the standard, which is what you just talked about, not every box in the world is a shipping container because that's pretty big. So it's like the backbone of the world's freight industry is built on it. But I don't get shipping containers delivered at home. And so that's, but you know, what's crazy is that's what happened with the software containers too.

I mean, the OCI spec, which standardizes the Docker format, a lot of applications get packaged up in Docker containers and deployed as such in Kubernetes or whatever, but not all of them. And it will never be all of them because there's just too much variety and scale in the kinds of compute that's out there and the kinds of software that's out there, you know? And so there's a limitation to how much you can

standardize the actual box, you know, the actual shipping of goods in the world. And it's the exact same thing with software. I think, and I'm going to go into slightly unexplored territory here because this is, so this might turn out to be a confusing point, but I feel like we're tuned in, in three months or six months, and maybe you see a really well packaged version of this story on our website, I hope. But if you look at the, you know, the manufacturing problem here, the software supply chain, I think if you're pushing anything container-based on the application platform, if you're Red Hat pushing OpenShift, I think if you're pushing anything container-based on the application platform, if you're Red Hat pushing OpenShift, okay, if you're any Kubernetes vendor pushing Kubernetes for everything, then you're pushing a vision that ultimately needs the shipping

container to be the only standard for shipping things around. We believe that's never going to happen. Not because we don't like it, We just don't think it's physically possible for that to happen. So as soon as you're saying everything I'm building, it's Docker containers only and communities only at the application level, you're right there and then never going to be a standard. Beyond your new thing, it's never going to be a standard for how you organize your supply chain. Because you're going to have blind spots. So how do you solve that? Because you still need to standardize the supply chain on something. I think you can standardize how you ship the factory itself to the edge.

Okay. The problem is we're stuck in this analogy of the software factory. Okay. When you think factory manufacturing, what are you picturing? You're picturing a big centralized thing. Yeah. You know, all the cars are built here and then you ship the cars to everybody. So there's only one factory here. There's not, there's not a lot. And then shipping is the bulk, does the bulk of the work after that you ship the car to wherever. I think in software, in fact, the correct model is flipped where i mean it's

software you can just ship a factory to everybody and they get their own custom you know it's like they like star trek you know the the food synthesizer right you know they don't ship all that food to the to the spaceship just to synthesize they ship the ultimate machine that knows how to like manufacture the food you need right there and everybody gets one i'm sure you know in their homes or whatever that's kind of i mean that's the software that can do that you know so you're basically teleporting what you need over and and the beauty of it is that you can standardize because that factory it's okay if it gets shipped in a in a big bulky shipping container because you only need it once you know so the question is how does it work you know how does it once you know so the question is how does it

work you know how does it link up with the other factories and how does it get the intelligence to know like how does um how does the food synthesizer and the star trek ship gets its software updates like does someone say oh i have a new recipe right and you know is there a community of uh star trek chefs that share the software for the recipes i mean that's basically that's what we're trying to do for Dagger. It's a hard problem to solve, yeah. Where does it get

its ketchup and how does the ketchup enter in? I told you it was going to be a little outside the beaten path but it feels like there's something

important there around just how that supply chain, how that software is assembled, manufactured. Right. Ship the whole factory

but you have to standardize how the factory works basically. Yeah, everybody gets. Right. Ship the whole factory, but you have to standardize how the factory works, basically. Yeah, everybody gets a factory. It's like decentralized. You know, like 3D printing is kind of like a hint at that, you know. Oh, I can make it myself, you know. That whole controversy around, I guess,

3D printed guns as an example. Like it has profound implications because you make a bunch of assumptions starting from the fact that it's really hard and centralized to manufacture things and then all it all goes through shipping shipping is the most important thing right but if everyone's got their little factory that's gonna nothing gets shipped except the factory the little parts the factory gets there and then all the filament to make the things well yeah yeah you know so i mean that's kind of like the the software pieces yeah yeah actually

you're right a lot gets shipped but it doesn't have to be standardized how it gets shipped you know so because uh you're kind of encapsulating the intelligence of how it gets there and i'm gonna try and bring this back into more concrete terms here but like the big problem we deal with is your typical team has a lot of different build tools they have a lot of different build tools. They have a lot of different package managers. They have a lot of different deployments targets, you know, front end teams, back end, you know, their containers get deployed to container things, but there's the JavaScript world, there's the MLOps world. So artifacts of all kinds and deployment APIs of all kinds, packaging systems of all kinds. At some point when you deal with, when you have to integrate it all,

it's pointless to say, okay, step one, everybody across all these ecosystems, everyone change how you ship your artifacts. You have to all adopt this one standard for how your artifacts are shipped. And do me a favor, all these cloud providers, just use this one API that everyone uses for how you, it's like the open stack problem right let's convince everyone to adopt to standards on how they ship things around

and then we can integrate but we're taking the approach that that's never going to happen so if instead you just let each endpoint just whatever do whatever the hell they want in the back end and we just worry about standardizing how they fit together in this factory locally, you basically have... The customized arm. I think this is where Kelsey's argument, his statement was like... Exactly, that's exactly right.

I'm paraphrasing. It's a customized robotic arm in my factory. What you're saying is you want to be able to give everybody the ability to have their own customized factory. Not just the arm itself, but literally the entire factory. And no matter where your artifacts come from, you can put it together at endpoint, at edge, wherever you need to put it at.

Exactly. Well, it's really about which part is customizable infinitely and which part is standardized strictly. And it's really about where do you draw that line of where the standard should be. And with this robotic arm analogy, this factory analogy, we're just moving the standardization points at a different place

because it's still, at the end, it's a standard dagger factory. It's just that each robot in it to use Kelsey's, I should just use Kelsey's analogies to start with. Always just use Kelsey's analogies.

That's what I've learned in my life. CICD is kind of like when you go get Jenkins or any of the kind of prepackaged solutions that are out there, it's almost like buying or using someone else's factory, right? They tell you, these are the only cars we can build at this factory. We're fitted for these things. So if you're going to build something, it has to be within these parameters, and we can build a million of them for you.

But these are the parameters. That's all you get. The Acura feels like the ability to create custom robotic arms. Like when you go look at a Tesla factory, you can't make an F-150 in a Tesla factory. You can only make the cars that Tesla makes in that factory. And so Dagger is kind of like this idea of whatever complex step you have, it could be deploying something to a server. I think what Dagger gives you is the ability to create that custom robot for that one step.

Would you use Dagger to run the build command? No, it's overkill. You could put it in there, but that's like kind of a solved problem. So just use your standard conveyor belt to get the base of the car to this arm. But that arm is going to be really customized, right? You want it to work a certain way. And so I think what Dagger gives people to do is,

if you look at your whole factory, let's say you're using Jenkins, you still could use Dagger as part of the inner loop. So hey, Jenkins, you do the checkout from GitHub, you do the build, you do the packaging. But when it comes to deploying to Kubernetes across six regions and the low balancer updating Cloudflare and busting the cache on something like Akamai, I ain't trying to articulate that in a bunch of bash scripts. I also don't want anyone else trying to do that five times throughout the organization. So what we're going to do is we're going to use Dagger to build this custom robot, give it an API, and then Dagger as an atomic unit. When I look at my Jenkins pipeline, this fifth step is Dagger.

Get rid of the 25 bash scripts. Throw the Dagger robot in for dealing with Kubernetes. Throw the Dagger robot in there for dealing with Amazon Lambda. And so I think if Dagger is going to be successful, just like what they did with like Docker images and like having those kind of be ready to go units. I don't know if they really thought about it this way, but this is how I express that I thought about it. You got to think about Dagger as these customized units of robots and if they're shareable. So if I come to a company and say, hey, what is the best way to deploy to AWS Lambda, IAM, load balancer, the whole nine, and do rolling updates. Oh, man, that can be challenging. So Dagger already kind of give you some if-then-do-this kind of primitives

out of the box. It gives you enough feedback loops so that you can integrate with a Jenkins, like a higher reporting tool, something that has a UI that can decide when things go and when things don't go. So, man, if I could just reuse those in an organization to say, yo, to me, Dagger is a last mile technology that allows people to do all that customization that Bash is not equipped for. That's the way I think Dagger will add value to the world and tools like it.

Let me try and connect those two analogies because I was talking about shipping versus manufacturing and where you standardize. Right. And Kelsey is describing the robotic arm and the factory. And the factory analogy is perfect because it's about manufacturing software. Where shipping comes in is maybe that robotic arm, if it's really customizable any way you want it, that means maybe that robotic arm, you know, maybe it's soldering, okay? So it needs supplies to solder, right? Where do these supplies come from? Maybe that arm has an incredible tech and it could just, you could hook up a cable and it's connected to this

worldwide network of, you know, that just brings this, you know, the required material straight to the arm. We don't have to standardize how to package up that particular material to get into the robotic arm. It just does whatever it wants in the back end. The important thing is I can always connect it to the standardized conveyor belt, and it's going to fit. And the operator will have a standardized display

somewhere that's red or green to say it works or not so the the front ends to the team operating it has to be standardized at some level but how it does this thing including shipping artifacts around to the outside world and from how it gets its credentials you know how it communicates over the network none of that needs to be standardized by us okay i think i follow let's simplify let's get back to brass tax as which is a saying that i'm not sure what that means let's get down to brass tax i think that's where we pay each other money or something but let's not do that part let's get back to basics like let's imagine me a developer with an application let's pick a a

pretty typical let's go uh like a 12 factor web app right i've got a front end i've got a server-side api i've got a database and i want to bundle that up into a factory and ship that factory somewhere with dagger but all i have right now is a repo with the code in it and some sort of command i can run that executes that code what do i do from there dagger daggerize that sucker well the starting point of dagger is there's always you always have something okay and then you can daggerize that something so oh i thought i had something well you have something that's what i'm saying okay okay you're not sure you're never starting from zero right I have an app it runs like if you run

this command my Apple compile and it'll run yeah on my machine and so you're saying it's not live anywhere yet no it's on my computer I have it in a repo maybe it's on

github in a private repo so what happens at that point is someone maybe you or someone else needs to make a bunch of decisions on how that application is going to

be delivered right like maybe you're going to say well this is a really simple app so we're just going to be delivered right like maybe you're going to say well this is a really simple app so we're just going to go for heroku for now or one of the 10 000 heroku competitors now right and that's it that's basically it it's never really completely it that's maybe but maybe that's what i would do i would do that that's 90 of it right and they're remaining 10 maybe there's a there's going to be a make file and NPM scripts if it's a JavaScript app or a rake file if it's Ruby. So there's a little bit of scripting and automation. So that's basically your version one of your very own software factory.

You got those pipelines hooked up. They're very simple. They only run on your local machine. And then they hand off to, they upload a bunch of things to heroku but basically as soon as you start working on that app and shipping more versions and if it's a real app you know yeah it is it's not frozen in time then really quickly as you add more code and more features you're also going to start adding more pipelines your factory is going to evolve alongside the product exactly like a real physical product, by the way.

You know, manufacturing, it's always deeply embedded in the product design, right? I mean, there's a reason it's called industrial design, right? Great, you got a prototype. How are we going to ship this thing? You know, let's talk about the factory layout, you know, like let's talk about suppliers and stuff. So same thing there, you know, instead it's Gerhard art telling you, okay, let's talk about, you know, scanning your container images. Let's talk about using Python for this API component now.

So let's talk about Python packages and you know, so you just, your factory just grows and growth, just how it is. And eventually you reach a point where you say this factory is out of control. Oh, I've definitely said that. And I need to go from, you know, an artisanal workshop that is honestly getting a little messy and I need to industrialize this, make this an actual factory because, you know, it's time. We've reached that threshold of scale or complexity or business requirements, you know, like, okay, now there's a customer

they depend on. Something happens and someone will say, usually a DevOps person like Gerhardt, will say, as your designated DevOps person, I am telling you it's time to industrialize. And that's where Dagger comes in. Okay. That's where you Daggerize.

I'm not picking up Dagger right away. I'm picking up Dagger eventually. I think okay that's where you daggerize i'm not picking up dagger right away i'm picking up dagger eventually i think today that's true we're trying to make dagger more and more approachable and more and more of a an obvious no-brainer decision from day one like why would you not start there as of today that's not the case i'm hoping that in six months it'll be different by the way the same thing happened with Docker. Yeah. It was a long journey swimming upstream, making Docker gradually more and more accessible

to developers on day one. And we got there eventually. So I think the same thing will happen. Eventually you'll say, well, it's, you know, might as well just start here. It's all easy and everything. You have this ready to go sample factory, you know? Right. So in six months, I might tell you tell you well jared the way you get started is you install that the dagger cli right you select one of the 10 000 ready to use environments you you know you said

ruby and rails posgres so search for that you'll find it type one command boom you know you're in business so that's that's the answer in six that's more what i was expecting you to say this time well now i yeah fair enough no fair enough it's just where it is is that what you're working on then this cli and this magic behind the scenes because i mean you said six months oh yeah yeah there's a specific project that's happening that's not very far away yeah when i when i have no idea when it's gonna happen i say 12 months when when when internally we're like this is totally going to be ready next month i say six months okay cool that's good so it's in motion is the point it's in motion i'm not trying to hold you to a line but just you know it's in motion

this is something you're doing no no it's in motion we're at the phase where we're deeply uncomfortable showing demos because it's so rough and early but we do it anyway so actually we have we showed we showed a rough prototype to our community at last week's community call. So it's on our YouTube channel. You can find it. But it's inside baseball still at this point. If you're not hanging on our Discord and kind of in the know,

you're going to see it and be like, okay, this is a science project. But that's how we start everything. But yeah, things are definitely in motion. Cool. So who's your ideal community member today? Somebody who's in DevOps already, maybe they're running a bunch of systems.

Yeah, I think someone who either is or aspires to be what I would call a platform engineer. I feel like that's becoming a controversial term, but someone who's the DevOps person and uses code to automate what they're doing or aspires to use code to automate what they're doing, as opposed to purely clicking and writing a bunch of YAML. There's an evolution in the role where at some point it becomes a real engineering function. That's part of industrializing this whole thing. And anyone who was either doing that today and

looking for better tools or aspires to do it and isn't sure where to start should join our Discord today and immediately see how people doing it every day talk, what they're doing, and they're kind of converging around the Dagger community. Yeah. An interesting place to be for sure. Daggernauts, by the way, we call them Daggernauts. Daggernauts? Daggernauts, like astronauts. Daggernauts. Okay. That's better than Daggernauts. Yeah. Daggerna dagger knots is good i think that's a cool place to be because i can tell i mean especially as this mirrors now to a certain extent the docker story this is the time to be involved you know and if this thing is going to take off like a rocket ship you know

be part of it. Early community members, there's just so much opportunity and so much you can contribute to that because it's easy contributor's time. Maybe you can speak a little bit to the contribution, maybe policy or ecosystem because it's difficult. You said you're 100% community growth-led.

I'm sure as a product owner, that's challenging because you have ideas, they have ideas. Do they always jive? If I hop into the community, how does it work? That part is not really a problem today. There's always tension between the power users and the people building the product, so that's healthy.

Open source just means there's a more tangible way to resolve the tension because the power user can open a pull request. This is exactly what I mean. You know, the difference with I think what you're picturing of, like your typical CNCF project, let's say, where community is very different. I would call that actually an ecosystem. Okay, what's the distinction for me is that our community is defined by a shared vision and a shared excitement about specifically the Dagger brand and the product behind it and using it. So it's all power users or aspiring power users. It's not a bunch of different companies that have different interests, different motivations, maybe are competitors, but happen to collaborate on a well-defined piece of tech. That does not meet my definition of community. I'm saying other people can do what they want, but what we call our community is different, right? With Docker, we learned that the hard way because a lot of people said the community, talking about our

community. So we allowed others to define what our community was and wasn't. And usually it was in the shape of competitors who were not excited about our brands, didn't have a shared vision with us. We're not particularly rooting for our success as a company. We're not using our products. But they did want to collaborate on a shared piece of tech, which is completely legitimate. But that's an ecosystem. Eventually, we spun out some separate open source projects and donated them to CNCF so that, you know, that ecosystem had a place to go. And so if you're, you know, there's ContainerD, there's RunC,

the OCI spec, et cetera. Is that a community? Yeah, sure, maybe. But the Docker community, really the core of it, which we realize are Docker users. You know, mostly they're developers and DevOps people, less so infrastructure folks. I think the infrastructure community either actively hated or just generally resisted adoption of Docker for good reasons, because it was kind of designed to kind of bypass them. And it was not designed, you know, primarily for them. And it's something new and unstable and who's going to be on the it was not designed, you know, primarily for them. And it's something new and unstable and who's going to be on the hook to fix it, you know, for in the morning, of course,

it's not going to be the developer, right? So what I mean is we realized over time that the Docker community is a community of developers. And so over time, we just kind of, in fact, the tension was internally

was that it was hard to kind of reconcile everyone at Docker, that that was true. And there were just a lot of possible paths to take. Anyway, in our case, when we have tension with the community, it's users saying, this is broken. You know, could you fix it or could you prioritize things? Or, oh, or we bike shed a lot on the design of the next API thing. But we kind of turn it to positive thing. Like we have this private joke that, you know, bike shedding is healthy, you

know, in the bike shed once a day and it's, it's all done in good spirits. So it's a lot of fun. There's really no room for like, like let's say Red Hat decided, oh, Dagger is awesome. I'm going to build OpenShift 4 on it, you know, and I'm going to say, I'm going to support, you know, the Dagger, I'm going to be the Dagger corporate sponsor. Well, there's no place really to do that. You know, it's just not the right place to do it. So the other half of this is that we've tried to design the commercial parts of this product into the overall experience early so that it's well, it's clear to everybody what's open source and what's not, what's free to use and what's not. And it's sort of organic as opposed to later becomes a source of stress

and confusion. We're trying really hard to do that early. Would you describe it as open core? Honestly, I don't know because I'm not, I don't understand what the boundaries of open core are, but I guess roughly, yes, I would say yes. I mean, the engine is open source. The SDKs are open source. The CLI is open source. So you have everything you need to go and develop and run these pipelines. You can daggerize to your you know, it's a bunch of machines, you know, in a data center somewhere, then you need a control plane because you have a bunch of engines running pipelines and parallels. You need to coordinate, you need to monitor them. They need to share cache. Caching is a huge part of what makes Dagger great because everything's cached by default. It's a property of the DAG. So everything becomes way faster very quickly compared to old school pipelines. But there's a data sharing element. So you need to coordinate that, the distribution of the cached data. And then you just want a

place to see what's going on, visualize on the web interface, your DAG, and then troubleshoot, collaborate, things like that. So all that is in this commercial product that we're building. We started rolling it out in the early access, but it's not launched yet. But it's a pretty natural line. Basically, it's purely product design driven. For any given feature, we just ask, what's the best user experience? Where does it make sense to put there? Should it be in an individual engine running on an individual machine or should it be a globally shared cloud service? Like literally, how does it work better? And whatever the answer is, that's where it goes and that's it. You know, so there's really so far, fingers crossed, it's pretty aligned. And I think, yeah, we'll see where it goes from there. You know, the test will be one day will be big enough that a big company will come in and say, I want to sell my own dagger control plane. And here's a pull request to add, you know, a plug in support so everyone can swap out their control plane and you better merge it. Or I'm going to blog about you and say you're mean. I'm going to say thank you for your pull request, but it's denied, you know, and you're free to fork.

But you call it something else, you know. So we have a very open copyright stance, you know. It's like a very open license, a real open source license, but we have a very strict trademark policy. By the way, that's the Red Hat model. Right. So Dagger, the open source project, has to use Dagger, the service. No, it works without it. It's optional. It's optional.

It's optional. In production, like in a production CI setup, it'll be faster. Well, it probably will not meet your criteria for production deployment without it. But you're free to build your own control plane. And maybe one day you'll be able to buy an alternative control plane from someone else. I mean, there's no such thing on the market right now because we're too small, but you could do that. But we will always be at an advantage because if the Dagger brand and platform is where you're here, that brand and that platform identity will remain

ours. So that's more of a HashiCorp model in that sense. Yeah. But you said that you wouldn't accept the pull request for a plugin or something that allowed you to plug in a different control plane. Yeah. I mean, we'd make it technically possible, but like for example, very simple. There's a Dagger login command where you can log into Dagger Cloud. There's not going to be a Dagger login dash dash alternate provider. I mean, not that today there would be anywhere that you could do that because no one offers that. But yeah, we don't plan on offering,

like on allowing that. And that PR would be on the pull, that would be on the open source, right? That pull request would be on the open source and be denied? Because the CLI is open source. Yeah, that would be denied. Yeah, I don't have a problem denying that.

You know, the reality is our community, if our community cares enough, then we'll have to like really get to the bottom of why, you know, and then take it from there you know it should not be impossible to buy a competing product but um you're in here building on a brand and a community that we spend money and time and energy uh building i mean the community exists because we're doing a good job people love what we're doing and you get to come in and play on

a level playing field. No, go build your own community. I think that's fair. Or hey, guess what? We have a new marketplace offering. If we don't create fair, exciting opportunities for other companies to make money in our platform, then we're screwing up the opportunity, right? We have to kind of, as the value of the platform increases, we have to offer opportunities for the ecosystem around us to share in the value or worse, we're going to miss on the opportunity. By the way, that's something that Docker did not do well. But part of the reasons Docker didn't do it well is because we didn't have enough carrots, but also not enough sticks. A lot of the potential partners are like, oh, great. No, thank you.

I don't need to partner. I'm just going to take this brand. Thank you. So no, you don't get to do that. What's the percentage of production environments or deploys that would be greatly benefited by having your paid for control plane? Well, it depends on what you call production. But today, the path is really you start using Dagger on developers' laptops. And then once you're comfortable and there's a team that really loves it, you start running it on your CI platform. So GitHub

Actions, GitLab CI, Circle CI, your Jenkins cluster, etc. And so that's what I would call production here. And so in that CI environment, I would say 100%. Like 100% of those deployments need Dagger Cloud today.

Are we using this control plane, Jared? Do you know much about if we're using the control plane that he's talking about? We're not. So we're able to do what we do without, and we use GitHub Actions. How are we doing it? Tell us how we're doing it, Solomon.

Yeah, how are we getting around your control plane, Solomon? Just to be clear, I mean, there's lots of ways. If you're like a Daggernaut and you're familiar with the platform, there are many ways to make it work without this. So, for example, the way Gerard's doing it is it's running on one dedicated machine. And so, you don't need to distribute cache data around because it fits in one machine. It's all in a local storage.

And he's set it up that way. And anyone can do that. So, I'm painting with very broad strokes here. Every step of the way, what I described, there's a million ways for the community to make it work for their use case. We're talking about businesses coming in and saying, I want to rely this to industrialize my pipelines. Let's talk talk about slas let's talk about budget let's talk about uh you know lock-in you know let's let's talk about a commercial partnership yeah and that's where these things really matter you know i mean yeah that's where they matter i think our questions are more like parameters or more like boundaries what's the boundary of you know come

be part of the community as you described it, give you all feedback, be, you know, excited about where you're going, you know, have that shared vision, you know, how can they, you know, freely adopt all the ideas you're doing and at what point do they have to convert to a, any sort of paid scenario to leverage and use the tool is all we're trying to map out yeah yeah what that boundary is it's purely yeah when you need to run it at scale and you need this control plane running alongside it right and that's the boundary of our commercial offering so that you know for example we don't we don't have an hosted version of our engine that is somehow better we just we don't sell compute at all uh we don't have an enterprise version of the engine or add-ons to the engine

that make it better. None of that, you know. So all of that is open in the truest possible sense. So it is very open. I've been focusing on what's been on my mind was, you know, where we insert ourselves and making sure that's solid. But I mean, day to day, I'm sure 99% of the agronauts, even today, are not aware of any paid offering because it's not on our website. It's starting to come up more now because what happens is people come back on Discord after a few months and they say, hey, so I love the agro. I'm doing this and this. And now we're going to CI. And I just realized when I run it in a CI, it works, but my cache is always empty. Or it's frequently empty because the machines are ephemeral.

So the local cache goes away. Hey, I'm just wondering if there are solutions to that. And we say, yes, there are hacks here and a commercial product we're building that will solve exactly that. Would you like to talk about it? And it all happens very naturally. Well, that's a downfall, I suppose, of the early Docker model. To go back to Docker, it was enterprise, it was salesperson-driven,

it was very much not bottoms-up, it was top-down. And I think that they realized where to capture the value was to actually create value for individual developers and then charge per seat, essentially. And so you're learning this lesson, I suppose, early that your value really is the ability to talk to those who want to scale or operate at scale and charge them for a great control plane, right?

You don't need that at a smaller scale and that's fine because you want Dagger, the idea of DAGs, Dagger Notch, you want this to be a standard across all developer environments and it begins on the dev's machine. What a better place to begin rather than somewhere else. But the moment you need to do some sophisticated CI, GitHub actions, etc. Well, now you're a different kind of customer and you have different kind of concerns. You probably have different kind of pockets with willingness to separate yourself from your money for value. And I'll be honest that the

exact interface there, that transition, we're still sweating the details. Literally, that's the other topic on my agenda today. We're figuring out the pricing and the packaging of it. You know, like one topic that comes up a lot is it's good for monetization that a path to production requires your commercial product, but it's also, it can be bad also because it can be a source of friction if your commercial product, but it's also, it can be bad also because it can be a source of friction. If your commercial product doesn't have the capabilities that a particular team needs, or if it's not even launched like ours, then you have the dependency can hurt

adoption of the open source thing. Like, oh, I was going to use Dagger, but to take a concrete example, our control plane, you know, it orchestrates the caching operations of the flow of data between the engines and back and forth to a storage bucket so that the right cache data is at the right place at the perfect time. And so right now, that happens in S3. So that means if you're running your CI on Azure or Google Cloud, for example, then there's a lot of data moves back and forth and that has latency, but more importantly, it costs money. So architecturally, it's not ideal. So we're rushing to add support

for storage buckets in those other clouds. It's not a massive project, but it's not done yet. Meanwhile, we have people saying, hey, I'm ready. I'm in Google Cloud. Let's go. I want this control plan.

They're like, wait, wait. So this is where maybe it wouldn't be better at this particular time for the health of the platform if there was a ready, if it was possible to go to production without buying Dagger Cloud, because there is no Dagger Cloud to buy at any price right now that does this. So there's this, we're trying to figure this out. And also then the price. What if you pay a hundred bucks a month for your CI? In other words, not a lot. And you want to use Dagger on it. What if our first plan is at 500 bucks a month? So you're paying five times more for your control plane, you know, if you're Daggerized environment, then the compute that actually runs it, that doesn't compute. So probably we would need a cheaper plan, but then how cheap do we make it? How much of this should be a commodity infrastructure? It feels like the more of a critical dependency it is on a commercial thing, the more responsibility

we have to make it easy to use for smaller teams at a lower price and also to be more reliable because it's going to be everywhere. So it better work properly and not be priced ridiculously, at least not at the entry level, you know? So that's the stuff we're figuring out now. Yeah. Community led growth. Community led growth. Once you get to a certain scale, you can pay them. You're not sure how much to charge yet. Interesting times. Yeah. I love this though remember six months ago we were dealing with people

trying to learn Q and failing and now we're dealing with people who want to buy something from us and we're figuring out

how and that's it's I'm appreciating that change yeah that's a much better problem to solve

for sure well Solomon thanks so much for coming on the show. Thanks so much for thinking out loud with us. I can tell. This stuff is like you're actively mulling a few things and metaphors. You're working on the metaphors. Work in progress.

It's a work in progress. I think things take a while to formulate. And I'm now starting to understand Dagger much better. I think that that will be one of your challenges will be education as things go forward. Of course, the website currently says CICD has code that runs anywhere. Not a bad starting place,

but there's definitely way more to Dagger's story. So exciting times, man. Yep. Definitely interested. It kind of feels like you came over and visited my mental workshop for an hour here. You definitely workshopped a couple of things, which, hey, we're here for that. We love to think things through together. It's fun and stimulating.

By the way, this is how our Discord feels every day. This is basically what we're doing with our Daggernauts all the time. We have all these discussions openly, and it's really fun. I really enjoy it. Yeah, that's cool. I'm looking, since you're talking about, Jared, we're sort of like ideating the value proposition, how it's described on the homepage.

The only positions open are in engineering. It seems like maybe content production or things like that, I mean, could be the educational front of things, which is all content, really. I agree. Yeah. Actually, that page is out of date because we've been pursuing directly some people on the marketing side. We actually have a VP marketing as of last week. And so that's very exciting. We're also hiring, we're going to start hiring for a head of content marketing on of last week. And so that's very exciting. We're also hiring,

we're going to start hiring for a head of content marketing on top of that for the reasons you explained. And the other trick we use is a lot of the people on our team who are engineers or have engineer in their title

also wear a dev relations or advocacy hat. You know, and there's all sorts of interesting variations you can have, combinations. I mean, Kyle, who's our solutions engineer, he's out there supporting production users and now customers, you know, pre-sales, post-sales. He's also doing a lot of, I mean, he wrote a lot of blog posts, guides, produced a bunch of videos because it turns out

if you have operational experience supporting customers at scale, you can come back. If you're the right kind of person, you can come back and package that into a story, right? I mean, Kelsey is like the ultimate example of that, right? Yeah. Dickram, who came in as our tech writer, is increasingly active on the engineering side also. So it's kind of a, you know, there's a lot of these interesting hybrid roles in early stage teams that I think are really fun and also really hard to hire for because they don't fit one neat label, you know, but, um, so it's, we share the burden of helping, supporting, educating, but yeah, it's, there's a lot work, a lot of work remaining. Yeah. Yeah. Well, the number one question I have there is what the heck is this thing? Tell me what that thing is. And I think, think you know part of this workshopping we did was you know finding the right analogy how to describe that

analogy and like even when you talked about the boxes and the you know standardized boxes like that started to make more sense to me you know and how it was a factory and what's inside the box is what dagger made essentially i mean that all began to piece together for me that's half the battle when you're starting to innovate on the edge like you are. You cannot describe what it is because it's not quite defined yet, you know? You know, we just passed a milestone that was running our investor update. And, you know, we count the number of pieces of content that were created, you know, blog poster videos basically about Dagger. And for the first time, our community of Daggernauts produced more than we did. Wow.

And I think that basically sums up our strategy for this. We're going to get better at telling the story, but it's just such a complicated and nuanced story that our website will never capture the full story. It's going to get better, just to be clear. But we're very dependent on our community taking that story and packaging it in their own words for their own audience and then going out there and telling it. And that's been the number one way we've grown and seen new users come in is through these early Daggernauts. And they go out there and they make

the story their own and then more Daggernauts show up, you know. And then so we're trying to enable them as much as possible to fill the gaps in our own storytelling. At least they got it,

so let them tell it. Maybe better than us, you know, or more tailored to their audience. Yeah. Are you proud of your docs? Like, should we send folks

to some like the quick start guide on your docs? Is that a good spot to send folks to? Yeah, I mean, dagger.io and then follow the signs, you know. But yeah, the one thing I would say is, if this is a topic that interests you in general, then I recommend joining the community, joining the Discord,

and saying hi and hanging out, independently of whether you have a use case for Dagger right now. Because that's really, there's's just you mentioned it before but there's an ongoing change right now in this whole devops landscape how applications should deliver it's huge it's bigger even than dagger uh and so if you're even remotely interested in where that's going come hang out you know we get to ask you questions about your experience of that and and vice versa you know very cool we'll get you back on in six months or maybe a year see what's changed maybe everything's changed maybe nothing's changed maybe uh i hope something's changed

yeah i think if your lines keep going up it'll be nice up and to the right until next time thank you i look forward to it. Thank you, guys. So I want some predictions. What's going to happen in six months? We're going to get Solomon back here in six months. Where do you think Dagger will be? What do you think is going to change between now and then? Should we even have him back on in six months?

Let us know in the comments, on Slack, on Twitter. Pick your flavor. We'll go where you go. Up next in the feed is Team Zoggin' Friends. I talked to my good friend, my new friend, Techno Tim, all about Homelab. We cover Proxmox, networking, Unify, lots of Unify, low-powered servers, being power efficient, the mobile Homelab that Tim built, and the business of YouTube. Make sure you check it out. Thanks again to our friends at Fastly, Fly, and also TypeSense.

And of course, those beats from the beat master himself, Breakmaster Cylinder. Those beats are banging. But that's it. The show's done. We will see you on Monday. Thank you. you