Software Huddle - Infrastructure as Code with Dax Raad

Episode Date: August 13, 2024

Today, we have Dax Raad on the show. Dax is a must-follow on tech Twitter, known for his blend of humor and insightful tech opinions. We talked a lot about SST, which is the infrastructure as code tool that he works on. They've got a new engine called Ion that's releasing. So we talked about what that looks like and how that's going to help users. We picked his brain on questions like, "Which cloud providers do you trust?" "Which services do you rely on?" and touched on topics like frontend development, databases, and more. We also talked about marketing to developers, he's got a unique take on that. So a lot of interesting stuff here.

Are you using PlanetScale? Is that right? Yeah. So for SST, our console uses PlanetScale, MySQL. Terminal also uses PlanetScale. The reason for this is we just love the workflow around schema changes. I think it eliminates a need for migrations, and it just provides a lot more comfort to making schema changes, knowing that nothing can go wrong. What do you see about what works for marketing to devs?

If you spend a bunch of time building companies. You know that when you work on product, you have to go for really big swings that have a shot at being huge. For some reason, when it comes to marketing, we forget that mindset and we just spend time on things that never has a shot of reaching a lot of people. How do you position SST? What's unique about it even compared to some of those other ones? Hey folks, this is Alex. And I'm excited because today we have Dax Rad on the show. If you're on tech Twitter at all,

you probably know Dax's bitingly funny humor that he has there, but also just like a lot of really good tech opinions. I think he's a really smart developer and wise about a lot of things. So I was excited just to get his opinions on a ton of things, including like, hey, which cloud providers do you trust? Which services do you trust? Front end stuff, async jobs, different things like that. Just understanding like, hey, where are you sort of seeing the lay of the land? We also talked a lot about SST, which is the infrastructure as code tool that that he works on. They've got a new engine called Ion that's releasing.

So we talked about what that sort of looked like, what those changes are like, and how that's going to help users. We also just talked about marketing to developers, because I think he's got a unique take on that. So a lot of interesting stuff here. If you have any guests that you want on or questions or anything like that, feel free to reach out to me or to Sean. But with that, let's get to the show. Dax, welcome to the show. Thanks. Glad to the show. Dax, welcome to the show. Thanks. Glad to be on. Yeah, for sure. So I'm excited to have you here. I'm assuming a lot of people know you,

but I think you're just like a top tier Twitter follow in terms of, you know, like legit, like strong, but legit good tech opinions and things like that. But also just like getting, you know, getting kicked in the teeth every once in a while from like some hard stuff stuff. So I'm excited to have you on like talk about some of that tech stuff. But I want to start off talking about SST first, which you're on the team for. So maybe for everyone, just give a little background on you and SST. And then we'll dive into some stuff around that. Yeah, so I've been an engineer for I think, probably, I would say my official start date, I could say 2012. So it's been, I guess, 12 years. Pretty much always worked in very early stage, either as a founder or like the first hire. Found a few companies, failed a bunch, etc. That whole thing. My history with building stuff as an engineer uh because i was at these early stage

companies there wasn't any division between you do application you do infrastructure so i kind of always did a lot of the infrastructure work and my approach to infrastructure was always how can i do like how can i maximize like what's out there so i can do as little as possible this can be most automated as possible because that's not my main that was never my main job uh despite that approach i never really discovered all the like real serverless aws stuff until much later i used to build in a very cloud agnostic way i think my approach was more i want to learn stuff once and then be able to like run it everywhere and that was my sense on that's how I lower my own burden. But around but in 2019, I was kind of bored of how I was doing things. So I tried the opposite extreme, let me like

really try to learn AWS as much as I can discover the whole serverless community. This project called SST was just getting started. The idea behind it at the time was really small. It was just, let's take some of the best components in AWS, wrap it up in a nice package, give you a good local dev experience. And you can pretty much build a lot just with 5 or 6 things that are in AWS. I ended up investing in that team. I ended up joining the team a few months later. And it's been three years now and our scope has massively expanded to helping you deploy literally anything.

But I think the twist for us is we use a lot of the tools that DevOps people and DevOps teams are very familiar with and already use. But we're trying to kind of trick application developers to start using those things. Because I think as much as infrastructure as code and things like that are well understood, almost no application developers even know anything about that stuff still. And I think letting them tap into

some of the crazy infrastructure that's out there so they can kind of fully build and deploy their app we think there's a large opportunity there still so that's what we focus on yep for sure and so like i guess i'm guessing people are familiar with some sort of infrastructure as code framework whether that's cloud information or terraform or something higher level like cdk or sam or servos framework i guess like what's unique about how do you position s guess like what's unique about, how do you position SST? Like what's unique about it even compared to some of those other ones?

Yeah, so obviously infrastructure as code has existed. We're nowhere close to inventing it. Like we don't actually build any of the infrastructure as code pieces. I think it's just the mindset with how these things are designed. I think most of these tools,

their primary concern is breath, like they need to make sure for every possible resource that exists in a provider, every single configuration option, there is a one to one mapping of of that in the infrastructure scope code tool. What the downside of that is there's no like on-ramp or like progressive disclosure with that if you want to go create a resource you are now seeing 20 options you don't know that only three of them are relevant like 95 in cases so for us we look at that's what that's one side of things we try to create like a nice on-ramp where you can create some of these resources we do simplify you know like do you necessarily need to know that when you create a queue in a lambda there's like this other resource in between that

integrates the two together and like all these little details yeah we don't think that a lot of people need to know that right away that said you if you build anything that is uh that grows and gets more complicated you will eventually need to get into those details and dig into them. So we like that bottom layer because it's really powerful. But we try to give you a nice on-ramp onto it. And the other side is, again, because these tools are built for DevOps teams,

they don't think about the dev experience because there isn't as much of a dev experience. It's more like, yes, you're deploying infrastructure. Yes, you're going to be making changes. But it's very different than working on an application every single day, which might need infrastructure changes every single day. But we, a lot of our focus is doing some pretty crazy stuff on like the local dev. I mean, local in quotes, more like just your development process and how it looks the other day. Yeah, when I think of early SST, I think of live Lambda and just like how big that was.

It was like, hey, you can be running your Lambda function code locally, but still getting almost the full Lambda experience in terms of IAM permissions and all that sort of stuff to work. I guess, did that drive a ton of early adoption like were there other things that were sort of like that that other other tools weren't doing yeah so there if you're trying to launch a new framework or launch any product really you need some obvious hook where people will see it and they'll be like i recognize that problem i need to drop everything and go try it. If we hadn't come up with live Lambda, we would have never been able to do anything else because there's no reason that, you know,

at that time there was framework was around. CDK was around and it was, it was early for CDK. But if you just enter the market with a framework that was similar and maybe even was better, it's not obvious. So live Lambda was huge. And even to this day,

I think what we see happens a lot is, especially on larger teams, a single person on a large team will get into SST and they'll understand the full breadth of why they like it and what it does well. But when they pitch it to their team, who are usually very skeptical, the thing that sells them to this day is a live Lambda thing. Because it's just such a painful workflow that's existed forever. And yeah, just being able to hit save on a file and then see the changes within no seconds is people that clicks for people immediately. Yeah. And local debugging and all that sort of stuff yeah it's pretty wild um was was v1 of sst was that on cdk or was that only v2 uh the first v0 was a fork of cdk and i hated that so when i uh when i joined i was like how can we possibly get this to not be a fork

and eventually it wasn't a fork it was just something that went side by side with CDK. You install CDK and you install SST. Then V2, we were just kind of refining our patterns that exist in it. And then we eventually ejected off of CDK, which is Ion, which is experimental initially. But now it's going to be promoted to V3. Yep. Okay. So tell me about Ion. What's the big difference there from previous versions of SST? So going back to this idea that all these tools are designed for DevOps teams,

again, with the idea of perfectly mapping to the underlying resources, if you look at CDK's approach, as fancy of a thing as it seems, at the end of the day it's a code generator it's a yaml generator it generates yaml for you um and that is like somewhat of a paradigm mismatch because you're programming code that looks like it executes sequentially it looks like you can have something happen after something else does uh but it can't because it just gets run once and turned into the ammo, which then gets sent to AWS to be deployed. And that mismatch leads to

an incredibly large set of problems. It's like the root issue for a lot of things that happen. I think with again, if you're a DevOps team, this probably isn't a big deal for you. But if you're an application developer, you do have a much more complex chain of things that can happen. You might have situations where you are...

A lot of these modern front-end frameworks, they do a lot of stuff with build time. So the build shouldn't happen until certain resources are deployed, like even your database. Sometimes builds depend on a database. I mean, I don't necessarily like that pattern, but it is just a thing that that is true now to generate these large static sites. So being able to wait on a thing being deployed before you do something that seems local like a build, that approach just does not happen with a code generator.

And we have like 100 of these issues that fall into this category. And almost all of them were in the initial... A new user trying out SST will run into these issues in the first couple weeks. And it was very painful for us to be like, yeah, I know it's a bad experience, but we can't fix it. Like it's a worse situation to be in. I realized we needed to switch engines that gave us a lot more control over how a lot of this stuff works. Like being able to make tweaks around. I think, again, like things like like cloud formation are really optimized for make sure a production deployment never goes wrong for like any company on earth, no matter the size,

which is very different than someone locally just working on something like you don't need that level of security. And so like being able like like own that process and have more control of it was really important for us so we ended up switching our core engine to palumi uh to be honest we didn't really understand a lot of what i just told you until we started using palumi uh because for a long time we just assumed oh yeah they're just also a code generator like they're just code to terraform type of thing it turns out they're not it's like your the program you write is actually what gets executed and like the stuff you write in sequential order it actually has happening in sequential order so you can really control

this needs to happen then that needs to happen yep so like you didn't you didn't understand that till you got in like did that unlock some things that you didn't even think were possible like when you're like hey we're switching to plumi for ion and then it's like oh man this is even better than we thought yeah so early on when we were like we definitely want to switch want to store switching engines in like a 24-hour window we were able to build a typescript to terraform generator it's pretty easy it's like not anything that that hard and we're like oh yeah this basically is like the underlying thing that we can build there a TypeScript to Terraform generator. It's pretty easy. It's not anything that hard. And we're like, oh yeah, this basically is the underlying thing that we can build everything on.

Very simple. And really randomly, the Pulumi CEO saw my tweet about something and just sent a nice message being like, oh, we like what you guys are doing. We've been following your stuff. And I was like, oh, you want to chat sometime and we started talking and i found out on the call because his co-founder joined uh luke who worked on typescript and i was like oh i didn't know that like someone that worked on

typescript like shifted to this space and they were telling me more about their stuff and i was like okay i need to like give this a real try before we go off and do our own thing. Then when I started using it, I was like, whoa, they solved so many of these underlying things. It's fundamentally a completely different paradigm. And it really wasn't obvious to me from the outside at all. It looked very similar to all the other stuff, but there's a lot of great ideas in it.

And it's super simple. Simple in the sense that once you get it, there's only like two or three concepts. And they compose together everything you possibly need. So once that clicked, I was like, okay, we got to use this because we're never going to build anything better. Yeah. You mentioned the CEO and CTO reaching out and helping you. What does Pulumi get out of SSD being built on it? Do you hook into some of their tooling in any way that would be like potentially helpful for them in

the future or what's that look like there was a meeting we had where i showed up was like why are you guys helping us so much because they were they created a slack channel like both of them responding uh like very actively um in there and i'm like this is like we love it and we appreciate it and this is amazing we're able to move way faster than we've ever moved with cdk but like we didn't really understand it um i think it's pretty indirect for them because we're not necessarily like directly driving people into their cloud product or anything i think they built this thing called the automation api which is a way to programmatically use Pulumi, which to me, I'm like, they literally built it for us. That's what it feels like. Who else really needs to programmatically

bring it? Yeah. I mean, it does happen. There are other cases. I see why they did it. But when they were building it, they were hoping... They really wanted to see the full vision out of something like SST would choose to build on this. So I think for them, it was just kind of seeing that vision through. I think the most direct thing I can say is there's a lot more eyeballs on Pulumi from an audience that doesn't really look at this stuff at all. There's more eyes on their providers.

I've actually seen someone contribute a bug fix to Pulumi from our community. So I think over time, there's just more mindshare around this stuff. So it's not very direct, but I think from that angle, it can make sense. Gotcha. Okay. And so I know Pulumi supports any Terraform provider. How much of Pulumi is using Terraform as the lowest layer, almost like CDK uses CloudFormation?

Or is there a lot of just Pulumi native stuff you can use Pulumi native providers rather than using Terraform at all? Yeah, so the Pulumi engine, like the infrastructure as code engine, is 100% their own thing. You can have Pulumi native providers. There's almost, like nothing major is really a Pulumi native providers. There's almost like nothing major

is really a Pulumi native provider. I'm sure there may be some that I'm forgetting, but most are adapted Terraform providers. So I think their approach was build an engine, but then just tap into the existing ecosystem of providers. But besides that, there's really not coupling there. Yeah. And is there like a separate state management than Terraform?

Are they using Terraform state management? No, that's all completely separate. They take different approaches to that stuff. Is there anything you miss about CloudFormation or CDK? Or are you just like, no, this is great? Well, it's funny because our uh our console is still a cd2 app and we had to go work on it a bunch to add support for ion but we haven't migrated it to ion yet so it was funny dropping back down into that and i think if i dig really hard i'm like what what uh what do i like what the straight answer is no.

It's just so slow. There's so many weird things that happen. There's just... Once you get used to Ion, it just feels like really taking a big step back. But it is a native thing in AWS. So you can go to the console and see your stack and see stuff. And that feels a little bit more secure than like a purely like client side tool

that's creating stuff in your AWS account. That's a big reason we have to go and work on the console because the console now becomes your source of truth for all your deployments, all of your state, what state every resource is in, which is that it's actually way better than CloudFormation, but there is like a safety thing that you feel when it's in

in your aws console yep yep um what okay so if i'm migrating from you know v2 to ion like how hard is that migration gonna be yeah it's pretty challenging we actually just wrote our migration guide um and it's like our first take on it. So using the migration guide, we're going to attempt to move our console over. And yeah, I mean, just like, just frankly, it's not,

it's not an easy process. The reality is, is you are a very different system. The one thing that makes it easy is Pulumi can import resources like without compromise. Whereas imports and cloudFormation was always like one difficult process. Also, random stuff just wasn't importable. Again, giving us the ability to bring anything in makes a migration at all viable. It's pretty different than...

Other migrations have been happening. People that were on... A bunch of people that are on serverless framework moved to SST. And they managed to do that without even import so it's going to be hard because it involves infrastructure but the important thing that this softened a little bit yep yep like what's your plan for v2 and like long-term maintenance like will you still sort of work on v2 understanding that like some people aren't going to migrate or like how are you thinking about that yeah i think uh v2 at this point we considered done in that there's like we just started to hit ceilings which is why we started to look at something else like anything that we continued to want to do any problems we want to solve which is not possible in v2 uh so we we keep up updating it for new cdk versions and fixing whatever is there but we're

not going to be developing any new features we're pretty good about merging and we're okay about merging prs but it'll probably be community maintained at this point um yeah and so when you did the v1 to v2 um like are there still a lot of people on v1 or did most most migrate and it's sort of done there what did that look like going through this before uh v1 to v2 was like not that crazy of a switch it was just a bunch of breaking changes in your code not so much like a full stuff you have to manage we just had introduced some new patterns and clean stuff up so yeah i think any anyone with any significant is on v2 gotcha and will console support v2 and ion for a while or will you sort of just like roll off v2 after a while yeah right now it supports both i don't see any need to drop v2 support we were able

to segment it in our back end pretty well i think the future of the console is that it's going to run in your own aws account and most of the work is going to run in your aws account i don't think we'll bring that to v2 i think that's going to be a an ion only thing gotcha what does that look like you know from a business standpoint if if people are running console in their account i guess like how do you all make money what's your plan there yeah so this is i think this is like a weird thing about us that's totally different from like traditional software companies we could actually bring our cost to zero uh right now, our costs really come from our console, which processes

your logs to handle issues. That's like the big bulk of our costs. But our new CI feature, auto deploy, that doesn't run in our account. That just runs on code build in your account. So we have zero costs for servicing that feature. And we really should move our log processing into your account as well. What's great about this is that there's a few benefits. Our costs are zero. Big companies need this anyway, because they don't want to ship their stuff to a random third party. So you have to do that anyway. And I think that allows us for some interesting pricing. So we're still charged for it. If you want want to fully self host it with zero help from us, I think that's always going to be like, we're not going to charge for that of course, but most

serious companies, even if they're running in their account, they want like some kind of guarantee. So that's where we'll charge, but it's literally a hundred percent like profit for us. We have no cost of operating it. So we really plan on driving these costs down continuously. Again, because our team is small. It just takes very little for us to keep going. So for us, whatever gets us more people using it, we'll do. Yeah.

Yeah. You say log processing. Is that like... Are you in the console? Are you, I guess, are you indexing logs and then they're like searchable in the console? No. So we're not doing that, which is why our logs feature is not great. Cause it's capped by what AWS can natively do. So I think we've done a better job than what you would get by going to the cloud watch logs. We we group stuff by invocation. We like pull invocation. We do as much as you can on top of the underlying capabilities. But for our issues feature, that one, we basically add a log subscriber. So anytime we deploy an SST, anything SST related,

we look at all the log groups and we add a subscriber looking for anything that looks like an error. And those events are sent to us. We analyze them, we'll apply your source map to it, and then we'll pull out what looks like a sentry error with the stack trace. And we even show you... Source maps get uploaded, so we show you the exact code where it broke. What's nice about this is it just works out of the box. There's nothing that... You don't have to wrap your handlers and stuff yeah just deploy and it just works um yeah so it's it's a useful feature and yeah we just want everyone to have that without having to go

sign up and set up the console i've seen you talk about this multiplexer that you've been working on recently like what's going on there what's the multiplexer yeah so this seems like a super random feature uh but the the idea here is when you again going back to the focus application developers when you bring up an sst app there's a lot of components there is the actual infrastructure that's getting deployed there's your functions just probably serving some kind of api and you might have your front end uh that also needs to be brought up, which might be like a VTAP or like a Next.js app, whatever it is. And you might have some like

additional stuff that's helpful, like a tunnel, like a Sage tunnel to your database. Just, you know, other tasks you need to bring up. And we found it really difficult to create an easy experience around that because it's like, you can't bring up your front end until you bring up SST because your front end needs to know about

values about where your API is. And there's this whole coordination, we had to open up several terminals and start things up in the right order. And we wanted to give you an experience where you just run scdev and everything your application is pulled up into a single window. But that required like really understanding how terminals work, which I had no idea until a couple of months ago. So we have like a nice switcher, everything gets brought up. You can add your own commands too. So I've been doing some interesting stuff where like, like drizzle the, the RM for JavaScript, they have like drizzle studio, which is, which you can bring up locally and have a database explorer in your browser. That gets brought up automatically. And all the values that

it needs, it gets injected automatically. So it's a really nice experience. And again, it's one of those things where solving that solved a dozen downstream issues and bugs that people run into and places where they get stuck or confused. Yep. Yep. And that's like, I imagine that was like pretty difficult. How, how did that take you to build that out? Yeah, it was one of those things where, uh, it was funny. We were talking about this the other day, cause once we got it out, we're like, oh, this actually worked out. But, uh, the process of it was, I kept vaguely talking about it, like every couple of weeks being like, oh, we should try that. Or like, I've been messing around around with figuring out how that could work.

And Frank was like, yeah, every time you brought that up, in my head, I was just like, there's no way this is ever going to work or it's going to happen. And to be fair, I also felt that way. It just is one of those things where you're like, oh, I can sit at this level of abstraction and build this. And you find out, no, you can't. You have to go one level lower. You do that for a couple of weeks.

No, you can't. You do that one level lower. So eventually you just end up hitting the bottom. But what's weird is the bottom is actually the simplest place to be because once you know how it works, you have full control over everything. So it was just a process of just getting pushed further and further down. And it was something I would dabble with every couple of weeks. When I had some time, a couple hours, I would mess with it and I'd make a little bit more progress in my understanding of how all this stuff worked. And then eventually,

there was a week where I was like, okay, I know for sure we can do this now. And I just spent a week just building it over and over and over again. And eventually, we got to work. I will say a big part of what drove us... And's funny i think a lot of product happens this way like we've been talking about this and thinking about this forever but always felt like too crazy or too difficult to be worth it and then turbo repo shipped it because they're one of the other products that have the same problem and that they don't manage a single thing in your code base they like bring up all kinds of stuff like front end, back end tests. So they basically have the exact same problem and they ship the multiplexer.

And once I saw that, I was like, okay, now we have to do it because one, you see it and you're like, well, we're just going to not have it now that we know for sure it's possible. But it was really fun at the end of the day. Like it was really, I think programming of the day. Like it was really... I think programming has gotten pretty abstracted these days. So this was like a nice reversion to like really basic stuff. Like when you scroll up, like you have to move these lines forward and you have to pop these lines out. And then how do you like...

It's like really basic. It's the type of like interview questions I think you would get, which aren't necessarily good interview questions, but it is like kind of fun puzzly stuff to work on yep yep cool um okay let's switch gears a little bit and i want to do just like tech thoughts i have a bunch of areas where i just want to get your opinions um kind of gathered up so first of all let's start with cloud providers i know you have like you have support for cloudflare you've you've talked about Cloudflare for a while. Is Cloudflare ready? How and where should I be using Cloudflare?

Yeah. So I think right now with Ion, because our big thing was we added support for not just AWS, but pretty much all providers. And we've done some nice things. It's funny, the place where it's massively used is we provide a really easy way where when you spin up, let's say, like a Cloudflare distribution, or you spin up a front end that involves a Cloudflare distribution, you can easily just say, actually, my DNS is hosted by Cloudflare. So everything still lives in AWS, but then the DNS and certificate validation is done with Cloudflare. This ends up being a massively popular setup because so many people use Cloudflare for just DDoS protection, like the thing that they're known for. It's pretty interesting because they obviously started that a long time ago. And these days, anything public-facing just gets hit.

No matter how small you are, you just deal with this problem. So it's almost become a fundamental thing. You need to have something in front of all your stuff. So I think this is an example of how a lot of deployments, even if they're primarily in AWS, eventually need to mix in. Even in a small place like that, need to mix in something else. So that's where we're seeing a lot of initial traction but then obviously cloudflare is trying to build a platform uh which you know build your whole thing on cloudflare is it ready yet for most applications i don't think so and i can explain a little bit why on that um but they they also

know this like they know i think they publicly say, we know building a platform takes 10 years. I think they're on year 8 or something like that. I think they're close, but there's a few problems right now. So I say primitive-wise, they have everything. They have a lot of really cool stuff. It's a very small surface area.

You can pretty much build whatever you want. I think the entire challenge right now is the database. So if you want to do fully Cloudflare native, your option is D1 if you want relational stuff. They have their KV store. It's nowhere near as robust or capable as Dynamo. So it's not really a comparison. It's more like a simple KV store not really meant for like primary application data they use d1 which is their like sqlite offering d1 isn't really viable for most apps there's like a 20 gig limit or something on it or like a 10 gig limit on it um there's all sorts of gotchas uh i personally don't know how much i believe in like sqlite underpinning like primary database offerings for someone like cloudflare but that is the option you have right now um but you know there are a lot of really

good database providers out there that you can use outside of cloudflare it's not a big deal right you can use plant scale you can use whatever all the different postgres hosts uh but the issue is almost all of those databases are hosted in AWS, plant skills in AWS, like all these big deployments are in AWS. So you incur a latency penalty, going from worker to AWS, which kind of defeats the point of using workers, which is like all about performance and speed and all that stuff. So today, that's the problem. It's not... The latency issue isn't like a geographic problem where they can never fix it.

It's actually there's... Currently, worker traffic goes to this older path. That's just less efficient. As they migrate onto a newer path, you'll start to see latencies that are pretty good. I think once they do that, I have a hard time saying that, yeah, you can't build an application there. That's just as good as, or better as AWS. Yeah. How do you like the worker compute

paradigm as compared to Lambda or Fargate or something else? Yeah. So I think obviously if you're coming from Lambda, it's pretty familiar, like the whole stateless stuff it's even more stateless i think with lambda you can obviously cheat by having stuff like outside your handler function um but workers you really it really forces you to treat it like more purely as just a function um obviously some benefits come from that the downsides are like you imagine imagine all the mental model problems people run into, like moving from a container to Lambda, just like wrapping their head around. Okay. Things are a little bit different here. There's a similar shift. I think going from Lambda to worker, cause it's just even more in that extreme. Obviously not all the node JS APIs

work. It's not really node JS. It's a compatibility layer. They're closing the gap on that quickly, though, and they're investing a lot there. So I think it's just going to get better and better. So those are the downsides. But the CPU billing, I think, is great. I think most applications are like 80% waiting on network anyway. So price-wise, it's just super cheap. A lot of the pain points I run into Lambda is just making calls to very slow third parties that just increase my bills for no reason. But you can do a lot of interesting things

like delivering webhooks, stuff like that. It becomes a lot easier. They are still... Some of their stuff is still in beta. So the things you need alongside your compute like queues and uh they have this new workflow thing which is like a much better version of step functions basically you just write normal code again still in beta so early uh but like when you actually try to use all these things together you get like glimpses of like how good it can be

so still very early very early adopter type thing but overall i'm like very excited about building something real on it yeah yep on that okay on that last note i've heard you talking about workflows being in code rather than like you know some dsl like um step functions do you just generally prefer writing stuff in code rather than like declarative like if i look at sst compared to service framework or sam and then also these workflows compared to step functions that's your that's your short go-to yeah i think our fundamental thing is we don't want you take we don't want to take you out of an environment that you're used to. So most programmers spend all day in their editor writing code. Editors have all this investment in them to make them good at doing a lot of things,

autocomplete, documentation, all that stuff is built in. We know how to collaborate on code. We know about Git. We know about pull requests. We know about branching. We know about testing. So I think there's a lot about code that's very very uh well understood and just old like it's old and and very established and i think with a lot of these new paradigms they can seem exciting and they like because they like show you how you can build the thing you want to but it's missing all

this other stuff that augments code um even basic things like obviously giving people ability to write abstractions often backfires but just hypothetically we know how to write abstractions in code we know how to progressively disclose in code much less clear or basically impossible to do that in config based things. So our entire approach is to keep you in the environment that you're familiar with. And we take this beyond just other like infrastructure as code stuff. We believe this about other infrastructure products as well. A lot of products try to simplify things by moving things into a dashboard. I think the classic example I can think of is is retool. So if you look at something like Retool,

the pitch is, here's a drag-and-drop UI thing in your browser that you can use, you don't have to write code, and you can build internal tools, etc. And that's a promise. Non-engineers can use it, and it's going to make your team more productive. What definitely always ends up happening is it's still kind of technical.

So the engineering team still has to deal with it. And now all of a sudden they're like in this completely alien environment managing stuff. So I think this shows up a lot, like a lot of infrastructure products are like UI based and you're suddenly jumping into something else to configure stuff. And UIs are really bad at dealing with complexity. They just are simple to start off with because they target a simple use case.

Over time, UIs always get bloated. Everyone complains about this as it probably gets older. You're just filled with stuff that you don't care about. But yeah, we try to keep you in code. We think code is very good. Yeah, yeah, that makes sense. Okay, you mentioned earlier around uh workers billing model and just billing on cpu i guess like do you see a lot of your larger

customers where the cost of lambda starts to be a problem yeah even for us right like our issue processing thing is like 80 of our bill 80 of our bill is just Lambda compute. And I don't know if that problem would necessarily be fixed by workers, but it is a significant thing. And I think what we are interested in is giving people a seamless way to start moving some workloads onto workers, even if their whole app isn't there and cloud for themselves, that that's also their like next milestone. Like they want to see people like progressively moving stuff there like i said obviously not everything

makes sense but i think it's very common to have a high bill from io8 and i think aws can also just fix this overnight if they just allow uh like lambda to serve concurrently i think that basically gets you the same thing. And maybe the attraction of workers just kind of goes away pretty quickly. I know they're, they're kind of already talking about that internally. But yeah, as of now it's,

it's, it's attractive for that reason. Yeah. What about, do you see people moving to containers instead when, when Lambda starts to get too expensive or are they still just like, eh, you know,

the, the ease of, of Lambda is lambda is sort of worth it yeah uh so a big thing with ion is we really focus on our container support and we're going to like continue to improve it uh it's actually pretty good now i'm really happy with it i'm we're using it for uh our terminal coffee thing. That's kind of what we built it for. And containers are pretty easy these days. They're not so bad. I think a lot of what's great about Lambda is more the integrations versus the compute model itself. Like very easy to put a Lambda at the end of a queue. A little bit more annoying to do that with a container.

And plus, like now you multiply that with all the services that exist. So, so yeah we do see people moving to containers after a certain level so for us again with our with our workload we've known for a while if we move to containers our costs would drop a lot but we just don't want to deal with that like it's uh we still like whatever extra costs we're paying for this event-based model uh we like the mental model and it does simplify a lot of things. Can you process log subscriptions with a container or is that only like how do you how do you handle a subscription filter? Yeah. So that ends up in a Kinesis thing. So you process. Yeah. So this specific case, I think, demonstrates what Lambda is good at.

So we receive a log event, and we find an error in it. And then we need to apply source maps to it. The source map is applying a source map is a very memory intensive thing. And some people's code is tiny, and it's not some people's code is massive. And it's like a big memory thing. The memory isolation properties of Lambda is like perfect for something like this, because we don't have to worry about some big company spitting out 10,000 errors and the container just focusing entirely on that and not dealing with other people's stuff. So the resource contention stuff, I think people always forget about this. But that's one of the main reasons why we end up using Lambda in a lot of cases. Yeah, for sure.

Okay, you mentioned better container support. Is that on Fargate? Is it like on ECS or EKS? Or like how is Ion running containers? Yeah, so we are committing to Fargate. And we have a lot of our work, like we've done a million times before, is just trying every possible configuration

to figure out like what is a good default. So with containers, there's a combination of things. It's like, what's the best VPC setup? What's the best ECS setup? What's the best service discovery setup? There's many ways you can configure all these things. So we want to create a situation where you can create a cluster, you can very easily add services, which we can do today. What we're working on now is those services should be able to discover

each other in a sane way. A demo that we really want to do is we want to show how you can deploy an Elixir application. When you scale the Elixir application, it automatically clusters with all the other containers because Elixir has clustering. For people in the Elixir world, setting up clustering is so powerful but a lot of people don't do it just because it involves a lot of infrastructure know-how um so yeah just like going into each ecosystem and figuring out what's the best container experience we can deliver yeah yeah okay um okay switch it up a little bit what about i know like event driven applications have gotten

sort of popular and things like that, I guess, like, where do you see event driven and especially do you use a lot of event bridge? Like what do you, what do you think of that verse more like SQS or just like straight Q based stuff? Yeah. So we still really like event bridge. Uh, we have our type safe bus implementation it's in ion and just not documented yet, but it's one of those most, it's one of those things where we famously have undocumented two things are off and our event stuff. And it's become a running joke is like every single day someone is like, I want to move, but I'm waiting for this. I want to move and I'm waiting for this. So people still really like events. I think we tend to use them in a very light way. And I think that's our general

pattern with everything. We like this stuff, we use it in the lightest way possible. So we emit events, everything into an event bus. For terminal, our coffee thing, I'm trying this new pattern where we only have a single subscriber and there's a big switch statement in it that just switches between different events. So obviously we lose some observability benefits but i think for a lot of applications like this is good enough because what they're looking to do just offload uh async work to something um and having to go set up a new subscription and new patterns and all that for every single event is a lot more tedious than just going and adding another case in their switch statement

so uh yeah we still like it but we don't we don't go crazy with it what like what do you when you said ion publishes a bunch of events like what do you what do you do and react how do you react to those like what are you doing with those events uh i mean so what we meant is we have a framework to define events in your application and the shape of them using Zod or Valibot or any of those schema tools. And then your consumers will get them in a validated with the right shape. And it's all type safe and all of that. And we found when we gave people those tools, like they use it a lot. So people do like events.

OK, OK, gotcha. OK, switching again. I used to hear a lot about tRPC. I feel like I hear about it less. Is tRPC still this thing? Where's, like, do you like tRPC? Where are we at with that? So I actually never really used it.

We have a lot of people that do use it. I was helping someone yesterday that was studying with tRPC thing. What I've been using a lot for my APIs is Hono, which is a very, it's like an express alternative, but it runs everywhere like it runs in lambda runs in cloudflare and it's crazy performing crazy fast crazy lightweight it's it's really cool and they have this cool like a trpc like thing built in where they will export they will give

you a type that you can import into your front end and that gives you like this type safe client on your front end off normal rest endpoints so i'll just get posts etc uh which is very cool and they also have native open api support uh which we're using for terminal uh which is it's very very good and open api stuff is great so we're generating uh go sdks off of that. There's actually even going to be a public API for buying coffee. Nice, nice. A much more user-friendly way than the current way, right? Rather than SSH.

Now you can just use a REST API to order your coffee. Yeah, cool. Okay, what about databases? I know, I guess like what sort of database are you using planet scale is that right yeah so for uh sst uses our console uses planet scale mysql uh terminal also uses planet scale the reason for this is we just love the workflow around uh schema changes i think eliminates a need for migrations and it just provides a lot more comfort to making scheming

changes knowing that nothing can go wrong. Plus all the observability stuff that they do, plus all the, uh, like the insights stuff they do, like overall it's interesting because the reason they exist is to handle these like top 0.1% scale, uh,

sites. But as a side effect, they've just built a very good platform for even stuff that isn't at that scale. So that's why I use it. That said, I feel that database space is still very unsatisfying. I think I didn't plan to get a great option for my sequel. I don't think there is a great option in AWS. So we not looking for the freest option ever. We just look for reasonable situations that you can start and grow with. Whereas in AWS with RDS, it's pretty expensive when you add in the VPCs and then that, and the, and then your database on top,

uh, like the entry fee is pretty high and that just puts a, it just creates friction for new people adopting AWS and like learning. And there's all kinds of problems with it. Uh, I know they're going to fix it, but yeah, right now there's,

there's a gap. Yep. Yep. Do you trust any, I guess, are there any other database providers outside of AWS and outside of PlanetScale that you feel like are doing pretty well or that you would trust? Well, everything I've been talking about is only on relational databases.

I still talk about that because for mass market, that's what people need. The answer is no, honestly. Uh, I think there's a lot of like traditional Postgres hosts that are, that are great. Um, like crunchy data or like even super base.

Like I think just giving you a straight up Postgres instance that you can rely on. I think there's, there's great ones there. Uh, but something that like planet scale feels kind of like next gen where it's like, like a new architecture, more focused that like planet scale feels kind of like next gen where it's like, like a new architecture,

more focused on like being able to connect to it from Lambda and workers, which, you know, you might have a thousand of them running and the connection point, all that stuff is just, just fixed and hidden from you. Uh, I don't feel like there's a lot on the relational database side.

Uh, it still feels like a big gap and there's a lot of reasons for it, but yeah, it's, it's, it's been a pain point. Yep. Yep.

How do you feel about just like my SQL generally and maybe the direction of that? Does that make you worry at all? Uh, it's weird. Cause I, I wasn't really a SQL person for a long time. I worked on pretty much all of thoseSQL stuff for most of my career.

Then I did a bunch of Postgres stuff. And I was like, okay, I get it. Then I switched to MySQL, which has almost no features compared to Postgres. Initially, that was annoying. But over time, it kind of felt like Dynamo, where there's a very small feature set. And you just use it correctly. And you're never going to have any surprises. So I really like that attribute of MySQL. And it's just used by like the largest deployments in the world. And that makes me not worry about it.

It's so old too. So yeah, I don't really worry about anything from that regard. I think Postgres, I've gotten to the point where it's just too much for me. So I try to stick to MySQL where I can. So do you think like a lot of the Postgres energy is just because like it's saying yes to everything

and it's like has a ton of features and things like that? Yeah, it's great for application developers. Like if you're building an application and you want to do more stuff in the database,

it has like an extension for everything. It's got like just so much um and i think that's a trade-off that they made where my sql is really focused on making it easy to operate and focus on teams that were operating it uh and then they kind of slowly chase postgres on like features application developers want um but yeah that said like you know if you're using postgres in a normal deployment there's a lot of stuff you can tap into uh it just comes down i think case i just have like i said i like code and i like uh i think people are good at managing code so i have to do some more stuff in my application than in my database i have tried

the extreme of doing a lot in the database and i see the upsides of that, but it just doesn't really click for me. Yep. Yep, for sure. What about, so I know you were like an early proponent of Drizzle. What do you love about Drizzle? Yeah, I think it's been crazy to see their growth. It's, they're like getting integrated into like all these big companies now. It's pretty wild. It's just, you know, a team of three uh yeah i think what my sense is i like uh query builders over rms um

which you know isn't like an objective thing some people like rms which is fine uh but i believe the right abstraction has a query builder at the bottom like something that helps you build something that looks a lot like the raw sql in a type safe way helping you where you can um i think they got that right i think they got their adapter pattern right so that it can work with all kinds of stuff like it's been it's used being used from anything from like sqlite to like uh you know the postgres to my sql it's a very specific deployment of uh like htc Postgres, you can do like their their adapt layer is super powerful, and it's scaled to all these different scenarios. So early on, they had both of those, but like a lot of features were missing.

But because they had both those, I knew that they would just grow into go into the rest. And they have and they can build now an ORM like like a Prisma-like ORM layer on top, which they have built. They're still iterating on that and reworking. But again, this layered approach of raw stuff at the bottom, as raw as possible, and build up to the higher level stuff. So when you need to drop down, you can. I think especially working on SST, that's a thing I started to care a lot more about

because I just see how people have to eject out of things all the time because they eventually outgrow it. Yeah. Yeah. You mentioned SQLite a few times. Like, what do you think about the SQLite moment that we're, that we're seeing? To be honest, I don't, I'm not super into it. Uh, so I think here's a pattern that I follow. Uh, there was, I did one product where we didn't use a traditional database we use something called lmdb which is like sqlite it's like a single file on this super super super fast very limited in what can do but really really fast uh and we had a deployment curve per customer so every customer got their own file um a lot of upsides to this. Performance was crazy. It's basically almost like it's in memory. A lot of it gets loaded into memory.

And simplicity, etc. But the amount of tooling we had to build around it was kind of crazy. Okay, you have this. Imagine these different things. Okay, but it's a single file. You have to make sure you back it up. Okay, now someone wants to do analytics across all of your customers data. Cause you have that internal need and you need to replicate it into some kind of, you know, a lap thing. Uh,

obviously there's no out of the box things that will ship like a file to one of those things. Uh, you need to debug something. Like some data looks messed up in one customer's instance. You can't just go open a client and connect to it. You have to now go like SSH into it. So I feel like SQLite itself, it's incredible because it's meant for cases where a single file is enough. Embedded devices, like all the situations where it is massively used. But now when people are trying to use it to underpin or to replace a traditional database, they're rebuilding all the stuff that traditional databases have.

Okay, instead of it being a file, it's now a hosted thing wrapped with an API. Okay, now you've lost the simplicity of just a file. It's now another service that can go down. Okay, you need backups. You have this whole thing called Lightstream and replicate it into S3. So I think, yeah, you can get all this stuff working, but I see it just re-approaching more and more a traditional database without the ecosystem of it. There's way more things that can plug into your MySQL database and ship it somewhere else and do ETL and all that than there is in SQLite.

So yeah, for me me the stuff that i work on it doesn't meaningfully change stuff um i'm sure there's some cases where it does but yeah i just slowly see them rebuilding a traditional database yeah very good yep yep interesting okay what about the the front end world right now how do you feel about our options in the front world like are you totally off react is that right yeah i haven't used react in like almost three years now okay sounds and so what are you are you astro or what are you using no i use i use solid js everything we've built has been on solid it's just something that clicks for us and i think it's really well designed i think it really massively simplifies a lot of stuff that goes wrong in react apps in fact i didn't

really understand what was going wrong in all my react apps until I started using solid. Uh, I knew stuff was going wrong. I didn't get why every single one was eventually becoming a mess. That said, I think we're in this really like crappy phase right now. I think there was a time where I feel like there's moods to our industry.

There's a collective mood where that different times we collectively are open to different things. Uh, I think there was a time where we were really interested in, okay, like react has brought these new ideas that are now like spawning a bunch of other new ideas, really into exploring what can Svelte do,

what can Svelte do, what are different approaches to these things, what are these different frameworks? And there was like a lot of other new ideas really into exploring what can svelte do what can svelte do what are different approaches to these things what are these different frameworks and there's like a lot of energy around that stuff uh and i think it's taken longer than people had the patience for for that stuff to stabilize or like even get adoption maybe they didn't get as much adoption as people wanted and the ecosystems around them etc but i feel like now we're in a phase where the mood is i'm like tired of experimenting like i just want like stuff to to work and be straightforward and have the tools i need uh but that like doesn't exist anymore because react is also in in this like crazy phase where they're like trying some wild different stuff um i don't think it's going well so i think my sense of it is everyone's kind of unsatisfied and that like they just want to be able to do their stuff and there isn't really

anything that's uh that boring in this space right now interesting okay do you think you said like the react stuff's not going well like do you think server components i'm like is it going to be a major part of how all apps are built in two or all react apps are built in two or three years or do you think it's going to like sort of fade a little bit and you know some people use it but it's it's most people are going to be doing more spa type stuff yeah i think there's there's definitely interesting paradigms and if you look at what's in react server components are interesting and there's definitely places where it's going to be really cool um i think then bringing that to people through next there's like that whole process i

think next has done a horrible job with that um i think their implementation is really confusing really buggy really hard to understand uh and if you just look at people trying to adopt this stuff it's not it hasn't been going well um and then you look at people trying to adopt this stuff, it's not, it hasn't been going well. And then you look at, okay, what benefit are they actually getting? Like I'm not really sure about that. So I don't, yeah, it's like, like I said, we're in this weird phase where that, that is really experimental, but it's pitched as this is definitely the future of web. I'm kind

of skeptical about that still. Like I, I build, like I work on web stuff a lot. I do a lot of front end work. I haven't really seen how it's going to meaningfully change the parts of the place where I, where I struggle. Um, again, I work on productivity applications that people use for several hours a day when they're using them. Quite different than the heritage of Next.js, which is e-commerce sites, things that are short-lived. They really want to pitch the idea that it's great for everything, but I don't know. I don't really see that. The stuff I'm excited about on the front end really has something to do with rendering it on the server. Yeah.

What are you excited about on the front end? Yeah, the thing I talk about all the time is this local first stuff. So that's, I think, that's addressing a lot of the real problems that I have building apps that feel good to use. The SC console is fully local first. I wouldn't say it's a great implementation of it.

I think it's like good for our first experience doing something like that with it. But yeah, for productivity applications, stuff just needs to feel good and stuff needs to feel fast. And the whole model when you switch to local first is actually a lot simpler. So Liz is... My wife is learning. She's been getting way more hands-on with building products. She's actually coding. And she's entirely focusing on front-end. But what's great about our local first setup there is

all the data is just there for her to work with. She's not learning, how do I fetch from the server? What API input? She's not redoing that with every feature. The data is just there and she can access it. It has a much simpler model. But I think the results are quite good. I think it's still pretty hard to build a local first app.

There's a lot you need to know and set up on a back end and the syncing. And you run into distributed systems problems, basically, because you're thinking. It's like you're treating your browser as another node in your database. But that's what's more exciting for me. I think the results really speak for themselves but uh

yeah this this rsc stuff this next stuff it's just getting so so so so crazy complicated uh when i look at next before all this stuff like the pages router and all that stuff i think it's a good framework i actually think it's a good framework i would never make fun of it now i make fun of it every single day because it just, I think the whole industry is like lost on this. Interesting. Like, how do you think that's going to shake out? Like, will they, like, will there be some sort of backtracking there? Or is it just like, man, we're going to like forge through this all that stuff and eventually it's going to get ironed out and it's going to take, it's going to take a lot of pain on the way.

I think when you reach default status, it's very hard to screw it up even if you do all the wrong things so react is a default choice and next is a default choice for react apps people just will always pick it and just do whatever the docs tell them to do and not really think too hard uh we talk to people that when we ask them what they're using uh they say react, but they mean next. And they don't really even understand there's a difference between these two. They just think next is react. So achieving default size for something is a crazy accomplishment. It's awesome to get there because you literally can't screw it up. So I just feel like, yeah, they're going to keep going and people are going to do it. I think it's going to be a lot of...

I feel like nothing really is going to happen. Nothing's really going to matter. It's just going to be a lot more energy spent on this stuff than is needed. Yeah. Do you, at SST, do you get a lot of Vercel migration? So a lot of people come and say Vercel's not working for whatever reason. And we basically want to run our next app. I know you have Open Next and all that stuff. Do you see a lot of people come in and say, Vercel's not working for whatever reason. And we basically want to run our next app. I know you have Open Next and all that stuff.

Do you see a lot of energy from that? Or is it all over? Yeah, we do see a good amount of that stuff. I think when people come to us with that, our first reaction is like, are you sure? You maybe should just stay on Vercel. Next really only works properly there.

We do our best to get it working elsewhere. But that's the thing you're going to choose next. You're best to get it working elsewhere. And but like, that's the except that's the thing you're going to choose next, you're choosing to use Vercel like it's kind of the reality of it. So we try to get people who try to really understand why they're moving. Oftentimes, it's like kind of ridiculous costs, like causes just make no sense at all. But I think a lot of it is just as much as the energy around next is like, this is your application. This is the biggest part, most important part of application. For most companies, it is like 10% of their application.

And then the most important parts, their backend is not anywhere near Vercel. It can never be on Vercel. It's more like it's more begrudging being like we don't want to just move like just one part of application into. Into Vercel while the rest of it somewhere else and all disconnect again, switching between mental models. It's not it's not great. So when we see people deploy next year through SST, it tends to be more because they have a big back end somewhere else. Gotcha. What's the status on open next like does that require a ton of like active work or is it like pretty well stable or something i guess like yeah what does that look like uh it's a very tedious thing but it's very it's like very community run at this point. For a project like that, it has to be because it's all about handling edge cases in different scenarios.

So there's a community of people that use it with every release they find. With every new Next.js release, they find some things that break. A giant test suite that tests all these random cases that we've seen come up. And just figuring out workarounds. A lot of it is just documentation where it's always like, I'm getting this error. And then we explain to them, this is happening because of this. There's no solution.

It's usually something like that. There's some weird stuff, man. It's like, the thing that really gets me is if you use Next.js, either follow docs and use it and deploy it to a container, you'll start to get errors that you should only see on Cloudflare. Like you deploy it to a Node.js container and all of a sudden you're seeing like error.

You can't use this API. You can't use this Node.js API. Like you can't use process. You can't import from process. You can't use process.env. And that's like so bizarre as to some, again, these people aren't like tapped into the latest news.

They're not tapping into Vercel. They're just picking next as default. And all of a sudden they're like, what do you mean? I can't use process.end. Like, how does that make, it's impossible to explain to people.

Right. But again, it's because next is very developed around, uh, Vercel's setup. Um, and like, yeah, like they don't, they run their middleware in cloudflare so they like hardcoded all these constraints which make no sense for the average user so it's kind of what i mean by complexity it's like none of this is worth it it's not worth

it for the average person using next uh like you get exposed to all this like weird stuff that's going on yeah yeah okay are there any non-aws non-cloudflare infrastructure providers that you use i guess planet scale planet scale yeah i'm trying to think um i think for all of my projects the thing that i've added is cloudflare and planet scale and then everything else is aws uh so for myself no um we started seeing people start to use azure really which maybe it's just for some of those ai stuff that they have uh but someone was asking us for an azure because we need to store your state somewhere and right now we can store an aws or in cloudflare i was actually the other day for azure state so people are using it for other things uh

gcp basically never gets mentioned like at all it never gets brought up azure does get brought up literally never no one's ever asked us about that um and then any like third-party sass like you know clerk or ingest or like all those sorts of things, like not those specific ones, but like any other things that you use, or you're just like, Hey, I want to use the, the primitives on AWS mostly. No, I think our entire positioning. So our like positioning is, uh, I would say a contrarian bet around where all the investment is going.

So if you look at these unbundlers is what we call them. Like ingest pulls out one feature AWS and just focuses all their resources on all the investment is going. So if you look at these unbundlers is what we call them like ingest pulls out one feature AWS and just focus all their resources on it. Resend pulls out email focuses all their energy on email. But there's a ton of investment in that category. A lot of it is because Vercel got so big. And people see this as like investing more into that ecosystem. We're basically taking the opposite bet. We're saying that all those will fail and any companies that would give them revenue will eventually eject from those things. And there's so many reasons why we can argue that we might be wrong, but we're basically on the other side of the bet of saying all these one-off

SaaS things are like, they're all they're all gonna like fail together in the same way yeah interesting do you think is vercell too big to fail or like will vercell kind of fail as well i don't think they'll fail i think they will always be around but I'm skeptical that they're like a public company. I don't, I don't see how they, and from, from the behavior I've seen in the last year, it seems like they've brought on someone that's shaping them up to try to go

public. The giveaway is like a CFO or something like that. Like that's been in a public company. I don't know about any details, but the thing that sticks out to me is all these tiny little pricing adjustments for the, unlike the most trivial places where like, they'll, they'll like, they'll like, they'll be reconfiguring a bunch of stuff to make their financials look better. That's what, that's what it screams to me. I don't really see how the public market, especially right now is interested in them. So yeah,

I don't think they'll die. I just don't see how, you know, they get to the next level. Yeah. Yeah, for sure. Are there,

are there AWS services you won't use? And like, I'd say specifically like popular ones that some people do like, and you're just like, no, that's a hard no for me. Except functions probably. Except, yeah. I don't know how popular that that some people do like, and you're just like, no, that's a hard no for me. Except functions probably. Except.

Yeah. I don't know how popular that that is, but, uh, and I've tried a bunch, had a lot of energy for a while. And it's just like, yeah,

it's so, it's a confusing, it's a tough model. I think. Yeah. Yeah. I think my sense around all this stuff was when I was getting into the AWS stuff,

I think that was around the time where it was like peak, all of that, like all of this, like really, it was intense ways you could build stuff. I think that was around the time where it was like peak all of that, like all of this, like really, it was intense ways you could build stuff. Yeah. So I have to learn like the direct service integration, the API, all that. I remember Adam was using a, it was a VTL. It was like whole application was like VTL templates or whatever. Yeah. Yeah. yeah yeah so it was like that extreme and i think it was cool to see like what that was like but i think now we've shifted back to okay like a lot of infrastructure parameters are great but like use them in a dumber way like don't

try to like have all your business logic live there again Again, people still do it the other way. But yeah, for me, it's just like anything where my business logic or the code I'm writing is kind of existing inside AWS space. It just doesn't work out for me. Yeah. Yeah. Yeah. Switching gears a little bit on just like marketing to devs. I think you've had some strong opinions in the past. You're just like, a lot of people are doing it wrong. I think SST, like you yourself and then SST has done it well. I guess, what do you see about what works for marketing to devs? Yeah, I think the root insight we had was not all, but if you're pretty...

If you spend a bunch of time building companies, you know that when you work on product, you have to go for really big swings that have a shot at being huge. Most of the time, you're going to be wrong. But there needs to at least be a plausible shot that this is something that's going to end up in the hands of a lot of people. And that's how you focus on product. For some reason, when it comes to marketing, we forget that mindset. And we just spend time on things that never has a shot of reaching a lot of people. So for us... And this is so hard.

Even though we've been thinking this way for years now, we still revert to doing the stuff that can never be big. It's just a default state of being. If you look at a lot of dev tool companies, they launch a new feature. They will publish a 30 minute video going over using the feature. And that will be seen by, I'll get a couple hundred views. That no matter what you can execute on that,

like amazingly, it's never going to be something someone sees and like sense their coworkers being like, you have to check this out right now. Right. Uh, and there was a shot of, uh, at being huge. So for us with marketing, we're like, let's try never doing anything that never has a shot at that. So it's only try to make stuff that people will share. Like we don't always succeed. I think often we fail, but it could have worked. So I think a lot with marketing, top of the funnel needs to be the same way any other company markets even outside of DevTools, which is you go for the biggest numbers possible. I think there's this habit, again, like I said, we revert to it too, of just

putting out release notes, putting out a blog post, summarizing what's in the release. Like you're not, you're never reaching new people with that. Like no one's excited to read that. No one's sharing that. Um, but I think like having that click was pretty big for us. Um, and the other side of it is I, I really don't think this is a full-time role. I struggle to imagine, I showed a point to anyone that is purely doing this. That's not heavily working on the product. Uh, because if you go further down the funnel, a lot of it comes down to credibility and trust, uh, the reach we have. I talk about a lot of things. I don't just talk about SST. I talk about like all kinds of things

that I spent a lot of time in and people follow me for those things. And hopefully, they develop trust with me because I'm genuinely talking about this stuff. I'm not just talking about my product every single day. I say maybe I talk about it like one in five. And it's not artificial. It's genuine. I only want to talk about it one in five because all this other stuff I also want to talk about it when in fact there's all this other stuff I also want to talk about. So it's hard for someone to like their role just to be marketing and they're not spending every day building stuff. They're not spending every day working on their own product. It's really hard to come off as an expert, which is what people really care about.

Yeah. Yep. For, for this new iron release, you're talking about making a video. Will it be like a, like some of the funny videos you had in the past like free next js and things like that or like what were you what's that

gonna look like uh so we did it so when we launched ion we did like i did like a three minute video where it's like a fake documentary thing where i like really do a poor job selling ion on purpose uh that was like our funny take on it so this this one, we're actually not going to do that. We're going to do, again, with this terminal coffee thing, weirdly, it ended up using like every possible feature of SST to the extreme,

which makes sense because me and Adam worked on it, of course. So this is going to be a little bit more traditional in that we're going to go over how we built it and like every single detail all the way from basic stuff like the API and the front end, but also things like our internal dashboard and like how we set that up and our operations and all that. So I'm going to try to make a video of like, here's this fully crazy thing we built. It uses AWS, uses containers, uses functions, uses Cloudflare, it uses workers, it uses like really stretches everything we can do. So I want to do an in-depth

explanation of that code based on open source, all of it. Again, we don't expect that one to do that well, but that one's a little bit more traditional. But you still see how it's not just a straight up explanation of our thing. And it's like, yeah, it's not just a straight-up explanation of our thing.

And it's not just like, hey, we have this feature and this feature and this feature. It's like you're nerd-sniping a little bit and saying, this is how you actually build something. And I think people will like a walkthrough of an actual codebase, especially something as unique as Terminal. Yeah, it's like, we built a coffee shop in a terminal. That's a little bit more interesting than the traditional application yep yep um how you used to stream some why did you stop streaming is it just like too time consuming or like not affected as effective at like sort of marketing type stuff or yeah no i i miss streaming and i need to get back to it i don't i think the thing with streaming is i think it puts you into a good rhythm and mindset. So when I was doing it every single day,

I felt just very connected with a community, not the biggest community, but a community. And that like feeds into the other places. It's like your biggest fans show up there. Uh, and it's weird to this day. Like I was talking to someone from Experian that a massive company, you massive company, so many people just all over the world. And one of them was like, Oh, are you going to start streaming again? And I was like, this was really surprising to me because they're the most corporate...

It's like the other extreme side of tech that you never see. And this person watched my streams and mentioned it. So it's good in that like it's the most intimate thing you can do uh and people and people do watch if you know they might have it on the background etc the reason i stopped is i can only do this with certain kind of work uh cranking on ion and ion internals multiplexer like i'm not good i'm not so like i think prime could do this where you can work on something crazy and also be streaming i think i'm okay at that but i'm not good i'm not so like i think prime could do this where you can work on something crazy and also be streaming i think i'm okay at that but i'm not at the level where i'm

going to be that productive doing both uh i think of adam who's who's at the other extreme where he just cannot do both at all but he still does it uh so i'm going to get back to it once we're done like with this core ion stuff and it's more like me working on our console or me moving the console from v2 to ion that's like stuff that i can stream now as i said the last thing is uh the reason i started streaming is because i just committed to doing it every single day uh because it gets you good at talking and thinking at the same time with like all the distractions and i found that i was just a lot more comfortable in all kinds of situations after doing that so i want to get back to it because

it's like it's good practice for for all that yeah interesting do you think like has street has tech streaming gone down other like obviously prime is huge but it seemed like there was a moment where like a lot of people were streaming and now there's there's fewer i don't know if that's just like my circle of people that i knew or like has it sort of dropped off a little bit i think when prime got big and a bunch of people in that world got big there was like a bunch of interest which is kind of when i jumped onto it i think a bunch of people jumped onto it uh so yeah i think it's there's definitely yeah i feel that phase is over um again it's like one of those things where it's a lot of effort and you're only gonna it's only gonna be big if it's like your full-time job like it is for prime

uh so it's like you know only a few people really are gonna end up there but it's still growing like his numbers are are crazy like now he's uh doing stuff on instagram and he's gotten like insane numbers on instagram now too so yeah there's a lot of engineers in the world. Yeah. Yeah, for sure. Cool. Hey Dex, thanks for coming on. This has been, this has been great. Um, yeah, I love to learn all this stuff for you. I guess like where can people learn more about Ion? Where can people find you? Anything else you want to, you want to plug? Yeah. So, uh, our website st.dev we're working on

a new landing page today i think we will maybe get it out uh nothing crazy it's going to be like a single pager um and then me on twitter thdxr you find me there um as alex warned you at the beginning uh you will be insulted like three out of five posts. I will be probably insulting you, but the other two are maybe serious. It's good stuff. I love it. And also like your podcast with, with Adam tomorrow.

Dot FM. I always forget to plug that. Yeah, it's good. I was always, I was always missing being like, remember the podcast. Yup.

So there you go, Adam. I got it for you. So cool. Dax, thanks for coming on, you know, best of luck with,

with ion and all that stuff. And yeah, I'll see you around. Cool.