Software Huddle - Just use Postgres with Craig Kerstiens

Episode Date: February 13, 2024

Today's episode is with Craig Kerstiens, Craig has been in the Postgres space for a long time. First at Heroku, doing Heroku Postgres. Then at Citus, doing Distributed Postgres. Now at Crunchy Data, he's Chief Product Officer there. He's done a lot of Postgres advocacy and a lot of interesting stuff. In this episode we'll talk about the Postgres ecosystem, some of the Postgres features, some of the naysayers about Postgres, and just get Craig's thoughts on those.

How did you fall in love with Postgres so much? Or maybe like, why are so many people falling in love? Why is it so popular, you know, all these years later? It just works. It's a really good database. It just does 80, 90% of everything I want it to correctly. And I don't have to worry about it. If you can invest in one company that's not the company you work for, who would it be?

I want to find the companies that are not using this shiny new database that came out last year that are using Postgres, that are using Tailwind and Django or Rails. And like that's your tech stack. Like that's how I'm going to invest. Which person influenced you the most in your career? Hey, folks, this is Alex. Today's episode is with Craig Kirsteins, who when I think of Postgres, that's who I think of, right? He's just been in the Postgres space for a long time.

First at Heroku doing Heroku Postgres, then at Citus doing distributed Postgres, now at Crunchy Data. He's chief product officer there. He's done a lot of Postgres advocacy and just a lot of interesting stuff. And I want to do a whole series on Postgres this year. This is the first episode where we'll talk about the Postgres ecosystem, some of the Postgres features, some of the naysayers about Postgres, and just get Craig's thoughts on those. So I really enjoyed this conversation. I hope you will as well. If you have any comments, feel free to reach out to me, to my co-host Sean Faulkner. If you want comments on the show, on the series, on anything we can do

here at Software Huddle, feel free to reach out. With that said, let's get to the show. Craig, welcome to the show. Thanks. Glad to be here. Yeah, I'm super excited to have you on because I want to do a whole series on Postgres this year because I feel like, you know, when I got into tech, Postgres was super hot. And somehow, like all these years later, it's still super hot. I get one day base of the year again from dbingens.com, which is the fourth time in seven years, which is really interesting. So I think there's no one better to talk about Postgres, the whole Postgres ecosystem, all that stuff than you. So maybe to help folks understand why, could you give a little bit on your background, what you're doing, where you come from, things like that? Yeah, maybe I'll start on where I'm at now and then kind of back way up because I think there's some kind of interesting history there.

So I'm currently at Crunchy Data. We're kind of everything Postgres. I think when I joined, we would say we were like the red hat of Postgres, do a lot of on-premise support Postgres. I came on board to run product and engineering and help build out CrunchyBridge, which we'll probably get into at some point, which is, you know, I kind of joke that it's unfinished business from my Heroku days. Heroku was very early on in the database as a service landscape. And we'll probably get into it a little bit more. But I'll back way up, right? So I came on the board to Crunchy about three years ago.

If you need Postgres, come talk to us. We'd love to talk to you. But if I go way, way, way back, shortly after college, came out to the Bay Area. I'm originally from Alabama. There's not a lot of high tech startup scene there. Um, there's a lot of defense like NASA and defense work, but was kind of interested in the startup world, came out to the Bay area, started at Accenture, um, doing kind of consulting and research stuff, ended up at Treviso.

Treviso, this was, man, was this 15, 16 years ago? And it was at the time known as CEP, complex event processing. We forked Postgres to turn it into a streaming map-reduced database as data came in. And I think we actually had MySpace as a customer. This this was when like huge crazy volumes of data, fascinating tech kind of before its time, open source wasn't a thing, Postgres wasn't a thing, did a few things and ended up at Heroku. And when I joined Heroku,

I was actually joined to launch their Python support. I'm not a Rubyist, I'm like one of the few non-Ruby people from Heroku. And we were running postgres and one of the the gm of the postgres team came over to me and was like hey you keep doing all this blogging about postgres how about you come do marketing i'm like i know marketing people are sleazy right like that's what we think of them as developers um and he ended up convincing me and i ended up kind of running product and product marketing and all of that for you know mostly product uh for heroku postgres and if we go back to the heroku postgres origin story it was you know we had all these rails

developers asking for a database and thought how hard could this be? RDS didn't exist yet, like no RDS. Rails developers wanted a database. We picked Postgres and years later, I have the email from the GM of RDS saying like, you're the reason we launched Postgres. Like, thank you, Heroku. Cause we had all these, like we got so tired of listening to our customers talk about how they don't want MySQL and they wanted Postgres.

Went from Heroku over to Citus, which turned Postgres into a sharded distributed database. They were acquired by Microsoft, we're just running all of Azure Postgres. It's funny, I actually think of myself as like a DevTools person more than a database person. Like most of the internet is like, oh, Craig, Postgres. But let's go talk buildpacks and deployment processes. I'll spend as much time on that as Postgres, but also it's a really good database. And so I've kind of been pegged as the Postgres person just by championing it for being a great database and for what it is. Yep, absolutely. And I remember like, that's how I ran into like my first programming experience was

like Python Django apps on Heroku using Postgres, like reading your stuff and just like fall in love with it and just be like, okay, this is, this is awesome. So, um, yeah, I greatly appreciate all that sort of stuff. I guess like, how did you, how did you fall in love with Postgres so much? Or maybe like, why, why are so many people falling in love? Why, why is it so popular? Um, you know, all these years later, it just, it just works. Like it's so sad. Like, it's just like, Oh, I want to like write a report. I can write all of this in, in Python or Ruby or whatever, and extract my data. And like an hour later I get a report or I can actually just write some SQL right like it's for a long time I thought developers really should love SQL and appreciate SQL and I've kind of given up on that battle like maybe I'm wrong but like the idea of like window

functions or CTEs or like when I got deep into Postgres it was it was really around 8.4 when like CTEs and window functions came in. And I'm like, oh my goodness, I can write this report that does this, and it's so fast and efficient and all of that. And I spent so much of my time at Heroku, just not even externally, internally. I was yelling at our developers of like, why are you writing this active record stuff that generates this query when you could just write sequel um and i recall in actually one of my interviews it and heroku they were like well how would you generate this report that does this and i just kind of wrote like sequel

pseudocode and they're like oh my goodness like what is this black magic stuff it's and i don't think as a developer you have to be afraid for from it like that's the thing that i think like development communities have really come around to is like i don't have to be terrified of my database it doesn't have to be it's so funny because engineers by by nature we're learners right we're curious like let me talk to you about how like clear ice is made and you're like huh that's like every engineer has some weird hobby right that it's like the learning part databases though it's like we don't want to touch them it's like no no we don't want to go that crazy we're not actually a masochist and want to

go into like how databases work but it's a really good database that just does 80 90 percent of everything i wanted to correctly. And I don't have to worry about it. Yep. It is surprising to me how limited people like, or as developers don't spend that much time just figuring out basics on like database and like performance and realizing how much you can actually do in a database compared to just doing your application code.

It like, or just like how basic indexing works. It surprised me like, you know, how like fairly senior devs still don't understand that as well as I think they should. So that's super interesting.

I think it's interesting that you think of yourself as a developer, tinkerer, dev tools type guy. Because when I was talking to Sam Lambert from PlanetScale, my SQL guy, I'm pretty sure he said, developers love Postgres, operators love MySQL. Does that map at all to you? Do you think that you agree on that or have any thoughts on that?

It's a loaded one because it's so funny. If you look in like the Ruby on Rails community, like I just came back from Rails World a few months ago. And man, it felt like the Rails community is like vibrant and popping. And there were like three talks on MySQL and the two biggest like Rails apps or three, Basecamp, I think Shopify and GitHub are all MySQL. Everyone else in that ecosystem is Postgres. And so I think you get like an outspoken voice from the like couple of huge, massive scale. And it's like, oh, Postgres doesn't work at this scale.

Uber is a great example where they're like, look, we migrated away from Postgres to MySQL. What's covered up is like they before that they migrated from MySQL to Postgres. And they had a new VP of engineering come in that said, I don't like Postgres. Let's migrate away. And that's the entire, like, could Postgres, like, are there flaws with Postgres? Sure. Are there flaws with MySQL?

Absolutely. I would not over gloss and say like, you know, operators love one versus the other. And I think some of those things, I want to talk about some of those later too, but I think some of those things have gotten better over the, like replication and some of the things like that in Postgres to like where that operational stuff with Postgres has gotten a lot better as well. So, so yeah, we'll get, we'll get into that later. One thing you mentioned is just like the, the size of those companies and like those big MySQL installs, but just like, it's funny looking at your different stops, Heroku, Citus, and Crunchy. I imagine that your Postgres is sort of all those, but I imagine very different sort of customer profiles there, right?

Like Heroku, especially that Citus one in the middle. I imagine that's a lot of enormous installs as compared to Heroku being more lots of smaller ones. I'm sure some big ones as well. And then Crunchy probably more in the middle between those two. Am I right on that? Has the sort of customer base? Generally, yes. Right.

Like we ran over a million databases at Heroku. The idea of like a one in a million problem was a once a week. Right. Well, but like we had, you know, it was way back. Right. We ran, I don't know if you remember like GroupMe or Reportive. They were like massive databases. Think through math.

There was a bunch of way back at Heroku. like these are multi-terabyte databases, right? And when you're talking about multiple terabytes, like sure, it's not Facebook or Twitter scale, but like that's pretty massive. And these are transactional databases, right? Not like a newsfeed where it's okay if you refresh and it looks different, right? Like those are very different problems when you talk about the massive, like even even google when you think about search results it doesn't have to be consistent right it's it's roughly this and there's a lot of caching and otherwise and inside is yeah our average customer i want to say was around 10 to 40 terabytes our largest was 960 terabytes so i i love it it when something happens on the internet and I write a blog post and it hits Hacker News

and they're like, huh, Postgres doesn't do this. And I'm like, well, I think I've been there and I've run it before and I think it can. I'm pretty sure at that kind of scale, it doesn't matter whether it's MySQL or Postgres, you're making trade-offs and it works. At Crunchy, we've had actually Heroku send some of their largest customers over to us in the past few years. They're outgrowing the bounds of Heroku, but like super specialized.

Like we know how to run Postgres at scale. We've got customers at 20, 30, 40 terabytes on single node Postgres right now on Crunchy. And so it was different like situs was a special purpose thing it's fascinating we actually just today launched official support for it on on bridge it's not everyone needs situs if you do it's it's pretty magic but are you at that point like do you sort of try and talk people out of Citus or not like talk people out of it? Like it's a bad thing, but just like, are you pushed back and are just like, Hey, make sure you really need Citus because I, you know, there are trade-offs once

you go sharded and things like that and those difficulties, or like, how do you, how do you sort of approach that when people want, want Citus? Yeah, definitely. I mean, even at Citus, we had one customer that came on board that was like 40 gigs You don't need this. You don't use and they were playing like everyone's gonna be the next Facebook, right? It's like I am I'm running an e-commerce for for hamsters and the scale and the market of hamsters is massive And we're gonna need petabyte scale and it's just like you're not but there there's a balance there like as a customer right like cool yeah i'll show you how to use it and there's certain apps like if you're a multi tenant sas b2b app like situs actually drops in really well and it just solves a lot of problems

you don't have to think it like you don't have to worry um but there's a lot of folks out there that are like oh you're not google scale but we're selling google scale technology and i'm just like yeah postgres runs the tld.org like the.org tld is powered by postgres it all flows through that's wild and like all right if that works that's good enough for me um I mean, I try to be as consistent and transparent as I can over the years. Like, there's a lot of folks that are now bandwagon, like Postgres fans that were actually talking crap about it, like five years ago, it doesn't do this doesn't do that. I'm, you know, what you see is what you get. Like, I think it's a good database. I think it does 90% really well. I think like, if I'd rather not have another piece of my stack to add complexity,

that's what I want. Like if I don't have to maintain on call for another, for a full text search system or for my vector database or for my graph database and Postgres gets me 80, 90% of the way there, then like I'm sleeping easier at night. Yep. Yep. Okay, cool. I want to talk about some of those things later as well. Okay, but you mentioned, so CrunchyBridge announced support for Citus today. Before that, was CrunchyBridge mostly, almost

exclusively single-node Postgres, or did you have other mechanisms for doing multi-node Postgres or sharded Postgres? Mostly single-node. We have some customers that have scaled out with read replicas. Postgres will maintain its own cache for a read replica. It's a clever approach if you're like, I've got my leads table over here,

my opportunities table over here, and send all the queries to my opportunities table over here. Postgres will maintain a separate cache there. It works. It's pretty simple. It's pretty simple. And it's a pretty, you know, brute force kind of approach, but it does work pretty well. We have had some Citus customers for a while that it wasn't quite so official, but it was there. They were really savvy users. This is more of a first-class support for Citus.

Gotcha. Okay. Sounds good. And so you mentioned that Crunchy sort of started out as like more support. You're, you're the product officer there and doing more product type stuff. What, what sort of product offerings does, does Crunchy have? So we've always had products, but it's been a little more of like, um, the red hat model, right? Like I, I need, you know, enterprise, like i live and breathe this open source world right like i came up through heroku and like that's of course you know this open source ecosystem

i mean the django world i feel like is my family and home even though like i don't write django these days i don't contribute to django but it's like show up at, you know, I was dealing with Simon Wilson earlier today and like Jacob Kaplan Moss, like when he's in town, like if he's not coming over for dinner, I'm pretty annoyed at him. We were just emailing this week about something. And so like, I love that world and ecosystem, but like, that's my, my world and ecosystem. Where was I at? So, crunchy product

offerings. Yeah, what's that sort of look like? The enterprise isn't used to open source. Still, today. And they're like, all right, we've been using Oracle, and we're going to go to this. We've heard

about this Postgres thing. Okay? And they're like, now we've got to go to this we've heard about this Postgres thing okay and they're like we're gonna now we gotta like what's our monitoring stack what's our HA stack what's our disaster recovery and what they're gonna do is a bake-off of every single one of these products and it's like three years later they're like all, we're ready to deploy Postgres. And the whole landscape has changed. And I look at this and it's just like, at some level, I cringe, right? I'm just like, why can't you just know? Here's the best of breed and you deploy this stuff. And if it doesn't work, you change it.

But that's not how enterprises work. And they say, I want to trust Postgres and that I can deploy it, right? And so with Crunchy, you get trusted deployment of Kubernetes or bare metal. And it's like, oh, I want, I need HA, I need backups, I need disaster recovery, I need monitoring, I need connection scaling. How many connection pullers are there right now for Postgres? There's Odyssey, there's a new one I'm blanking on, there's PG Pool, there's PG Bouncer, each of those has serious trade-offs. And it's similar to Red Hat, right? Enterprise is saying, I want to run Linux. Okay, well, what flavor of Linux and how do I get support for it, right? And so that's where we were at for a lot of years was large enterprises, large banks,

public sector, all of that, that I want to run Postgres. We're making a strategic bet, but we need someone that can tell us what to do. Give us a curated deployment. And so we started there, right, which was was still software and products, just not like I'm'm gonna click a button and swipe my credit card and get a database right um and crunchy bridge came to be about three years ago um and i joke but it's it's also true that it's like unfinished business from the heroku postgres days like heroku postgres really solid but it kind of stalled out. It's like Heroku Postgres was Postgres for Heroku.

And RDS is fine. There's nothing wrong with it, but I think we can do better. And you see that from our customers that like, they ask a question and the level of support they get the, you know, if we can ship a feature before we respond to the support ticket, and that's not because we don't respond to the support ticket for like six months it's like no we're going to respond to you tomorrow but we're going to see if we can get this live today as a feature improvement if it's a good feedback like it's it's huge and you want to ask about like shared buffers or huge pages or like what this means for like hey there's this weird query plan. I mean, we employ Tom Lane, who contributes.

I mean, he is Postgres. He's been ranked as the top six, I think, contributors to all of open source. He helped write TIFF, then JPEG, and PNG. He co-authored the spec for PNG, and then he's like, I'm bored with that, let'sPEG and PNG. Like he coauthored the spec for PNG. And then he's like, I'm bored with that. Let's work on his database system. Like if there's a bug in Postgres, like I can get the answer to it.

Or it's not fixable, right? Yeah, exactly. And just like having that sort of responsiveness and like expert knowledge on board where it's not like you're filing a support ticket to like some like, you know, first level support person that's going to like dither around forever. But like, you know,

it gets escalated pretty quickly and you like real experience back is, is just like so valuable. I think you're seeing that more and more like companies that are tying together some, some operational and management stuff, but also just like that, that wisdom that goes with it and just like making it super easy or like helping you bridge that gap is, is, is awesome. I love it. So

if I'm sort of getting started with Postgres, what's your sort of recommended strategy? I imagine like you want to start with a managed offering, right? There's no reason to be running Postgres yourself anymore. Is that, is that standard advice? I think generally like I, like, you know, to each his own, his own right if you if you want to like become a postgres operator like and and puts around sure like i don't think that's bad or wrong right i you know like if i were a vc like my investment philosophy would be like i i want to i want to

do the kit shit done stack right it's like rails like Rails or Django, Tailwind and Postgres. And like, I'll run it on Heroku or Render or Fly and the database on Crunchy. Like, I don't want to be like doing security updates. I want to be building features. Right. And like, it's a super anti-pattern from a lot of developers. I think like want to learn everything versus like i want to build products um that's my ethos and i you know i want to make

databases easy so you can get back to building products you know if you want to learn postgres right like i there's a couple of books i think dimitri Fontaine has one. I think it's The Art of PostgresQL. Andy Atkinson has one that's for Rails developers. That's a great, great book that just came out. Highly recommend both. be curious just like you are about everything else like we've got the postgres playground with crunchy data which is like postgres in your browser um one of our folks on the team went and created a like tutorial for like i don't maybe not elementary maybe middle school kids but maybe like late stage elementary that's like she was helping with a summer camp and wanted like kids like there's no cost to this. It's running in your browser Like learn databases and how they work We've got a ton of tutorials in there that it's like I think the biggest thing is don't be afraid of it ask the question be curious just like you are for other things

Yep, gotcha. And so, you know just going back if I if I'm getting started with postgres I want to manage offering like I hear you, support and things like that from CrunchyBinge, but you are CrunchyData, but you, you all have like a sort of entry level offering, like in, like an RDS or something like that, where it's like, if I have a new database, I can just come in and you, you'll totally handle that and sort of scale with me until I need more of like that, that support and things like that. Is that, am I understanding that correctly? Yeah. It starts at 10 bucks a month. Um,

if you're like a complete, like poor college student, you actually, um, it philosophically like paints me to charge you. Like I'm in the Bay area. Coffee's not cheap. Like it pains me or avocado toast.

Take your pick. Right. Um, it pains me to charge you like a charge of credit card for $3. If, if your bill is less than $5 in the month, we don't charge you. Oh, wow. That's awesome. So it's like, it's not necessarily free to run continuously, right?

Like I I've seen enough of the like freemium world. I was at Heroku for a long time. Pricing changes were hard. It's not a good situation. Like, yeah, we're running resources under the covers. Like there's not a free run your database all the time on us. I mean, if you're building a product, $10 a month is not crazy. If you're learning,

just suspend your cluster at night, every night. And there's like an easy suspend button that like data doesn't go away, unsuspend it like as a student, you should be able to not pay us a penny and learn postgres cool okay tell me about i guess like the postgres ecosystem and just explain this to me right now because as i look at it there's like there's so much going on there's there's core postgres

then there's this world called extensions which includes like you know sort of like add-on extensions like postgis or pgvector but then there's also like you know almost like full full-edge extensions like like situs or or timescale which is like almost a different GIS or PG vector. But then there's also like, you know, almost like full, full edge extensions, like, like Citus or, or timescale, which is like almost a different, different thing.

And then there's like these Postgres with like application level features, Nile, super base, things like that. And then there's like this Postgres compatible world of, of you go by Amazon Aurora, things like that. Like,

I don't know if I have a question here, but like what accounts for the, what accounts for the energy here? And like, what's going on in this whole ecosystem? It's chaos, right? Postgres core and community. Let's just start there, right?

Like the core team, you think of the core team is like that's the side what happens, right? No, like the core team is like a governance body and steering committee. I think the core team is nine right now and maybe seven. There's like, I don't think it's written. I've never read the core team is nine right now and maybe seven. There's like, I don't think it's written. I've never read the core bylaws. I don't think it's actually in the bylaws, but there's this unspoken rule that like the core team can never be more than 50% one company.

No, like crunchy has members on core team. EDB does like when EDB acquired second quadrant, it was like, Oh, we're more than 50%. What do we do? We add people.

Core doesn't determine what gets committed in postgres all of postgres development happens i would say all um 80 90 on the mailing list now there's there's back channels that happen right and there's development meetings like pgconf.dev used to be pgcon um i actually hate the name pgconf.dev because you think it's like oh this is for developers if I want to learn Postgres. No, no, no. This is where the Postgres hackers show up. You want to learn how true... It's like Linux kernel development. It's mailing list and patches on the attachment, right? postgres you can commit something um i think there's about 50 60 committers i'd have to go double check that there's also this like vague term of like major contributor minor contributor um doesn't really quite mean anything it's like oh you contributed a major feature you maybe didn't even commit um you there's this kind of unspoken street cred of like you commit something you maintain it i recall a couple years

ago there was there's one you know committer that got their commitment committed a whole bunch was really excited the whole next postgres release he was cleaning up books like there it's not taken lightly um it takes a while to get a commit bit. I think the average age is probably 40, 45, 50, right? It's not like young upstarts. There are some younger folks there, but it takes a while. The idea that, oh, you don't need to know how B-tree or Gen Indexes work and this CS academic stuff, the whiteboard problems actually you, you do to commit to post. They matter here. Yep. And there's, there are a lot of, is there a lot of active development and stuff happening within core Postgres? Like when I see that extension ecosystem,

I wonder like how much gets pushed out to that versus like, how much is core Postgres just like like charging ahead and like making a ton of improvements? Is there a lot of energy in core Postgres itself? Yeah, I mean, there is. It's usually slow and steady, right? It's usually like performance improvements. Citus was pretty instrumental in like its first Citus was a fork. And they pushed to actually get like extensions, like there always has to be a hook exposed for the extension to work or not.

And at first, Citus wouldn't have worked as an extension. And so Citus aggressively worked, you know, like Marco and Andres and team, um, to get these hooks in. Right.

So they wanted to be an extension from the beginning and just couldn't, so they had to fork it until they could become it. Cause I remember when they did switch from fork to extension. And that, so that was sort of the plan and desire all along. Yeah, it was, I don't know if it was all along, right? But it was, you know, it wasn't a possibility on day one. Back when I was at Treviso, it was a pure fork, right? Like you look way back, like Redshift was a fork.

Redshift was Paracel. Amazon basically acquired the IP and didn't hire anything. They didn't acquire Paracel. They licensed Paracel and then kind of let it dry and then hired the engineers to keep doing it and then rebranded it as Redshift. There was Greenplum back in the day, Astrodata, which, you know, part of Netiza, Truvizo. There was several of these that were all forks, and that was the only option back then. Now, with more and more of these hooks, now, when we get into extension territory, right, there is Contrib that ships with Postgres, like HStore, some of the UAUID stuff Trigram there's like I forget what it is

like point stuff like geospatial like earth distance I think there's some basic geospatial stuff then there's like post GIS is massive I wouldn't say it's the most advanced extension it's maybe the most breadth like it had this weird parallel community.

They release on their own cadence. There's their own community. You'll see some of those folks at Postgres conferences, but also some of them not. And it's huge. It's well regarded as the most advanced open source geospatial database. It's like you kind of have Esri, which is high end commercial, and you have PostGIS. There's not another thing in the same ballpark.

So what determines, like, so some of those things in Contrib, you're saying like HStore or Distance, those are still extensions but they just ship with Core? Is that what you're telling me? Yeah, you still got to run create extension. Okay. I'd have to check. I can't remember the last time something was added to

contrip okay and are those just sort of like small and stable enough that you know they can they or was it just like sort of past decisions and now they're like we're not doing that anymore like extensions are separate now i i can't remember of one being added in the last five years maybe one of the uuid ones not sure um yeah i don, I don't think we'll see a lot more contrib. It's my hypothesis. And I'm sure there's going to be some, like, committer that's like, ha-ha, Craig, I'm going to show you wrong, and I'm going to slide one in. For the most part, like, PG stat statements is the one that I can best think of.

That was 9.2, but it already existed. It was like a complete rewrite. Like, PG stat statements before 9.2 is not the same PG stat statements that we got in 9.2, which I think is the most useful extension, hands down. It should be installed on every single database. It does have a minor, minor, minor performance hit. It's not like it's free. And so that's where it's like Coral say, nope, we're not going to do something and make this decision for everyone. Yep. Yep. Interesting.

On this same sort of note about the committers and core and things like that, I was talking to another database guy. I don't want to talk about the say which database because I don't want to stir up stuff. But he was kind of bemoaning his database of choice and how performance had been getting slower and slower over releases. And part of that reason he was saying is just like, hey, there's more features being added, which benefit the few, but sort of add just performance regressions,

slight performance regressions for the many. And he was noting that Postgres has managed to avoid that for the most part. Do you have a sense on why that is? Is that your sense as well? Is that true? Yeah, I mean, Postgres is slow and stable right like i mean you mentioned sam's comment like oh it's for the operators and it's like they layer things in and it's like postgres is gonna be right like it's not

like its goal is to be correct always like like my sql plays play fast and loose with like, the the data, like structures underneath and the storage engines. And the idea it's like, oh, we don't need foreign keys, like who cares, like you'll as an app develop every single time I've encountered a database that was like, oh, we we don't have foreign keys in our database. I'm like, let me run just a few queries, like which which record is correct. And they're just like, they, we don't have foreign keys in our database. I'm like, let me run just a few queries. Like, which record is correct? And they're just like, they're spending weeks cleaning. Postgres has always been, the goal is secure and correct. And then fast, right?

Like, then we'll work on the fast piece. We don't need bells and whistles, right? And the extension stuff was thought about from day one it's it's there's not another equivalent like i i it's it's not oracle it's not mysql it's not like the extension hooks are low level c hooks that gets exposed like you know there's you can write to dev null you can like create data types you can create like all of this stuff like there i don't know of a parallel to it and i think like it was a bet early on i think if heroku hadn't bet on postgres and it hadn't gotten adoption i don't know postgres would be where it is today that's interesting yeah it's great kind of wild yeah yeah like does

that feel wild to be like sort of at that pivot point of of of that you know and changing how that ended up happening i mean i feel proud lucky yeah it's it's wild it's it's it's um and i i think we made the right choice like i don't think there's any question of that um and i i look at the likes of you know the the amazons of the day they were trying to be like no my sequel no my sequel no we can change it in this way and that way amazon aurora postgres took five years late it took five years late like they were like no no we'll just do what we did for mysql i'm like it's no no this isn't how engineering works. It's completely different. And so I, you know,

the, the team of Heroku Postgres, the time that I left, it was, it was eight of us. Um, you, you made a decision.

You were talking about Redshift came out of Park Cell and, and things like that. And that got me thinking like, what's, I guess, are there sort of columnar or like OAP style databases built on Postgres right now? I'm less familiar with that. Or has it just like shifted so much to Snowflake or, you know, S3 based stuff? I guess, like, what's it look like for OLAP on Postgres at like those, you know, much larger scales?

I don't think it has to be the much larger scale. I think like if I look like we have a lot of customers doing kind of OLAP-y things on, it's a different workload, right? It's lower concurrency, like different kind of planning. You can tune Postgres to do some of that. I think, you know, Snowflake is a runaway, you know, Snowflake kind of dominates the market right now. I think like Postgres is well-primed for that.

Without saying too much, stay tuned. I have thoughts. I just had three former colleagues from Citus, the two lead architects and engineers behind the Citus engineer join. We're cooking up some interesting things. What does that mean? Redshift is still pretty good, right? I think BigQuery is still pretty good. I don't think you actually have to go columnar these days to to do data warehousing i think like columnar is one approach and interesting but if you look at like the landscape like data bricks and snowflake aren't necessarily purely columnar and operate on different kind of like file formats and data formats um that world is i mean you yeah that world is changing and evolving and

i think like there's not a big player right now that's a pure postgres player um greenplum redshift or the two um astrodata and nathiza have postgres roots like if you go way back on the tree of databases there was like ingress which postgres is post-ingress. I'd have to go check. Like, I think like DB2 is like, like there's kind of a world of Oracle-based and there's a world of Postgres-based, like way back in the like 25-year history. It's fascinating. Like there's not like 23 databases that everything's derived from it's two. And it's like ingress is one of those. And the name is terrible, right?

Like, let's just like call it Postgres. Please don't call it Postgres QL. Like, cause, cause for a while I didn't even have SQL support. I think it was like 95. Then it was like,

and we added like the SQL back on and it's just a horrible name. It's just, it's bad. Yep. Yep. Okay. Cool. Okay. Now that we know the ecosystem i want to change gears a little bit and look at i guess like some features that are offered by postgres

and i want to basically like should i use should i not use or maybe like some caveats on like um you know when when should i not think about using or like where should i be worried about some of these things so just to just to get a sense on on like you know there's there's just so many features like when should I use those? So first of all, let's start off with custom data types. One of the core things about Postgres early on is you can create your own data types. If I'm an app developer, not an extension developer, not a library, anything like that, just an app developer, should I be creating my own custom data types?

Or should I just use the sort of bog standard ones? We create enums a good bit. I mean, I don't know if that's quite a custom data type, right? It works for some consistency. On the data types topic, don't use money, please don't. It's just a bad data type. I think we have a blog post on it somewhere. It should be removed. It came from a different era of one world view, and money and math is hard. What should I use instead? Decimals or numerics with proper, or just ints, actually,

and maintain the... Almost anything but money, basically, is a better choice. I don't know. You should go crazy on your own custom data types. I do think you should use the data types Postgres has. Like, if you're doing anything with time and calendars, like, range types are so amazing. Like, if you're building, like, a university, like, scheduling system that 30 kids can be in the class and you can't have a schedule,

like, range types. Oh my goodness. Um, check constraints are amazing when you layer them on. So use the ones that Postgres has. I don't think you need to go crazy with like, I'm going to go in and invent a data type. Yep.

Okay. All right. Second one, Jason B. Should I use Jason B? Yes. When should I use Jason B versus like, you know,

defining the different columns or like when is JSON B a good choice? I mean, it's probably not helpful. When you see it, you know it. It's like, oh, we've got this response coming back from an API. And sometimes it has this value. Sometimes it has that value. Or feature flags. If you're using that in your application, it's not crazy. Most of the time, application it's not crazy most of the time you want json b most of the time you don't want json i really like json if you're recording like if you're running an api and you want to record just the raw stuff and you're for the most part you're

not querying it you're not replay it's like we want the raw like we want a history of our api traffic and what hit it for the last week. And then we're just going to print it off, right? Because it's going to preserve white space, but it's way cheaper. You don't have to convert it to actual properly typed JSON. But mostly JSONB, yeah. Like as soon as you have certain columns that it's like, okay, like, you know, org ID and team ID and customer ID. And I like pull those out so you can easily index those. You can also index JSON B with like a gin index. Yep.

Should I be indexing my Jason? Like if I'm indexing my Jason B or, or querying my Jason B directly, a lot fields in there, does that sort of signify I should split it out as separate columns? Or is that still a fine pattern to you still? Yeah. You can still just like create a gin index. And as long as like you're querying like because a lot of times on json it's

like it's fairly undefined right like sometimes you have this column and sometimes you have this you know key and i want to like i want to know how many provision requests i got right and so i'm looking for like type of requests of provision and like create your index on that and You're fine. It works. So I don't think you have to overly normalize I'm actually a fan of denormalization like in multi-tenant apps like where you have like customer ID or org ID Put that on every single table, please Okay. What about full-text search? Yeah, use it. It's not great internationally. Like it's generally like if you need to load in,

like then you've got to kind of have access to the server to load in the new, like the language or, I forget what the actual official name is, but basically you've got to like load that in. It's not great on a managed service if they don't support it, right? It's still really good. Like it's like, on a man service if they don't support it right it's still really good like it's like instead of like running elastic i've got a friend that's gonna ping me as soon as he hears this he's like no craig this is so much faster use this and it's so much better

yes it's faster yes it's better and now i get woken up for two systems at night yep and like elastic is like i always say this but it's like the system that scares the crap out of me the most operationally like it's just it's just a different beast i think elastic is so i like to avoid it where possible but um it's it's fast it's good it's not a bad system i've got a good friend that runs like an elastic as a service and like i appreciate him doing that but don't yeah i like i'm with you i don't i don't want to be woken up for Postgres, much less Elastic. When it's like, uh-oh, we accidentally triggered a complete reindex of our cluster and we're down for two hours. That's Elastic for you.

And so should I... Hey, full text search is fine to do on a last name column, but you shouldn't do it for like a larger blobby you know a bunch of text or is it fine to do on larger blobby bunches of like user input and things like that you're fine with that as well totally works and it's really you know what's your what's your constraints and use case and you know like are we talking about like a 100 gig database like go to town right like i'm used to like the largest postgres database i've run was 960 terabytes like what how much storage do you have on your like really nice macbook right like not like this is all a matter of size and scale and magnitude like yeah i mean

a terabyte two terabytes like fine go it. Okay. What about row level security? It's great. It's not overly used as much as I've seen. I've actually, man, I've got a blog post that I'm like about to publish. I'm like using row level security for like multi-tenant apps to enforce policies. There's a bit of a balance there of like with PG Bouncer, which everyone should use PG Bouncer.

And when you're now doing like transaction level like pooling, which is how you should run it, like, okay, how do we manage session like in between sessions and all that? PG Bouncer just got a lot better with like transaction pooling and some of the stuff from the Citus team and what they contributed. So it's like way better now, like RLS works great. I mean, it's a, it's kind of a defense in depth view, right?

Do you need it or not? And I actually see it more valuable in multi-tenant apps than like, you don't want to give every database user out to like, you don't want to reconnect with a new user to enforce RLS. You want to like create the policies at the app level. Yep. Yeah. The area I see it the most now would be like with super base,

you know, where they actually are having client side users connect directly to a database. Have you seen that at all? Yeah. I'm, I mean, at,

at scale, like you just can't manage that many connections. Like that's a new connection to your database and 500 users or a thousand, at scale, you just can't manage that many connections. That's a new connection to your database. And 500 users or a thousand... I mean, I'm running databases that have hundreds of thousands of concurrent users. I don't even want to think about it. My colleague, Steven Frost, is actually the one that like contributed RLS. And it's funny because I was talking with him about the whole like multi-tenant stuff.

And he's like, huh? Yeah, actually that works pretty well. But you didn't think about that when you were like, it's like, yeah, this works fine. It's like, yeah, it's very security minded. Right. I don't know that I would, I would go and give it out to like end users because now I'm managing how many users connect into my database. What about stored procedures? Should I use stored procedures? Man, I'm not anti them.

I had an email from someone nine months ago. Cold email talking about stored procedures. We got on a call and he's like, yeah, tell me, Craig. like how have we solved the testing story and deployment story? I'm like, we haven't, I'm sorry. Like how do you roll out version one versus version two when roll back? Like all the things that we get with GitHub

and deployment and CI and CD, it's not there. One of our customers, that's a big personal budgeting app. You may use it. You'd know it. I don't know if I can name them or not. I think I saw it on the website. Is it WineApp? Yeah. Yeah. Okay, cool. It's not a website, so I'm not getting in trouble with marketing then. I have to be careful on who I get in trouble with. Yeah. I have to be careful in who I get in trouble with. Yeah, like, I mean, they, you know, it's a great budgeting app. I ran their database back in Heroku.

They came over to us. They have a ton of, like, stored procedures that do the computations and the recalculations of your budget. And, like, I love you guys. Man, I actually don't think I want to like come work for you and manage that app. But like, there's this like, how do you version and test? And it takes a certain type

that you really got to be comfortable with SQL. So I wouldn't be like all in on sort of procedures. I do think the pendulum swung too far where it's like, no, no, databases are dumb. Don't put logic in them. I think we swung too far away from that. no, no, databases are dumb, don't put logic in them. I think we swung too far away from that. Like the idea of like, well, how about foreign key constraints and check constraints?

Like let's at least have some integrity in the database. I don't know that I'm fully back to like, let's put all the logic back in the database. Yep, yep, be pretty judicious, pretty limited with it, but a few things can be kind of nice for you there. Okay. What about listen, notify? It's okay.

It's fine. I wouldn't, you know, do you need to guarantee delivery? Because like, if you're not listening, you didn't notify. Right. As a cheap kind of queue approach, I don't think postgres is a great database for a queue yet it can be done if you're really good about it i that and as a graph database there's options there's approaches i don't know that i'm like yeah you're using postgres you should definitely do that geospatial yeah for sure full text search yeah for sure json yes um vector yes like pg vector is good

but i don't know that i'm like yeah as a q or as a um like graph database i don't know that i'm quite yeah no brainer yeah gotcha for graph is there a is there an extension that people use for graph or is it mostly like hey how you're modeling your data and then you know the queries you're writing and things like that or like how are people or it's just like not that many people using postgres as a graph right now there is an extension it was aiken's graph and they may have rebranded i think it's part of the apache foundation i every time it comes up, I guess asked about it and I'm like, I've never heard of someone using it in the wild. So, I mean, it's like, I can't say like,

like Craig, you follow everything Postgres, you've seen it all. And I'm like, I just, I haven't seen that in the wild, not to say it's not out there, not to say there's not people using it. I just, I haven't come across it at all. Yeah. I think there are just like fewer graphy problems than people think too. Like a lot of them, you know, people, I think overreach for graphs sometimes and don't understand how to model it and it's like a lot of that could work in just like a sort of standard database without all the graphy stuff so i mean if it was what 12 18 months ago and it was we were still at the nft like heyday then maybe right like that's clearly a graph problem is what it is i i don't know a bunch of other problems that fit into that territory

yeah absolutely what about sort of outside of postgres but orms are you pro orm anti-orm somewhere in the middle somewhere in the middle right like my favorite orms are not the mainstream ones it's not active record like cql in the ruby world is a really nice ORM. I like SQL Alchemy over the Django one. It's just slightly better, and it also makes it easier to drop down to raw SQL. So I love within the Django world that it's getting support for Postgres native things like range types, like JSON beat.

I love that the ORMs are not like, oh, we're just going to treat all databases equally, and SQLite has the same data types as Postgres. Man, the discussions and arguments I had with Django core people, I'm like, no. And all of them ran Postgres anyways. They're like, no, no, no, but we're going to be equal to MySQL in exactly how well we support. So I do love that ORMs have kind of jumped into embracing databases.

Yeah, my favorite is SQL for the Ruby world is a really good ORM. SQL Alchemy, what Mike does for that is really, really awesome. So a fan of those. I can't speak as much to like the node world or the Java world. I know most of the others a little bit better. Yep. Okay. Yeah. And just like sort of on all these features,

the article I always think of, I don't know if you know Nelson L. Hodge. I think he used to work at Stripe and he just had a really good post once about just like how he likes relational databases, SQL and all that stuff. But the problem being, they're just like so many features and it's hard to tell which ones are going to scale well and which ones aren't. I guess like, do you have that worry or are you just sort of like, you know, explore the studio space? It sounds like you sort

of like the features of a relational database and probably just not that many people run into that space where they're going to have the scale problem that comes up. Yeah, it's a lot less than you think, right? And yeah, I mean, how much can I say on Stripe? Like Stripe created a lot of interesting, like Stripe's a fascinating company and great engineering but i like brunder who works on our api team like you know talk in in depth of like working with this and that and like why did we create this problem or that problem right and you're building products you're not necessarily making the best technology decision postgres works really well and it scales really far and like if you get to stripe scale and like this doesn't, this feature doesn't work at that scale. Guess what?

Stripe has the engineers and the resources to then like try it five different ways. Like, if you get to that point, you've won. Right? They're different problems. I haven't ran into many problems where I'm like, oh, man, Postgres just fell over at this scale. Yep, Absolutely. Okay. I want to ask a few questions that are, I'd say like maybe like responding to naysayers, like some of the things you maybe hear people naysay about Postgres and just get sort of your opinion on one of them you already alluded to, like in the, I would say like

the, the right amplification issue that like Uber mentioned in their sort of Postgres to MySQL posts. I think like Andy Pavlo kind of talked about it in his like least favorite thing about Postgres was something about their MVCC and the right amplification. I guess like, do you see that being a big issue, the right amplification issue with Postgres? In extreme, extreme cases. I mean, it's not like it doesn't exist,

but you've got to be really pushing crazy. And I think there were like trade offs and other approaches like uber could have made right uber uber came in with someone there was a new vp of engineering that said i know my sql my sql doesn't have this problem we're migrating to my sql right and very blunt force like yeah and i can i can go and point to 10 databases that are at larger scale than uber that run postgres fine so I think it's you know if if you don't like the tool that you got and you want to change to something you're more familiar with great um you know the z heap and

other approaches to some of that like they've been explored and thought about, you know, you take it a long enough approach, we wake up in five years or 10 years, it's a solved problem in Postgres, right? If you're worried about that problem today, okay, maybe Postgres isn't right for you. But like, you're making other trade offs. What are those other? What about connection management? That's another one I hear as well. What's sort of the state with that? Like, are we just like in a, hey, you need a sort of a connection pool outside of Postgres and that's how it's going to be? Or do you think we'll get some improvements in core? I still always recommend running PgBouncer. Like, I just do.

Like, it's not a bad thing. I see applications drop it in and it's the biggest free launch that I've ever seen in running a database. Postgres 14 got some pretty massive, like, in large thanks to Andres. Andres was a co-worker at the time at Citus. I would sit there and debate and debate and debate him. He'd be like, Craig, you're wrong. And he ran an in-depth benchmark. And he's like, okay, maybe you're not as wrong as I thought. And huge improvements, like connection management in Postgres 14 and on way better way better like thousands of connections not a problem still why not run with pg bouncer it's an amazing connection puller like we to me that was a no-brainer to ship like i wouldn't allow us to

ship crunchy bridge without it i'm like every database as a managed service should have built in connection pulling because it is a weak spot in Postgres. And we shifted on day one. And I think there's still major database providers today that don't have it. I'm just like, you've got to do better if you're a managed service. So it got way better in Postgres 14. I think it has a worse wrap right now than it should, but I would still run with one.

Gotcha. What about operationally? You all are handling PG Bouncer for folks. How hard is PG Bouncer operationally? And just thinking back to, you know, the two systems you're talking about, Elasticsearch and Postgres, like if people are not using CrunchyBridge and they're like, do I really want to pull on PG Bouncer and that maintenance burden? Is that an easy thing to maintain? Is that a hard thing operationally it's it's it's pretty straightforward and easy it's it's it works pretty well um i would say like there's not a an easy ha mechanism for it so you do kind of need some monitoring and that sort of thing it's not like postgres ha is i want to say it's solved because it doesn't ship like postgres doesn't ship with ha and you've got to grab

something off the shelf like you know it's why people pick up in like the enterprise is called crunchy it's like okay what do i need for ha right um but it's it you don't have to go like oh man what are the options do they exist um pg bouncer is pretty resilient it's not falling over there's a lot of new ones that i get questions about i i can't say like operationally how those look i'd be more skeptical. PG Bouncer has been around for a long time and it generally works. Is that the most what you

used? That's what y'all are using, you said, at Crunchy? It's that or PG Pool. PG Pool, I am much more a fan of the Linux velocity, small, sharp tools do one thing and really well. I don't want my my editor to be like my you know uh like my my text editor to also be like

my image editor like no like I just I want this thing to do this thing really well pg bouncer does that pg pools also a lot of other like read, write, load balancing, and extra proxy, other things, which is a little more magic than I like for my taste. So we run pgBouncer just like right out of the box for you on Bridge. What about transaction ID wraparounds, vacuums, just that whole area? Is that still a big problem in Postgres? Like are vacuums a pain or is that fairly easy? Vacuum is fundamental to how Postgres works.

And people are like, look, vacuum's taking time. Let me turn it off. Postgres is an append-only log. Like when you do an update, it doesn't go and change a record. It says, this one's dirty. Let me write a new one, right? Now that's a gross oversimplification, but like that's what it is, right? When you delete a record, it doesn't go and delete a file.

It says, hey, you can reuse this space, right? And vacuum then comes back later and says, oh, says hey you can reuse this space right and vacuum then comes back later and says oh now you can actually reuse this space um most for the most part tune vacuum and you're good right auto vacuum will run it's good um transaction id wraparound not a bad idea to monitor it but it's not a like yeah monitor it if you're a busy database and busy application and you know you are right like the the e-commerce hamster shop you're not but like you know if you're a busy database yeah monitor for transaction id wraparound and you're you're pretty safe it's um and there is some work in Postgres to make this less of an issue instead of 32-bit, 64-bit.

It's getting better. It's improving. It gets a lot more noise than it probably should. Yeah, and I feel like I haven't heard a story on it for a while. It seems like for a while, every six months or a year, someone would talk about how they had a transaction wraparound issue.

I feel like it's been a couple years years since I've heard somebody do that. So maybe there's just been like enough, you know, word and knowledge about it that people are aware of it and monitoring it better and not hitting it. We had one customer recently that hit not a transaction ID wraparound, but an ID exhaustion. So their like primary keys were just like int32. They're like, uh--oh we're out and

so and that's like the orm layer right like that's not like a you know postgres thing it's like you created this and this is the key space um and there's actually a really clever trick like all right go negative like reset your sequence back down to like negative you know five billion and now let's change like let's let's gradually change this column type right or write a new column um so it's like oh yeah here's the fail safe escape hatch which works really well yep oh that's interesting um our last one on on sort of naysayers you know transactional ddl is something that postgres has and i believe my sequel doesn't uh but what i hear some people say is like hey that's more of a toy feature like something you wouldn't use in production like do you use transactional ddl do you recommend it yeah all the time all the time like if you're running a migration and it fails midway like i actually

don't even know what happens on the like like, half your tables have this column, half your records have this column, half don't. It's actually a pain because you can't concurrently create an index with transactional DDL, right? Because it's a, you know, transactional thing. I always recommend concurrent index creation because it basically runs two to three times slower, but it's in the background, right? It doesn't hold a lock on the table. I think my skill only grew the capability recently to not hold a lock while you had an index. I'm just like, the idea that operationally that's okay

blows my mind. I'm like, what do you do then? That seems really hard to back up. Yeah, exactly. What do you do? That seems pretty wild to me. And so, yeah, transactional DDL is wonderful. Oh, it blew up. Okay, great. Let's roll that back.

I wanted to write all these columns out and it couldn't. To be fair, I haven't operated MySQL in more than 15 years, but I'm around the space enough that I, I don't, I'm sure like there's some like, yeah, this is the playbook for every operator of MySQL and what you do. And there's, here's the like hack that works around it. Postgres just works.

Like, transactional BDL is amazing. Like, why wouldn't you want it? Yep, cool. Okay. I just want to close out with some common questions. These are six questions that we just ask all our guests, just get a feel on, on some of this stuff and see where they're at. So sort of rapid fire,

but if you could master one skill that you don't currently have, what would it be? Can be tech, can be non-tech, whatever you want. I don't know. Actually, maybe like better at fishing. Like I grew up in Alabama, like I grew up on a bass boat. I don't know that I ever like became good at it. So maybe that like, just like the idea of retiring from tech and like being on a boat and fishing all day is lovely.

Except for I'd like to actually like consistently catch, you know, price fish. So maybe completely like off the radar that. It's funny because you're probably like top 1% in the world already at that. Cause like, you know, people don't don't do bass fishing at all you know but now you you just like to be significantly better at it it's that or like i mean if i were to like completely swap job like i don't actually want to swap jobs but like like a like world-class like tiki bartender like that i would do that yeah that's a good one all right uh what wastes the most time in your day kids don't yeah like i love my kids don't have kids like oh no uh yeah like um people are like

what's your hobbies have kids i don't have kids what ages are the kids right now uh 11 and 8 so i'm like counting the day to like there. I'm really looking forward to when they go off to college because I want to like retire and then like show up at their campus, like their college campus at like Thursday and be like, hey, we're going to dinner tonight and completely embarrass them. That's what I want to be their dad. There you go. There you go. If you can invest in one company, that's not the company you work for.

What would it be? Ideally private company because otherwise you could just invest in one company that's not the company you work for, what would it be? Ideally a private company because otherwise you could just invest in them. I feel like I'd be like, I want to be a VC of the make money companies, which is like the rails. That's what I would do. It's not like one. It's like, I want to find the companies that are not using this shiny new database that came out last year that are using Postgres, that are using Tailwind and Django or Rails. And like, that's your tech stack. Like that's, that's how I'm going to invest. That's your filter. Yeah. Tell me that you're like, I'm working on this distributed database,

this, that like, Nope, Nope. Like you're, you're going to be working for three. Like, that's what I would like. I go bug Jason Warner and other friends and pull our money. The boring VC sounds fun. There you go. That's a great name. Would you ever want to be a VC or do you like building product too much? I like building teams. I like operating.

If you asked me eight years ago, I thought maybe. I like operating and building teams, and teams that execute really well. Being part of the Heroku team, every one of us there felt like we were an imposter. Like, we didn't belong. Like, the way we leveled each other up was unbelievable. That collection of talent will never exist again.

Sidus was good, not the same the same crunchy is getting closer to that like that level of talent that like people filling in and growing that team um and there's a lot of i like high growth places there's it's not always perfect it's not always pretty it's not always easy or fun i i guess i'm getting old and soft and like it can be a marathon you can build great teams without like you know rolling over people and so i'm excited for what we're building maybe i'm getting old and soft that it's like you know it's not a you know grind to like wear everyone down and prove you're smarter it It's like, we're all on the same team, but like growing teams is really fun and building products and like,

man, it's fun giving good customer support. Like when's the last time you were like, I love my, like, have you called up United lately or Hilton and like, you're like, oh man, I got to deal with this. I mean, if we don't give good support to our customers, our team will hear about it from me. And it's refreshing to be able to actually deliver that.

I agree. I used to work for a dev tools company and it was so rewarding to help people with their problems. And it's just like, help people that you care about because you're in the same position as them. You get what they're doing. Like, oh man, it's so much fun to be working for dev tool companies.

As much as I'm a rusty old, I'm not a proper developer anymore. I like to think I joke that maybe in middle America I could keep up, like in the Bay Area where it's muscle to muscle. No, I'm not even passing grade. But it's like yeah, developers are my people as much as, usually that's my threat to the engineers is like hey When's this going to production? They're like it'll be ready when it's ready. I'm like don't worry. I'll push it no no no Craig You'll be done tomorrow usually that's my threat now because my coding ability is just falling off, but like yeah it's it's so rewarding to like we're all builders and to like

Deliver and support builders is fun. Just going off. You mentioned that Heroku team and you had said earlier, it was like eight people. How, how many, have you worked with any of those eight again? Or just like, what, what's that relationship with you?

Stay in touch with them. I imagine that it's like a pretty tight knit group, just being sort of the trial by fire of that group. Yeah. And so I, I ran a number of teams at Heroku. Most of my time was with the Postgres team. So when I left the Postgres team was eight people um when i joined heroku it was like we were like 25 people in total

so it was it was a small team um we employ 10 former heroku people right now at situs um some of those i worked with then some of those I like I just knew over the time like it's a small like the the tech world is small and says yours world like Heroku and GitHub were almost the same company like they I mean there was a lot of people that went back and forth between the two companies so it's uh it's a small tight knit and I'm always happy to add to it right my goal is never like a I only want to hire from ex Heroku people. It's like diversity of thought, diversity of folks, all of that matter. But it was a yeah, it's a close knit. I think there's five, four or five ex Heroku people that all live within a mile from me. And I think we're getting lunch tomorrow. So it's a it's a small, small fun time.

OK, back to the questions. Which tool or technology can you not live without besides postgres i hate it but twitter twitter it's yeah okay you're one of those yeah i i posted a picture of my halloween decorations like three months ago and someone shot me an email like oh my goodness craig i just realized you live down the street i've been following you for five years and and it's like hey can like is this weird can we like grab a coffee or a beer like this is not like like the world's small and i think twitter like make i've run into so many people that just like that like truly never be shy of dropping me a note an email like if i don't get back i'm sorry i just missed it like don't be afraid to bug me it's a

small world and people are what make this fun and twitter is i you know i don't know if blue sky is next or whatever but that's that's the one that to me is kind of like changes how we think about this and it's not transactional right it's it's relationships i i totally agree like i'm i'm in omaha nebraska and there's just like not a tech scene here and like all my all my friends are on twitter and it's just like you know we i see them at a few conferences a year but other than that it's all twitter and it's just like amazing how how much it flattens it and you can just like connect with anyone so yeah i i agree i still love it um which Which person influenced you the most in your career? That's a good question.

I don't know if there's one person. I've learned from a lot of folks. Like, when I first started, like, I grew up in Alabama, went to school in Alabama. You know, you kind of get job offers in, like, Birmingham and Nashville and Atlanta. And Accenture had a R&D group. It was like 100. Accenture is like 200,000 people. Their R&D group is 160 people worldwide. And they flew me to Chicago to interview.

One of my professors had some connection. And I interviewed in Chicago. And the job offer was for California. I'd never been to California in my life. And Chicago was like, well, if you don't want them, we do. And the partner at Accenture was like, no, I got dips. And, like, he wanted me to go right around. Like, Sanjay Mathur was amazing and, like, a great partner and manager.

And I think a lot of people kind of joined and were there for three and four and five and seven, ten years. And I was there for three and four and five and seven ten years and i was there for two and he kind of like um i don't think now he begrudges me for kind of moving on to the startup scene and actually getting to a startup and all that uh i learned a lot from bad managers and bad bosses um i don't know i mean uh at heroku adam wiggins, Adam Wiggins, I absolutely loved reporting to and working for Adam. The way he studied things on, I can't say enough about Adam Wiggins as a, he viewed software engineering as a craft and an art and not from a, like, this is how I should do it. But like, this is how I get better at it and perfect perfect at it and when he started writing uh launch announcements um james had started to like do less with heroku james was kind of the big marketing launch person at heroku one of the

founders and adam went back and read every one of james launch posts like from a scientific perspective of how do i write a blog post like a lot of people of how do I write a blog post? Like a lot of people are like, I want to write a blog post in my own voice. And like, no, he like studied what worked and what didn't. Looked at views, looked at tone of voice, looked at like opening. It was a work of art. And so like Adam would take a thing that he didn't understand and learn it to the nth degree. If you would, he hired you for this role so that he could better kind of work with you and so i think sanjay mather um adam wiggins were a couple of folks i mean i had a whole lot of amazing managers at her group peter van hardenberg or in tight like i kind of had a i was lucky i

don't know how like a who's who of like people that were amazing at heroku as managers um so there's not one person, but I definitely try to like take tidbits along the way, even the bad ones, the bad ones I've learned plenty from exactly what not to do. I think I've been pretty lucky in my career of like stumbling from point A to point B. I think now I feel pretty confident in my skills.

Like for a while, a lot of imposter syndrome, like, all right, I've done this a few times over. I actually think I know what i'm doing and it's it's weird it's really weird well but uh yeah there's a lot of folks in there that i like a lot of credit to yeah well heroku like man heroku is just so influential on on a lot of areas like you mentioned earlier how you know i probably changed postgres trajectory but just like the 12 factor app and and just get push heroku master that whole workflow and and just

like what came out of that um yeah truly an influential place and cool that you were there for that yeah it's it's one that it's it's really funny that like there's mixed opinions on like vc culture on was there a success or failure because it didn't make that like although it was acquired for a crazy multiple i like i don't think i can share that but like like a lot of people actually have opinions on heroku without knowing the details and so it's a pretty fascinating like that's a whole other one another story yeah well it definitely had an impact on you know uh a lot of other folks outside of the vc world and whatever that result ended up being. It definitely had a lot of impact on a lot of folks.

Cool. Yeah, last one. So AI being hot right now, people worried about PDoom or AI Doom. What's your probability that AI equals Doom for the human race? AI is interesting. I want more of it in my life. I don't want to start giving database support via AI. Maybe it works when I'm getting started and building how does JavaScript work and I'm building

I feel like a washed up developer these days because I'm used to jQuery and they're like, oh, we need a state machine for our front end. front end i'm like what state you click a button and it's clicked or it's not i don't i don't i'm not a front-end developer anymore these days um but i i look and yeah i i don't want ai driven support for us. I want amazing, amazing people driven support. That said, right? Like, hey, can I write two sentences and have AI, you know, elaborate on the extra pros? I think it's interesting and valuable. And like, we shouldn't ignore it and try to like brush it under a rug.

But it's where do we apply it? How do we apply it? If it's a support system and i'm paying for support i don't want your ai and that's my maybe contrarian view is like there's a huge boom of like look here's the ai tool for your database and automatically writing sql and i you know i'm i'm telling my team to think about it what's's it mean for you? But at the end of the day, like I think we're a ways off as my take, but there's,

you know, there's a lot of speculation on the space and I'm not an expert for sure. Yeah. Yeah, for sure. Yeah. I remember like last year, you know,

early last year thinking like, Holy smokes, am I going to be writing code at all in like three years? And you know, we're now we're, we're sort of traveling along the hype cycle and all that sort of thing you know you can tell it's not quite there and who knows where it's going to get but yeah i i agree with you in a in a lot of ways there um craig thanks for coming on like

just one last thing i think you changed like a lot of just like how um developer advocacy i don't want to say like developer advocacy stuff but just like the way you sort of wrote on Postgres was super helpful. I know like for me personally, like I do a lot with DynamoDB. One of the most important things I ever did was like publish this thing called DynamoDB guide. I spent like a whole Christmas on it and it was like a replica of Postgres guide,

which you had done before. And it was just like this, you know, and it was super helpful to me and like, I'm grateful for you being out there and like being a role model and influence and just like all the work you've done to help a lot of people. So thank you being out there and like being a role model and influence and, and just like all the work you've done to help a lot of people. So thank you for that.

Thanks for coming on the show. Like it's, it's been awesome to talk with you and thanks for doing this. Thanks. Yeah. I mean, it, it actually means like I'd probably do it even if no one read it,

but it's like, like the impact, it's like, I do this, I joke that I do it cause I'm lazy, but it's like, look, but it's,

it's how do I, how do I how do i scale myself right how do i like like i've been really lucky and so like it's it it's a tiny tiny way to give back to like i enjoy what we do and as developers like we it's it's kind of a fun life and so yeah i thank thanks for the the kind words and and reading and you know the the postgres promotion yep absolutely if people want to find more about you or Crunchy Data, where should they go? I definitely subscribe to the Crunchy Data newsletter, Postgres Weekly. I used to curate. I do a lot less with it. It's a great kind of

Postgres resource. Craig Kirstein's on Twitter, craigkirstein.com. On my personal blog, I write a little more about startups and things now than I do about actual Postgres. The Crunchy Data blog is a wealth of information about Postgres. So if you're like, I didn't know this about Postgres and I want to learn more, come check out the Crunchy Data resources. We've got a ton of them. I'm generally pretty easy to find, Craig Kirsteins. Even if you butcher my last name searching for it, you probably find me on the internet.

Cool. Sounds good. We'll link all those in the show notes and I can vouch for the crunchy data blog. You guys had a great post yesterday on like distributed Postgres and the different sort of framework for thinking through those. That was really helpful. I thought.

And so, yeah, go check those out. But otherwise, Craig, thanks for coming on the show. Yeah. Thanks for having me.