Postgres FM - pgrust

Episode Date: August 14, 2026

Nik and Michael are joined by Michael Malis, co-creator of pgrust, to discuss their Postgres rewrite, including reliability problems, compatibility testing, faster analytics, and a new JIT-co...mpiled query engine. Here are some links to things they mentioned: Michael Malis https://postgres.fm/people/michael-malispgrust https://github.com/malisper/pgrust The four horsemen behind Postgres outages (blog post by Michael Malis) https://malisper.me/the-four-horsemen-behind-thousands-of-postgres-outagesRebuilding Postgres for faster analytics: batching, operator fusion, and SIMD (blog post by Michael Malis) https://malisper.me/how-we-made-postgres-hundreds-of-times-faster-the-query-engine/kani https://github.com/model-checking/kaniAntithesis https://antithesis.comfsyncgate mailing list thread https://www.postgresql.org/message-id/flat/CAMsr%2BYHh%2B5Oq4xziwwoEfhoTZgr07vdGG%2Bhu%3D1adXx59aTeaoQ%40mail.gmail.comHow AI Changes the Economics of JIT Compilers (blog post by Michael Malis) https://malisper.me/how-ai-changes-the-economics-of-jit-compilers/Umbra DB https://umbra-db.com~~~What did you like or not like? What should we discuss next time? Let us know via a YouTube comment, on social media, or by commenting on our Google doc!~~~Postgres FM is produced by:Michael Christofides, founder of pgMustardNikolay Samokhvalov, founder of Postgres.aiWith credit to:Jessie Draws for the elephant artwork

Transcript
Discussion (0)
Starting point is 00:00:00 Hello, hello. This is PostGosafm. My name is Nick, PostGGGia. And as usual with me, Michael, PG Master. Hi, Michael. Hi, Nick. And we have a very, very interesting guest today, Michael Mellis, who created PGR Rust, which has already, I think, 5,000 stars on GitHub and a lot of noise around, like a lot of buzz. Hi, Michael. Thank you for coming. Yeah, of course. Thank you for having me. So, of course, I think the first question should be how it all started.
Starting point is 00:00:33 Why? Tell us the story, please. So Jason and I were looking for projects to work on, and we wanted to do something that we knew super well. And initially, what we were focused on was reliability and how can we help people make their websites more reliable. and we were working with a bunch of people and the common pattern that emerged was that a lot of the problems that caused reliability issues, a lot of them stemmed from their database
Starting point is 00:01:05 and how they were using their database, whether it was like Postgres or Redis or some other system. And so we started looking at what are ways that we can solve this problem? And we were throwing around a couple of things and we ended up figuring out that AI is actually a pretty good, at rewriting software at this point.
Starting point is 00:01:24 This is back in April. And so we're like, hey, why don't we actually try fixing these problems at the source and actually trying to modify Postgres and fix a lot of the things that cause people to have different reliability issues, whether it's connections or it's like certain queries just to take a really long time or like one long running query can take down your database or vacuums as we're all familiar with. And so the idea behind PG-Ross, the name's a bit misleading because it's not actually really about Rust. I think that's actually the least interesting part about it.
Starting point is 00:02:00 It's actually more about how do we leverage AI to re-architect Postgres and just build a much better database. Okay. And when you say better, what do you mean? Yeah. So there's a combination of like different. challenges that like I've just seen people repeatedly have with Postgres and I actually wrote a blog post called like the Four Horseman about a lot of the challenges that people have. And some of the most common ones are like you misconfigure your connection limit and like all of a sudden like nothing can connect to your database. You have like JSON support. Tons and tons and tons of people use JSON but Postgres doesn't have statistics for how to actually query JSON. So when you try to do it,
Starting point is 00:02:45 you're going to get really bad query plans and everything is just going to be really slow. There's like the wraparound vacuum and you have more than several billion transactions and like all a sudden if your vacuum can't keep up your database falls over and there's just like all these problems that have been around for such a long time and like Postgres is a great product and like it is a really great system but the way the Postgres core team approaches things is they approach it in terms of stability and how do we keep what we have today like how do we keep it working and how do we make sure we don't break it? Because there's like billions of Postgres instances out there.
Starting point is 00:03:25 And so number one priority is just how do we not break the existing stuff versus how do we actually fix the things that are not working? And so PG Rust, because it's a new project, we can take a little bit of a different approach of let's try to actually fix the things that aren't working. And maybe some of the existing stuff, it's not going to be quite as reliable at least up front as Postgres is, but at least we'll be able to fix some of the long-standing architectural issues that Postgres has.
Starting point is 00:03:56 Yeah, it makes sense. How did you do it? It's a lot of, obviously, it's with Claude, right? Yeah, it took a couple of attempts to actually figure out what is the best way to do this. And what ended up working really well was with Opus, we were able to take each file of Postgres and basically We transpile it to Rust. And so if you look at our code and look at the Postgres code side by side, it actually
Starting point is 00:04:22 looked very similar. The functions are the same. Some of the details may be a little bit different. But overall, it's actually really close to just a straight rewrite of Postgres. And so we did this across all the files. There were some things that had to be changed to actually work in Rust. One of the big ones was how memory management is done, because Rust is very particular about how you allocate memory.
Starting point is 00:04:47 But this approach of going through all the files, rewriting them, we had a bunch of plot agents, like going at this in parallel. There were like some conflicts between different files, but we were able to resolve those and then get PG-Rust to actually work. And then from there, we focus on the Postgres test suite, getting all that to pass.
Starting point is 00:05:05 And we have something that actually looks a lot like Postgres, but is in Rust. I have potentially a minor question on the, source code. Really interesting that you went far by file and that worked well. One of my favourite things in the source code is all the, is the comments and they describe in quite a lot of detail how things work. And I wonder if you've noticed how it rewrote those. I can imagine the code being somewhat reliable in terms of it being translated. But I can imagine the comments actually being harder in some way to be trustworthy. What have you found on that?
Starting point is 00:05:45 fun. Yeah. For the comments, like, Cloud has been, like, Claude is very verbose when it comes to comments. And like, every file have a whole, like, big comment about, like, what's in the file. I've actually had to give it feedback, like, I had to give it feedback to, like, comment less. And I think there's a linter that's files should not have, like, more than 5% of the line shouldn't be comments. I haven't paid quite close attention to the actual, like, content of the comments. But I do know Claude is. is making a lot of comments in the new code.
Starting point is 00:06:19 It does, yeah. So I remember, now it's version 0.2, right? Yep. Yeah, and I remember version 0.1, I think there was a claim that all tests pass, but when I wrote some silly stuff, like obviously not SQL syntax, it accepted it without any errors.
Starting point is 00:06:40 Now it's not so. In 0.2, it works fine, So it looks like PostGIS syntax is implemented very well. Was it about some lack of tests in PostGIS tests for checking negative stuff? I don't know. Or it just wasn't complete implementation before. What changed between 0.1 and 0.2? Yeah.
Starting point is 00:07:03 So PGRS 0.1 was largely rewritten by Opus. And Opus, I found to be a fine model, but it's a bit difficult to work. work with. They'll mislead you a bit about how much work it's actually completed. We would ask it to rewrite this file and it would rewrite half of the file and not all of it. And so we were working with this underlying unreliable model and like all models are like unreliable to various degrees. But with enough quality controls and layering on top of it, we were able to get all of Postgres ported and we were able to get the Postgres regression suite to pass. But that's actually a really low bar. The Postgres regression suite on Postgres only has about two-thirds code coverage. And so you can
Starting point is 00:07:48 pass the regression suite, but there's still one-third of Postgres that isn't even being tested. And on top of that, the regression suites are largely more like functionality tests that every feature in Postgres there is a regression suite for it. That's like there's one for cash joins. There's one for lateral joins. And there's 50,000 of these that basically just go through every single Postgres feature and make sure it works to some degree, but they actually just do a couple tests for each feature and then move on. And so the tests are more about just making sure this feature is there and it exists and works well, but doesn't actually really like, do you have a perfect implementation of this feature. So there is an idea to improve
Starting point is 00:08:35 regression test suite, right? Yeah. And this is already this would go to upstream. maybe, no? Yeah. So what we've been doing, the difference between 0.1 and 0.2 is 0.2 was run by Fable, which we have a more reliable model now that is more consistent about, okay, you ask it to port a file, it will actually rewrite the file completely. And so we do end up with something that is just baseline more similar to Postgres. And then before we released 0.2, we started doing a little, a tiny bit of differential testing
Starting point is 00:09:07 and fuss testing, and I can talk a lot more about that. but for 0.3, we're actually going to have a lot more of that. What we've thought of doing is we could add more regression test, but that, again, every single regression test is only testing a small part of the functionality. And what we really want is not just, hey, I can choose 100 tests and they'll all pass. I want to know is this thing actually identical to Postgres. And some of the things we've been doing, and I was very surprised by, by this that it's even worked. I would never thought this. We found this library called Connie,
Starting point is 00:09:44 which can take Russ code and C code and do formal verification over the code. Without an execution. So it does symbolic execution. So it'll run your code, but it'll run it in a special way that it is more just picking out that, hey, this is an if statement that is checking this condition, as opposed to actually having variables go through it. And then I'll be able to convert that into a format that you can and then do formal verification over that has the Rusk, they'll have an expression that represents all the rust. And then they'll have the same for the C code. And then we can do across all inputs,
Starting point is 00:10:16 are these two things going to produce the exact same output? And so Postgres has 3,000 different user-facing functions, everything from substring to regular expression matching to there's like the, like you can calculate the gamma function. There's a function for that. And so for about 1,000 of these functions, we were actually able to do actual formal verification that the Rust Code and C code are the same.
Starting point is 00:10:42 But then beyond that, what we've started, like formal verification doesn't work in all cases. It's actually narrow. And so what we've been doing for the rest of the code base has been, we'll take the Rust Code and the C code and put them side by side, and then have a fuzzer, like code coverage guided fuzzer, generate millions of inputs across the two
Starting point is 00:11:02 and make sure that they behave exactly the same. And in the process of doing this, not only did we like find over 100 bugs in like PG Rust, but we actually owned at this point a little bit over 20 bugs in Postgres itself. That's interesting. Have you reported them? Yeah. If you look, I think like end of July, like first week of August on the like PG SQL or like PGSQL bugs mailing list, like you'll see a lot of the form submissions are actually from me.
Starting point is 00:11:32 That's cool. I missed it. That's cool. Congrats actually. That's good. Yeah. None of the bugs. I found three serious bugs, but all of them had been found already by other people. I'd been testing on 18.3, 18.4 in 19 people have found them. There were a lot of not serious bugs that people had not found. One of my favorites is Postgres has a quad tree implementation, and it will use floating points to represent the positions in the quad tree. And there was a bug where, because of floating, point rounding and the arithmetic that Postgres was doing, it was possible for a point to not be in any of the four quadrants, that the comparison would say this isn't to the left,
Starting point is 00:12:17 this isn't to the right, and this isn't in the center. And we're definitely doing like very aggressive. This is in a spigist, right? I think it was in, I don't know exactly, I want to say It was in just, yeah. But basically, the amount of testing we're doing is so thorough that we're actually finding these extremely hard to find bugs. And we're finding quite a few bugs in Postgres itself. This is very important because it means that with the AI, you can, we basically have some mechanism which is finding bugs all the time, right?
Starting point is 00:12:53 And it can be maybe like, what could it mean? Probably there is Postgreas farm for testing. which is used for all releases. With this work, it may be extended to involve more AI and find more bugs at scale, right? Yeah, there's a couple things
Starting point is 00:13:12 I think that are really interesting. One is that with mythos, this whole thing about, mythos can, like, huge security risk. The interesting thing about the models is like the security, the way that they pose a security risk
Starting point is 00:13:24 isn't that like the model looks at your code and can find a bug in the code. It's actually that the model, that the model can build tools that can end up in the code. I've been using Fable and a lot of times it'll fall back to Opus 5
Starting point is 00:13:38 because it's, oh, you found a memory bug, that seems like a security issue. I think just from my experience, I also found a couple of bugs with AI and one of them. A few your security related, one was officially registered and was already patched.
Starting point is 00:13:55 And that one was just looking at the code. So model was looking. I didn't use, mythos, even not fable, it was like Opus 4.5 back then. And it just was looking at the code, it was very old code, and just looking at it, it found a SQL injection, which is there already 18 years. So in some, some, some, contribute model, which is not used directly, so nobody cared, but it was actually serious because it used as example.
Starting point is 00:14:23 So anyway, it was interesting that just looking at code, it also can find bugs. But you're right, building tools is even more powerful because just looking at the code, you can miss complex relationship between various code pieces, right? Yeah. With the models, what you can do is you can basically point them to be like, hey, I have 100 functions in this code base, write really thorough tests for all of these different pieces of code. And so the way I like to think about it with these models is if you can get them to do one task well, you now have a repeatable way to do that one task 100 times.
Starting point is 00:15:00 And so if you can get a model to be able to test one function, like there's any reason you can't get it to test 100 or 1,000 or 10,000 functions. Yeah. How much of AI capacity have you already used? Is it like just one, $200 account or you? I'm just curious, very curious. Asking for a friend, so to speak. The first version of PG-ROS, the 0.1, which was done with Opus,
Starting point is 00:15:24 which that spanned five different attempts. That cost about 100 grand. And then so far with the new version, like 0.2 and will become 0.3 in total, we've put in probably like another like 300 or 400 grand, where the, we're burning tokens so fast that you can burn through a five-hour quota on a $200 a month subscription in the order of like 15 minutes. because for a while, like, what we'll do is like we'll have 40 different fable instances, like running in parallel each like working on individual parts of Postgres. And that just burns through credits super fast. Yeah.
Starting point is 00:16:07 Yeah, that's impressive. So. And it takes also a lot of time, right? Because like it's just they are like, you just need to wait sometimes a lot, right? What was your, the longest run or loop fully autonomous? like days weeks or something. I usually will, I've had stuff like run overnight that worked, but I think I try to keep things on the order of, I try to break things down to like tasks on the order of like ideally like an hour because
Starting point is 00:16:41 any more than that I find that like the chance of the model going off and doing its own thing and like getting dis aligned. Yeah. Yeah. Yeah. I found that like models, they work really well. they enter two states. One is they are just working really well
Starting point is 00:16:54 and making forward progress. And then other times they'll just start doing their own thing. They'll just start making a bunch of random edits to code everywhere and not really go anywhere. And the more you can keep models on the productive side and not on the going in circle side is better. I'm curious on the, because that's a lot of money to spend on V0.2.
Starting point is 00:17:14 Are you being public with how you're funding it or like if you have investors? Yeah. So far, it's actually been entirely funded by me, where I had a previous startup, fresh paint, which did, like, pretty well. And I had the chance to sell some of my equity. And so far, it's mostly been funded by me. We're hitting the point where, like, it's starting to become, like, a bit unreasonable.
Starting point is 00:17:37 And we're starting to look at other ways to, like, fund money. And can we, like, be more token efficient and things like that? That makes a ton of sense. So, yeah, going back to the technical stuff, I was going to, the test suite is really interesting to me. and almost as a what can we learn from a Postgres perspective, as well as how could PG-Russ become trustworthy. So on testing the functions, I can imagine a huge amount of the kind of user surface area is going to be covered nicely, like especially for single user, single query, correctness.
Starting point is 00:18:10 But I can't help but feel nervous about like concurrency stuff. And even Postgres and some other data spaces that have had some interesting. interesting bugs come up when Jepson have got involved and that runs some like interesting tests along the like isolation level side of things. Do you does your testing approach covered that yet or do you are you going to have needed like a slightly different approach for some of those things? How are you thinking about that? Yeah. We're basically taking what is the modern and latest approach to testing and trying to incorporate all of that into how we do things. And the first thing we've been doing is we've started with the easy part of okay, here's all the peer functions. here's the ones that we can verify, here's the ones that we can just fuzz. And then most recently I've been working on how can we actually get full coverage of the database
Starting point is 00:18:59 where there's a lot of very stateful things that will only trigger bugs in very rare circumstances. And if you actually look at the issues people are reporting for PD Rust, it's usually the combination of two features of if I use a non-default coalition with certain string functions, it'll cause a bug. And so what we're trying to do now is actually fuzz until we get to 100, send code coverage. And then for the concurrency stuff, once we have a fuzzer that can actually properly explore the entire code base, we are, we purchased this tool called antithesis, which does Jepsen style fault testing. It's actually a super cool product. They run your software inside of the VM and then are able to inject faults into your running code. And so that can be
Starting point is 00:19:47 everything from the network is unreliable. your database can't talk to its replica or it can be like, hey, your server crashed and now is your data corrupted on disk or not? And they actually do stuff of they can, they have full control of the threading. It's fully deterministic. And so they can create all these weird out-of-border sequences for your threads to try to just throw the craziest stuff at your software. And right now we're working on the fuzzer to actually explore the search base. And then once we have that, we're going to give that to anthesis to then try to come up with all these crazy ways of running the code together to try to break PG-Rust.
Starting point is 00:20:21 And so I think that is, we've actually been talking with a lot of database people through this and basically they're all using antithesis for their testing at this point. Yeah, speaking of testing, I remember I noticed on Hacker News comments that on the thread that people asked questions about was a thing on and did you do rights in those tests and so on? Or it's just only from memory data fits like buffer pool or something and nothing touches the disk. Yeah, so we have, if you look at the code base, we actually have a crash simulator in PG Rust that mocks out the file system and then we'll try to just crash the database at different points. We've done a good amount of testing. The thing is that even Postgres does get
Starting point is 00:21:01 these things wrong sometimes, that there was like a Fync 8 almost like a decade ago now, I think, of like, yeah, fSync can actually fail and Postgres actually didn't handle that case, which be fair to Postgres, that's a case that never ever happens. Honestly, I also think to me, going to So Jepson tests and multi-node, basically network testing and so on, it's maybe not the biggest priority for this, maybe. Like in my opinion, I recently learned that Posgues' own tests are not checking properly disk outages, for example. So this is like, what happens if disk just has some problems?
Starting point is 00:21:40 And there are some tools to improve that in PostGar's own code base. So I'm very curious. This is purely wipe-hodod thing. You obviously created good harness to map and so on, but still people reported some sec-balls. It's obvious. It's early stage. Will it reach some point when it's very reliable so it can be put to production?
Starting point is 00:22:02 So obviously you say yes, but what's your like, why do you think so? Why do you think yes? Because I'm also very positive, so to speak. I think actually in a few months we will be able to revisit with new models and polish and so on. It gives me so joy to work with AI together, like human plus AI, you can achieve a lot. But still, there are so many doubts around. And people think, for example, it's impossible to achieve a point where it would be reliable.
Starting point is 00:22:32 You started with reliability, right? Then you said hackers are too conservative. But there's also they are too conservative to protect that reliability thing, right? now with wipe code a thing fully rewritten, won't it take 10 years to achieve very reliable state? What makes you think it's possible to achieve it much faster than 10 years?
Starting point is 00:22:56 So I think for the issues that people were reporting, I think one thing to clarify is that so far, for version 0.2, we had, or actually I think for version 0.2, that was actually before we had done even like most of the fuss testing I was talking about, where we actually sorted out those issues.
Starting point is 00:23:13 The correctness, has largely followed version 0.2, and so far we've only covered like 15 or 20% of the code base, and like there's still a lot more to cover. And so I'm like, I'm totally not surprised that people found issues in the version we released just because like we hadn't actually hardened it that much. And to answer your question of how we can actually get this to a place where people can trust this. Back to what I said earlier about, it's not like the models themselves are like reviewing all the code and like making sure it's correct. What we're doing is like we're building tools to actually make it make sure it's correct. And we're
Starting point is 00:23:43 doing it such a degree that like we're able to do millions of inputs for individual functions to the point that like Postgres this 30-year-old battle tested code base, we're finding bugs in it that no one had found before, even though people have been using it all this time. You find bugs in the old code, but maybe you have bugs which you haven't explored a new code yet. So this is a tricky question. What's your plan to prove it in production? Just wait for people who will try it in production on replicas, for example, because So you obviously like, this thing can work as a physical standby. Is this a way or use some mirroring, for example, PG-Doc has or rather maybe some others have
Starting point is 00:24:23 to mirror traffic and see that it works well? Or what's your plan to prove with production that it works? Yeah. So for proving it like aptly in production, our plan is to do exactly what you said and replicate off of people's existing databases and be a like read-only standby. That gives people a lot of the benefits of like what we're building in terms of. this like ultra fast columnar workload such that like people who are trying to do analytics in postgres and are struggling to do it they can stand up PG rust and then move to analytics over to that
Starting point is 00:24:54 and the really nice thing about analytics is they tend to be like semi-production workloads where either like it's internal usage and okay your database goes down okay my internal dashboards are broken for a little bit it's not the end of the world and then over time as like we start to get like battle tested in these not tier one database use cases, that I think will give people more confidence to, hey, this is actually something that works well. So what I like in this idea is that if PGRAST is powering a physical standby
Starting point is 00:25:27 and then it crashed, for example, but primary won't be affected, right? Because, okay, some slot is not used. And if it's working and you run very long running query on it, Usually it's a bad idea on regular standby because of Holtzlby feedback dilemma. If it's on, it will affect vacuum on the primary. If it's off, it makes your node basically a single user. It will start lagging.
Starting point is 00:25:54 So both choices are not okay if you want analytics. But it's only true if those queries last like hours. If it's already, as your blog post suggests, 300 times faster, then X-Men Horizon is not blocked by hosts and by feedback for long. It's blocked for a short time. And this is very interesting. So this looks promising, right? Like we have very fast standby,
Starting point is 00:26:22 which we have exactly like Postgres for, like primary things it's like Postgres, okay? And we have ability to run some aggregates and so on very fast. Yeah. That's interesting. Actually, the thing I find really interesting about, do rest is like there's actually a whole bunch of like unexplored design space or like new way that we could like you can do things where you mentioned for instance that like when you have a replica like due to the way like the the wall replay works there's all these like complexities about how do the vacuums work and how does like the replica like fall behind but I think there are options out there. I haven't explored them thoroughly but there's options out there. If we change the file format and the way the wall replay works, maybe there's ways to actually get around these problems. I think this is a lot of what's really exciting about PG-Russ is like there's a lot of like unexplored design area. And because it's this like experimental new thing, there's actually lots of opportunity to explore, explore the design space.
Starting point is 00:27:22 Right. But that's for sure. So like experimental, like you can go very far. But grounding is when you put it to production. Right. And this is the most interesting practical piece here for me. It sounds like it's a relatively. low risk to try it. There's a way, right? That's interesting. So yeah, that's cool. What else?
Starting point is 00:27:46 I saw you blog posted, I think, today, right, about JIT just in time. What attracted your attention? It was just switched off by default finally, recently. Yeah. As part of the columnar stuff, we looked at a bunch of the recent database literature and incorporated a ton of stuff into PG-Rust. A lot of it actually coming from the research into Umbra, which is this very experimental database being developed in by this team in Munich. And they've been publishing database research for the last decade more, and a lot of it comes from them.
Starting point is 00:28:21 But for Jig compilation, and this is a lot of what I talked about in the post that came out this morning, if you look at all the databases that have Jig compilation in it, they either use LVM or they use the right, C and C++ code and then compile that. And the big downside of these approaches is that there's actually a huge amount of latency overhead to compile this code. For C and C++, or for LVM, I think it's 50 milliseconds compile time. And so you're very limited in the cases that you can use it and you have this risk of if the planner estimates the query long, you now have this extra 50 milliseconds of latency. But there was this approach to jig compilation called copy and, I think like copy and fix,
Starting point is 00:29:06 or something along those lines that was released in someone published a paper on it in 2021. That makes a copy and patch. Yeah, that's it. It's much easier to write that you just have templates of assembly code and then you just fill in the bits as needed. You can combine these templates together to get the code you want. And so the PG-Rust jig compiler actually doesn't use something like LV. it actually just directly generates the assembly. And so because of this, the compile times are like five microseconds,
Starting point is 00:29:37 as opposed to 50 milliseconds for LVM. This gives us a lot more, like we can jit compile a lot more of the query that we actually, for our executor, we jic compile large parts of the query, not just expressions, which Postgres will need jit compile expression and tipple deforming. But we actually do a lot more than that. And so we get measurable speedups in many different places. Do you see any reason Postgres couldn't use a similar approach? So one of the big, one of the challenges I see with Postgres to campus approach is I specifically am targeting only the Graviton instruction set.
Starting point is 00:30:24 Where PG-Russ is being built in a very different world where today most people are. running their databases in the cloud. And for instance, there's no Windows support in PGRust yet. And so I'm able to, because I know, I'm just assuming that people are going to run this primarily in AWS and GCP. I'm able to target one specific CPU design and one specific instruction set and focus just on that. And make that work really well. And for Postgres, I think it's still possible. You could have a digit compiler only work in certain instruction sets or you, like, you can have different backends for
Starting point is 00:30:58 different instruction sets. But it's an order of magnitude, more work to get that to work well than what I'm doing. And even if it's not in cloud, they will run it on MacMews they have now, right? Anyway. It should work anyway there.
Starting point is 00:31:16 I've seen multiple people with like rooms of like they'll have a bunch of could be good postgast cluster. So that's interesting. And you don't use GPT like codex, no? like distributing work goals So in my experience, actually
Starting point is 00:31:33 a lot of the early work we were doing with Codex, like Codex 54 and Codex 55, the thing that made me switch to Claude is they release this feature called Dynamic Workflows, which this is why it was used for the Bunn Rust Rewrite. And Dynamic Workflows are, you can basically, it will write code that orchestrates a bunch of different agents. And so you can be like, hey, I have these 10 files. Can you rewrite all of them? And it'll
Starting point is 00:32:03 write some JavaScript. They'll then spin up 10 agents. They'll then rewrite each file and then review it and fix any issues and then merge it. And so that level of orchestration made it way easier to use Claude. And then in my experience, like Fable just works. It's like way more reliable and trustworthy. One of the actually like biggest challenges I had with the rewrite. or like one of the things that was actually really hard was the Postgres regular expression engine. I don't know what it is about it. I think it's I think it's 10,000 lines of code. It's like a decent amount of code but not too big. And I was doing this as a benchmark and I threw soul at it on like medium level thinking, not like the highest thinking, but like on like a decent like soul's like a pretty good
Starting point is 00:32:49 model and soul wasn't able to rewrite it in rust. I was like very surprised by that. I was like fully expecting that given how smart soul is, like, it should just be able to do that. Whereas Fable was like a whole level above that where it was able to rewrite in Rust and then it was able to do performance optimization on it to make it make it a tiny bit faster than the Postgres version. And so like, I think like it probably is possible to use like Soul, but like I've just found the difference in autonomy between the two models is like not worth the difference. I think we need to talk about licensing. The AGPL, yeah.
Starting point is 00:33:31 That's always controversial, but at least open source, right? Why pick that one? I think I know why, but why? Yeah, we're looking at the different options we had for licensing, and there's like four different options. There's, you have MIT or Postgres. You have like AGPL or like GPL where it's like the copy left. you have what they call
Starting point is 00:33:53 like source available like you have SSPL and like BSL and you have like close stores and we knew we wanted to do something open source and so it limited us to like the permissive or the like AGPL and the thing was like we wanted
Starting point is 00:34:09 something that like a third party could just take and sell themselves to the detriment of like the work we're doing where like there's lots of like closed source works at Postgres. You have LODB, you have Aurora.
Starting point is 00:34:26 Now, like, neon is closed source. And none of that money that, like, those platforms make actually goes back, like very, very tiny amounts of it, goes back to the actual Postgres core projects. And that was just something that, like, I wanted to avoid with PG-Rusts. But the JPL doesn't protect you from selling.
Starting point is 00:34:44 Anyone can still sell if they don't modify code. And even if they modify code, they just need to publish it. That's it. Yeah. Yeah. So at least the history for AGPLs, it is like extremely, basically no one has really done that.
Starting point is 00:34:56 But like just the cost of needing to open source your modifications to it has deterred, like, Amazon and Google from offering any AGPL software on their platforms, at least so far. Makes sense. So that seemed like the right tradeoff. Good. I know we're like almost out of time. It was interesting discussion. Thank you for coming, Michael.
Starting point is 00:35:17 I enjoyed it very well. Good luck with your project. we will be keeping an eye on it. Yeah, of course. Thanks a lot. Thank you. Have a great week.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.