CoRecursive: Coding Stories - Story: The Untold Story of SQLite

Episode Date: July 2, 2021

On today's show, I'm talking to Richard Hipp about surviving becoming core infrastructure for the world. SQLite is everywhere. It's in your web browser, it's in your phone, it's probably in your car, and it's definitely in commercial planes. It's where your iMessages and WhatsApp messages are stored, and if you do a find on your computer for *.db, you'll be amazed at how many SQLite databases you find. Today, Richard is going to share his story. It's the story of creating a small open source project and having it grow beyond your wildest ambitions. It's the story of following that success wherever it leads: From relationships with tech-giants to interesting testing procedures and more. Episode Page Subscribe To Podcast Newsletter  

Hello, and welcome to Co-Recursive. I'm Adam Gordon-Bell. Each episode of Co-Recursive, someone shares the fascinating story behind some piece of software being built. On April 1st, 2014, an open source maintainer got an email from Google about a security issue. And this was not an April Fool's joke. This was Heartbleed, and the project was OpenSSL. 17% of the world's web servers were affected, and by the time the dust settled, people started asking questions like, why was this open source

maintainer who received only $2,000 in donations a year responsible for 17% of the world's encrypted web traffic? Ever since then, I've been curious about these critical pieces of infrastructure. What happens if your fun side project ended up powering the world? Do you try to monetize it? Do you focus on it full time? Does the weight of the maintenance crush you and you just leave computers and go to focus on building furniture? I have a perfect guest for discussing this topic. I'm Richard Hipp, and I work on SQLite. Today's show, I'm talking to Richard about how to survive becoming core infrastructure for the world. SQLite is everywhere. It's in your web browser, it's in your phone, it's probably in your car, and it's definitely in commercial planes. It's where your iMessages and WhatsApp messages are stored.

And if you just do a find on your computer for star.db, you'll be amazed at how many SQLite databases you find. So today, Richard is going to share his story. The idea for SQLite actually came out of his frustrations with an existing database called Informax that was installed on a literal battleship. Richard was a contractor for Bath Ironworks, working on software for the DDD-79 Oscar Austin.

That is a battleship, the type that protects a fleet by being armed to the hilt. There's a big complex ship, and stuff's always breaking. Suppose a pipe ruptures. You need to isolate that damage by closing valves on either side of the pipe but then you also need to open valves elsewhere to restore the working fluid to other systems that are downstream so that they don't go offline and locating all those valves

and whether you open them or close them can can get very complicated. And so automated common diagrams is a program that says, oh, here's the problem, here's the valves you close, here's the valves you open, here's where they're located. And that was the original problem. And all the data for where all the pipes are running and all the valves are located, that was in the database. The computer was already installed on ship. We didn't have any control over that.

The database was already installed on ship. We didn't have any control over that. The database was already installed on ship. We didn't have any control over that. We just had to use what was there. Richard was brought in because the solution to this problem was computationally complex, and Richard was known for solving hard problems. Really, when you come right down to it, the types of systems that are designed by humans tend to be solvable in polynomial time. It's just the general description of the problem where you have an arbitrary directed graph is NP-complete. So they were trying to write code that would solve this, and they hadn't analyzed it, and they didn't realize this. And they were, you know know we're not getting a solution just running forever and chewing up cpu cycles what's going on well you know that's because it's

mp complete and and so you have to use heuristics that will find fast approximate solutions and and put lots of things in there to to verify that it's not stuck in a loop somehow. And really, for the way they design these ships, the heuristics can find the exact solution, the optimal solution, pretty quickly in every case. But it's just you can't write a simple, naive algorithm and expect it to finish quickly because you will get stuck in an exponential surge. Try every valve combination

to see which one's going to give you the best solution. So I was leading a team that was working on this, but Informix just wasn't really working really well. I mean, once it was working, it worked great, but sometimes the server would go down. And then our application wouldn't run, and that was embarrassing. Dialogue box would pop up. They'd double-click on the was embarrassing. Dialog box would pop up.

They'd double-click on the thing, and a dialog box would pop up that says, can't connect to database server. And it wasn't our fault. We didn't have any control over the database server. I mean, what do you do if you can't connect to the server? And so we got the blame all the same because we were painting the dialog box. Yeah, I can imagine when some pipe bursts and they try to use your program

and they get a database error, that they're not too happy. No, no. And of course, it's a warship. So of course, things are always breaking and they use it all the time. But the idea is it's supposed to be able to work if you take battle damage.

So it's more than one pipe breaking. There's going to be a lot of stuff broken and people are going to be crazy and there's going to be a lot of stuff broken and people are going to be crazy and there's going to be smoke and blood and chaos. And it's just got to, and in a situation like that, they don't want to dialogue boxes. It's cannot connect to database server.

That's just not, not what they want to see. So it needs to be reliable. So, you know, all we're doing is reading the data into RAM. We're not doing transactions. We're not doing transactions. We're not doing anything like that.

It's just we're pulling a bunch of data into memory so that we can solve this problem. Why do we even need a server? Why can't I pull this directly off the disk drive? That way, if the computer is healthy enough that it can run our application at all, we don't have dependencies that can fail and cause us to fail. And I looked around and there were no SQL database

engines that would do that. And one of the guys I was working with says, Richard, why don't you just write one? Okay, I'll give it a try. And I didn't do that right away, but later on there was a funding hiatus. This was back in 2000, and if I recall correctly, Newt Gingrich and Bill Clinton were having a fight of some sort, and so all government contracts got shut down. So I was out of work for a few months, and I thought,

well, I'll just write that database engine now. This is the year 2000. Wikipedia didn't exist yet. People with internet were mainly using dial-up and only 1% of US households had broadband internet. You couldn't just Google how to build a database and get pointed in the right direction. But Richard had a plan based on his previous experience building compilers. If we look at each, think about each SQL statement as a program, my task is to take that program and compile it into some sort of executable code. So I wrote a bytecode engine that would actually run a query.

And then I wrote a compiler that would translate SQL into that bytecode. And voila, SQLite was born. And it wasn't really used for that project that I was working on because that one was shut down at the time. It started back up later and we incorporated it into that project for testing purposes because the customer was insisting on Informix. That's fine, but Informix is a real hassle to use for development. So for development purposes, we would use my engine just for testing and whatnot.

But it was never an official part of the project. But I put it out there on the internet, and other people started picking it up. I remember this was before Twitter or anything like that, but there was NetNews back then. And I read somebody posting on there. I said, wow, I've got an SQL database engine running on my Palm Pilot. Not kidding, you know. And so it really attracted a lot of attention, and that encouraged me to work on it. So Richard kept tinkering with his database project on the side

until he got a phone call from a tech giant. It was from Motorola. Now back 2002, 2001, when this happened, Motorola was one of the tech giants. These days, the tech giants are Apple and Android and Google and Motorola and Nokia. And so I got a phone call from some people at Nokia, from Motorola. And they said, listen, we're designing a new cell phone operating system. We want SQLite to be part of it. Can you support this for us? And, of course, I was real cool about it.

I said, oh, sure, sure, I can do that for you. But I said, well, do you have any pricing information? Well, I tell you what, let's have a call tomorrow, and I'll get back to you on that. Of course, inside I was like, what? You can make money with open source software? What is this? How does this work? How do I price this? I have no idea how to do this.

But I scrambled around and came up with some pricing strategy they wanted some enhancements to it so they could go in their phones and i gave them a quote and at the time it you know i thought i thought this was a quote for all the money in the world it was it was just shit what was it it really was there i think it was 80 000 or something like that it was it was not very share? I think it was $80,000 or something like that. It was not very much money by today's standards, but for me, that was everything. And I brought three other guys that I'd worked with on to work on it. We worked that project, and that was sort of the beginning.

After Motorola, the next tech giant to reach out was America Online, who wanted Richard to visit their office and talk about a contract for some enhancements they needed. This was back when AOL was the world's leading service provider. Maybe you're too young to remember, Adam. We used to get CDs in the mail. $10, yeah, the CD. Put this in your thing, and $10 a month. And they needed a database on that CD, and they had some ad hoc thing, and they wanted to use SQLite on that. They had limited space, and so, hey, we need to put this on the CD. I just put in this new feature, which I thought was a really cool idea, that you could create a temp index on a real table. So it's a table that's shared amongst processes. You can create a temp index on a real table. So it's a table that's shared amongst processes. You can create a temporary index on it.

And I thought that was a really cool idea. And I was up there telling them about this. And mid-sentence, as I was getting ready to explain what this is a great idea, it suddenly occurred to me that if you have a temporary index, only one of the users of the database sees that index. And if somebody else, one of the other users, updates the primary table, the index will go stale. And I was sitting in their office. We had several people there.

They were getting ready to hire me to do something. And I was getting ready to brag to them about this new feature when it suddenly occurred to me that it was fundamentally broken. And so mid-sentence I had to change my thought and come up with something else to say. That's awesome. So is that how the engagements usually happened? So did they ask you

to come down to their... Yeah, that's how that happened. One of our early big customers that really set us on the road to making this work well was Symbian. The Symbian OS was the operating system on Nokia phones. And they gave me a call, and they wanted me to fly over to London to their headquarters. We want to have a discussion. And the meeting was going to be on – they didn't know this.

They gave me the date. It was Thanksgiving Day. I didn't tell them because they gave me the date. It was Thanksgiving Day. I didn't tell them, because they don't do Thanksgiving in UK. You don't do Thanksgiving in Canada. We do. We have it on a different day, though. Okay, well, yeah, but you don't do the

American Thanksgiving. Sure, I'll fly to London on Thanksgiving Day and have a meeting there. Traditionally, we go to my wife's family for Thanksgiving. That year, she showed up to the big family, because all the cousins are there and everything. She shows up there without me. Everybody's saying, uh-oh, is there something wrong? You have to go explain to them, oh no, Richard's

in London at a meeting. They flew me over. Come to find out that they had a big bake-off. They wanted a database engine for CME and OAS and they had a big bake-off. They had 10 different database engines, two other open source entrants and seven proprietary entrants.

And they'd run them all on a data set to see which one would best suit their needs. And SQLite had won. All the other nine, they'd given the vendors opportunity to tune, but we won. Who knew? So they flew me over and said,

hey, yeah, this is great. We want this, but we need some enhancements. Great. And we cut a contract to do some development work for them. And at this point, we were really into this. And I was kind of doing this full time. But they came to us and said, look, we need you to increase the bus factor. The bus factor means the number of people who have to be hit by a bus in order to stop all development work. And they felt like the bus factor on SQLite was too low. This is their words, okay? So they wanted us to start the SQLite

Consortium, which was a way of funding the project and getting more people involved to guarantee that it was going to be available long term. So we started this big thing, and I came up with a bunch of ideas. And it was this crazy thing where all the consortium members had voting rights, and it was this big thing. Somehow or or another i don't know how this happened mitchell baker she's the woman who runs the mozilla foundation she got wind of this and called me up says richard you're doing this all wrong let me tell you how to set up a consortium and she just says look she laid down the losses look the the developers have to be in control. Their decision is final. No voting rights on what gets to go into it.

The companies that are using it, they get the honor of contributing money, but you make all the decisions. She was very adamant about this, and she laid out everything. She's a lawyer. And I'm like, how do we get people to join? What's the incentive? She said, don't worry about that. They's the incentive? Don't worry about that. They will join. Just don't worry about it.

And in fact, if you do this, Mozilla will be one of the founding members. Put it out there. Sure enough, we got support from Mozilla and Symbian and Adobe. And they started the SQLite consortium with those three. But we're very blessed to have been in this. Like I say, I didn't plan any of this. When Simeon said, oh, you need a consortium, I didn't know what to do. I don't know how Mitchell Baker heard about it

and literally phoned me up and said, you're doing it wrong. I don't know how that happened, but it did, and it's been such an amazing journey. And because we would not be in this situation were it not for these circumstances. By this time, all major smartphones were using SQLite. And so Richard got to see early smartphone development from all sides. Embedded development, like building a smartphone, can be a slow process. Long iteration loops, time spent

waiting for things to be flashed onto new prototypes, people working off breadboards that look nothing like finished products. Then he was contacted by Google, an outsider to cell phones and embedded development. This was back in 2005 or so, and we were in meetings with Android. And this was before Android was a thing. This was before iPhone. Yeah. And I was in Mountain View, and we were working with a fella. And they had a prototype of their Android phone. And this was before iPhone.

So back then, the phones, they had the full corded keyboard at the bottom. Yeah. And a smaller window up on top, like a BlackBerry. But we were debugging something with SQLite, and we were plugged into the phone, and we were running the debugger on a workstation, which was pretty amazing. Nobody else could do that.

But we had the phone in the debugger, and the phone rings. And he looks at it, and he says, oh, this is my wife. I have to take this call. Excuse me. And so I left the room so he could have this conversation. But, you know, and of course I stayed completely cool about this.

But inside my mind was exploding because here we were. We were debugging an application in GDB on a phone that was on the public network and this was utterly mind-blowing nobody at motorola nobody at symbian nobody at nikia had anything close to that and in that one moment i knew that android was going to to be huge How did they work on phones like Nokia? A much slower process, I assume. There was a much longer cycle time between hardware and software upgrades. And that was the thing,

is that the Android people were burning a new operating system and pushing it onto working phones that they were carrying around and working on the public network multiple times per day, whereas that was a 30-day cycle for anybody else. I was under Indiana. It was clearly a prototype. I mean, the case looked like it had been 3D printed.

I mean, it wasn't like it was a production-ready but it was you know portable whereas the the engineers at other companies they were they had the big breadboard prototypes you know a big full-size prototyping board in the and the phone would run on that and it was not connected to a radio so they couldn't actually use it as a phone i could not not tell anybody. I could not go to Motorola. I could not go to CME and say, guys, this is important. You need to fix this. I couldn't tell them. And that made the difference.

That was kind of their undoing, I'm afraid. By this point, Richard's database is really picking up steam. He's a talented guy, and I'm sure his team is super capable. But some software consulting company of one to four people supporting the install base of every Android and Motorola and Nokia phone. And this is a database. Developers are hard on databases, and people don't like it when their data has issues. We were going around boasting to everybody naively that SQLGLI didn't have any bugs in it or did no serious bugs, but Android definitely proved us wrong. And look, I used to

think that I could write software with no bugs in it. It's amazing how many bugs will prop up when your software suddenly gets shipped on millions of devices. It is truly amazing. They were getting crashes constantly. And at about that same time, or slightly before then, I'd been doing some work for Rockwell Collins, an avionics manufacturer, Rockwell Collins, and they introduced to me this concept of DO-178B. It's a quality standard for safety-critical aviation products. And unlike so many quality standards, this one's actually readable.

Now, it does have a lot of bureaucratic stuff in it, but you can actually buy a copy of this. You do have to buy it. It's a couple hundred dollars. But it's a reasonably thin volume, and you can read through it. And with sufficient study, you can understand what they're talking about. And so I did that, and I actually started following some of their processes and one of the key things that they they they push is they want 100 mcdc test coverage that's modified condition decision coverage of the code your your tests have to cause each branch operation in the resulting binary code to be taken and to fall through at least once.

Oh, so it's at the level of the actual assembly, at the machine code level. At the machine code level, yeah. Actually, the test, MCDC is a little stricter than that. Let's not dwell on the details, but I had this idea, I'm going to write tests to bring SQLite up to the quality of 100% MCDC. And that took a year of 60 hour weeks. That was hard. I was putting in 12 hour days every single day. I was just getting so tired of this. Because with this sort of thing, it's the old joke of you get 95% of the functionality with the first 95% of your budget and the last 5% on the second 95% of your budget. It's kind of the same thing. It's pretty easy to get up to 90% or 95% test coverage.

Getting that last five percent is really really hard and it took about a year for me to get there but once we got to that point we stopped getting bug reports from android oh wow yeah so it just worked from there on out it made a huge huge difference we just didn't really have many bugs for the next eight or nine years. How many test cases is that? What does that look like? Well, the first test was just written in TCL. And that was normal developer tests.

And then after that, you had the airplane ones? How much is that? Right. Well, we still maintain the first from the TCL tests. They're still maintained. They're still out there in the public. They're part of the source tree. Anybody can download the source code and run make test and run all of those.

They don't provide 100% test coverage, but they do test all the features very thoroughly. The 100% MCD tests, that's called TH3. That's proprietary. I had the idea that we would sell those tests to avionics manufacturers and make money that way. We've sold exactly zero copies of that, so that didn't really work out. It did work out really well for us in that it keeps our product really, really solid, and it enables us to turn around new features and new bugs because it's very fast.

How do you count tests? I mean, we'll have a test case, but it'll be parameterized. So that one test case might run 100, 1,000, 100,000 tests just by looping, by changing one of the parameters. For a typical release, we do a billions of tests but we have i think it's on the order of a hundred thousand distinct test cases so a hundred thousand distinct test cases and then they're parametrized so then how many so we'll do billions of tests oh wow yeah it takes we have a checklist and we will run tests for at least three days prior to release.

Like, is that on one machine, or do you, like... No, but we intentionally test on different operating systems. I've got some old equipment around here, because these days all CPUs are little Indian. Yeah. Intel by order. But, you know, it wasn't so long ago that there was a lot of big Indian CPUs around.

So I've got a PowerPC iBook up here that we run tests on. So we make sure that SQLite works correctly on a big Indian platform. We test on 32-bit platforms. We test on ARM. We test on Intel chips. We test on 64-bit platforms. We test on Windows and Linux and Mac, OpenBSD.

So we test on as many different platforms, as many different operating systems as we can. But we also have lots of different test suites that we run. We've got the original TCL test. We've got Teach 3. we've got the fuzzers that are running constantly, we've got a thing called SQL logic test. Shane Harrelson did this for us about 10 years ago. He came up with this huge corpus of SQL statements, and

he ran them against every database engine that he could get his hands on. And we wanted to make sure everybody got the same answer. And he managed to segfault every single database engine he tried, including SQLite, except for Postgres. Postgres always ran and gave the correct answer. We were never able to find a fault on that.

Now the Postgres people tell me that we just weren't trying hard enough. It is possible to fault Postgres. But we were very impressed. I mean, we crashed Oracle, including commercial versions of Oracle. We crashed DB2. Anything we could get our hands on, we tried it and we managed to crash it. But the point was that we wanted to make sure that SQLite got the same answers for all of these queries or equivalent answers, because a lot of these queries have they're indeterminate in that the rows might come out in a different order, because

order by clause. We wanted to make sure that all the database engines got equivalent answers. Mostly we wanted to make sure that SQLite was getting the same answers as everybody else. So that's another test suite, and then we have lots of smaller ones as well. So between them all, it's a lot of testing code, we have lots of smaller ones as well.

So between them all, it's a lot of testing code, and it takes a long time to run. It's these tests that allow SQLite to continue improving and evolving. They let Richard be confident in each release, even if large parts of the code base change. And large parts do occasionally get rewritten, as improved ways to structure things are discovered. Remember, Richard built SQLite from first principles. Originally, when I set out to write it, if I went around looking, is there a reference

on how to write an SQL database engine? I found nothing. And so I just had to kind of invent it myself. So it was a completely independent invention. Most of the theory was happening at MIT and Harvard, up in Cambridge, or down at Berkeley. But if you didn't go to one of those three schools, you didn't even know this body of theory existed. And there wasn't a real good way to find it. It's curious, though, if you go back and look at the Volcano model used by Postgres and the Bicode model used by SQLite, we converged on all the same ideas.

They start out very differently at the top, but we converged on all the same ideas of how to make it go faster. So I think that that's kind of a validation of the theory. Two independent lines of development came up with the same answer. I had never heard of, for example, a covering index. I was invited to fly to a conference. It was a PHP conference in Germany somewhere. And because PHP had integrated SQLite into the product, they wanted me to talk there. So I went over and I was at the conference.

But David Axmark was at that conference as well. He's one of the original MySQL people. So David was giving a talk, and he explained about how MySQL did covering indexes. And I thought, wow, that's a really clever idea. A covering index is when you have an index, and it has multiple columns in the index, and you're doing a query on just the it has multiple columns in the index and you're doing a query on just the first couple of columns in the index

and the answer you want is in the remaining columns in the index, when that happens, the database can use just the index. It never has to refer to the original table. And that makes things go faster. It only has to look up one thing. It becomes like a key value store, but just on the index, right? Right, right. So on the flight home, on a Delta Airlines flight, it was not a crowded flight.

I had the whole row. I spread out. I opened my laptop. And I implemented covering indexes for SQLite mid-Atlantic. That's awesome. But you just pick things up. People tell you these things. And that happened to some with Bloomberg.

They'd come to us and say, hey, why aren't you doing this optimization? And I'd come back and say, never occurred to me. Well, can you do it? Let's see what we can do. And then it would go in. So, yeah, it's kind figure it out as you went along. I had to invent a lot of this myself.

I'd never, nobody ever taught me about a B-tree. I'd heard of it. When I went to write my own B-tree, on the bookshelf behind me is, I've got Don Knuth's The Art of Computer Programming. So I just pulled that down. I flipped to the chapter on searching and looked up B-trees and he described the algorithm. That's what I did. Funny thing, Don gives

us details on the algorithm for searching a B-tree and for inserting into a B-tree. He does not provide an algorithm for deleting from the B-tree. That's an exercise at the end of the chapter. So before I wrote my own bee tree, I had to solve the exercise at the end. Thanks, Don. I really appreciate it. That's awesome. Did you pull anything else from that book? Well, I mean, it's an amazing volume. I mean, I can't give you a specific example, but, you know, from my era, everybody has to have read or at least skimmed through, at least browsed through the art of computer programming and know the algorithms that are there. Maybe not Don's exact implementation.

I mean, I never took the time to learn Nix, which is, you know, a sibling language. But it's useful to flip through and look at all the algorithms he talks about. I think, yeah, I think that just a year or two ago, I needed a pseudo-random number generator. And let's see what Don recommends, you know? You pull it off, you see what he does. Building your own random number generator, starting from an algorithm description in the art of computer programming that you literally have on the shelf

behind you in your office, it seems like such a different way to work than I'm used to. SQLite has basically no dependencies that Richard didn't build himself except the C compiler and a couple of things from libc. He even built his own source control system

and his own bug tracker. What this gives Richard is freedom. Have you ever thought about the word freedom, what that means? Freedom means taking care of yourself. People go backpacking or stuff. They go on these long, you know, hikes, backpacking, where they carry everything they need on their back. And they talk about how freeing that is,

because they are taking care of themselves. That's what freedom means. And so when you write the whole thing yourself, there's a certain element of freedom in that because you're not tied to somebody else. You're not dependent on a different vendor providing you something. suppose I had elected to go with Berkeley DB as the storage engine for SQLite version 2. Well, at that time, Berkeley DB was open source, but then later it was sold off to Oracle, and it became sort of a dual-source proprietary model, and you can't get hold of the source code to the later versions without paying license fees, yada, yada, yada. So suddenly it becomes a problem. The parser generator in SQLite is one that I wrote myself years ago called Lemon. And because rather than using YAC or Bison, I use my own. And you think, well, why did you do that? Well, it turns out that

when I needed some new language features that you couldn't get out of Yacker Bison, because I own the parser generator, I could change it to do what I wanted. When I needed a version control system for SQLite, you know, it started out using CVS because back in 2000, every project used CVS. That was the dominant version control system. But moving on into the mid-2000s, we needed something better, and the whole idea of distributed version control had become a thing. And I looked at Git, I looked at Mercurial, and I looked at my requirements.

I thought, you know what, I'm just going to write my own. So I wrote my own version control system, which is now a project unto itself. And that has worked out very, very well, because Git... Think. Linus Torvalds wrote Git to support the Linux kernel, and it is perfectly designed to serve the needs of the Linux kernel development community. If you're working on the Linux kernel, Git is absolutely the perfect version control system. Now, if you're working on something else, though, maybe not so much.

And so it's a perfectly atrocious version control system for working for SQLite. Fossil is absolutely the perfect version control system for working in SQLite. Fossil is absolutely the perfect version control system for working in SQLite. And I wrote it for that purpose. And so because I wrote it myself, it exactly meets my needs and is the perfect product for what it's doing. So by doing things yourself,

you control your own destiny. You have more freedom. You're not dependent upon third parties. It's a very liberating experience. people living off-grid. They go out and they buy a little bit of land and they're going to raise their own food and they're going to live off-grid and stuff. What do they call that? Survivalists or preppers? Preppers, maybe. Maybe it's preppers, but they're not necessarily looking

for the end of the world. They don't want to depend on Trader Joe's. They don't want to depend on the local food store. They want to be more self-sufficient. And I guess there's an element of that in me. I mean, we live in the city. I can't get by without Kroger. If Kroger and Aldi and Publix go under, I'll starve. I'm not to that extent.

But I do appreciate that people want to control their own destiny and do things for themselves. Right now, I'm having some trouble with Gmail. I know that when we were setting up this meeting, you noted that a lot of the attempts to reach me by Gmail are bouncing back. And I've been struggling to figure out how to solve that. And I'm thinking, and so now, where does my mind go naturally? I'm going to write my own mail server. I was making notes on that, even as we were setting up this call. That's a big problem, and that's at least as difficult, if not more difficult, than writing a database engine. But, you know, I don't want to be beholden to Gmail. I don't want them controlling my destiny.

I don't want them controlling the record of all of my conversations. I want to control that myself. And so I'm going to go through a lot of pain and a lot of work and a lot of effort to come up with some solution that I can control myself. I can go out and lease a virtual machine out there in the cloud and run it myself and not depend on a third party to control my email. Like your backpack example, I really like. One thing I get from it is that freedom involves discomfort or pain. You have to carry these things on your back.

It involves doing without. It's attempting. Here's a warning to all your listeners. Whenever a politician comes to you and says, oh, we're going to take care of this problem for you, what they're really saying is, we're going to take away some of your freedoms. The exercise for you, the hearer, is to figure out what freedoms they're going to take away in order to solve this problem. And they're often well disguised. But yeah, if you want to be free,

that means doing things yourself. I mean, it's a trade I'm happy to make lots of times. Like I have somebody cut my grass. I do too. Yeah. Like I give, I'm losing freedom over the exact way it's cut or whatever. On the other hand, I still have a lawnmower in the garage. And so if for some reason that arrangement ceases to work well, I can take it over myself.

If my lawn people disappeared, I'd be in trouble because my push motor is broken. But I'd be in more trouble if SQLite disappeared. Richard's database is the most used database in the world. And by some counts, it's the most widely deployed software module of any type. If it disappeared, your web browser wouldn't work, your smartphone probably wouldn't start up, and probably your car wouldn't start up as well. Its impact on the world is massive, and there's plenty of places where it could have lost its way. The consortium could have stifled progress or run out of money, or the full year it took to address all the Android bugs could have easily burnt Richard out. But he prevailed, and now he's in a great position to offer advice to others who want to create impactful open source software.

I had this crazy idea that, you know, I'm going to build a database engine that does not have a server that talks directly to disk and ignores the data types. And if you ask any of the experts of the day, they would say, that's impossible. That will never work. That's a stupid idea. Fortunately, I didn't know any experts. And so I did it anyway. So this sort of thing happens. I think maybe just, you know, don't listen to the experts too much and do what makes sense. Solve your problem. All right. That was the show. I hope you enjoyed Richard's story. If you're listening to this and you haven't subscribed to the podcast, please do.

Also join my newsletter. The signup form is on the website, on the episode page that's linked in the show notes. I'm going to be sending out an email when new episodes are out. And also, I've got some written thoughts to share. A newsletter is obviously a better format for sharing those. So please sign up. Much like the podcast, this will be a low frequency newsletter. Until next time, thank you so much for listening.