The Data Stack Show - The PRQL: Do We Need A Dedicated Database for Every Use Case?

Episode Date: January 16, 2023

In this bonus episode, Eric and Kostas preview their upcoming conversation with Joran Greef of Tiger Beetle.

Welcome to the Data Stack Show prequel, where we talk about the show we just recorded to give you a sneak peek. Costas, the big question that this show starts to answer, but that I'm going to ask you to tease the audience, is do we need a use case specific database for every individual use case out there? And the context is that we talked with Joran

from Tiger Beetle, which is an extremely fast database for double entry accounting. And so the question that pops up, which we talked about on a recent Shop Talk episode is, why so many databases? So do you think that this is an anti-pattern or do you think that this is the way that things are going in the future? That's a very good question. For me, it's very exciting to see that this is happening. I don't know if I am just biased because I just like that stuff, right? At the end is the market that has to decide if it makes sense or not.

So we have to wait longer to see that. But I think it is time for, if we manage to get to a point where, okay, it's not like we are doing to have a database system for every like small use case out there, like with a huge, a long tail of use cases, but there are a couple of them big enough, I think, where having specialized database systems like makes a lot of sense. And as we reach a point where we cannot solve problems anymore by just throwing more servers there, just because there's going to be the acceleration in the way that like we progress with hardware.

We will have to focus more on optimizing and building solutions for specific problems. So yeah, like for me, it feels like it will happen eventually, especially as we have more and more tooling around, you know, having the fundamental tools out there to build and compose database systems. And not having to build everything like from first principles and have to wait like for a decade, like to get something to market, right? So it, it, it feels like the stars have aligned. That's what I'm saying. the stars have aligns that we'll see that's why like I also appreciate

you know attempts like Tiger Beetle where yeah they have this beef that it makes sense

to do that and they go after it although it might be like contrary something that is like too risky. We'll see.

Yeah, I agree. I think a couple of highlights that I thought were really fascinating as strains of all the conversation topics you just mentioned were, one, the tooling that's now available for people who are building databases, right? Like you said, we've sort of finally reached a place where this can happen pretty rapidly in ways that it actually couldn't before. And Joran talked really specifically about some of those things, you know, technologies that came out of Dropbox and other major technology developers, which was really interesting. So the actual toolkit available

is changing the game. And then the other thing that I loved was talking with Joran just about how a passion for trying to make something fast or performant or safe in and of itself can be a really motivating factor. And so I think we got to see a side of Joran that was pretty unique, especially when you consider people who have a drive to build something like Tiger Beetle. And so it was neat to get sort of a personal anecdote from him about how he just loves solving those problems for speed. He has a need for speed. So definitely tune in. Absolutely fascinating conversation. Of course, subscribe if you haven't,

tell a friend, and we will catch you on the next one.