The Data Stack Show - The PRQL: Database Tuning and Optimization with Andy Pavlo and Dana Van Aken of OtterTune

Episode Date: April 17, 2023

In this bonus episode, Eric and Kostas preview their upcoming conversation with with Andy Pavlo and Dana Van Aken of OtterTune.

Welcome to the Data Stack Show prequel, where we replay a snippet from the show we just recorded. Costas, are you ready to give people a sneak peek? I am, of course. Let's do it. Let's do it. What a fascinating conversation with Andy Pablo and Dana Van Aken of Ottertoon. Costas, where do I begin? I mean, of course, maybe the best part of the show was hearing about the name Ottertoon and the

influence of the Wu-Tang Clan on their brand. So I think listeners are going to love the show just for that, obviously. But it was also really interesting for me to learn about tuning. And, you know, the complexity of tuning and the skill of tuning, I thought was a really interesting conversation and really informed why something like OtterToon is so powerful because it can take so many more things into consideration than you know a human changing one knob and then you know waiting to see the result on the entire system yeah 100% I think like we okay we learned the tone through the conversation with Andy and Dana. A few things I want to keep from the conversation is hearing from them

that the tuning problem is not just, let's say, an algorithmic problem. The algorithms that you use to do the machine learning or whatever is one thing, but it's equally an observability problem. And maybe it's even harder to actually figure out like how to obtain the data that you need, how to collect this data, like from live database systems, making sure that like they have the right data and like all these parts, which I think is like super interesting. And how much of like, okay, having the technology again, it's like one thing,

building a product is another thing. Or like figuring out like the right, let's say, balance between like the technology itself, what the technology can do and how to involve the human factor in it, right, By providing recommendations, best practices. That was super interesting to hear from both Dana and Adi, that other tools that not just optimize for, let's say, based on the metrics that we collect and the knobs that we have access, but we also combine that domain knowledge and best

practices from running database systems to inform the user on how to go and do the right thing at the end. And I think the example that they gave was, yeah, sure, if you go and like turn off backups, it will be faster. Yeah. But is this what you want to do? Yeah. Yeah. It's like saying you can take the airbags and seatbelts out of your car and it will weigh less. Yeah.

Yeah. Do you want to do that? Yeah. So, yeah, amazing conversation. I would like encourage everyone everyone to listen to it. And hopefully we'll have them again in the future, like to talk more about database systems and what it means to start the company and a music label.

Yes, a company and a music label. So we'll definitely have to have them back on. Thanks for joining us again on the Data Stack Show. Subscribe if you haven't. Tell a friend. Post on Hacker News so we can try to get on the first page. And we'll catch you on the next one.