The Data Stack Show - The PRQL: Breaking Down Silos: Collaborative Data Engineering in the AI Era with Pete Hunt of Dagster

Episode Date: May 26, 2025

The Data Stack Show is a weekly podcast powered by RudderStack, customer data infrastructure that enables you to deliver real-time customer event data everywhere it’s needed to power smarter decisions and better customer experiences. Each week, we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.

Welcome to the Data Stack show prequel. This is a short bonus episode where we preview the upcoming show. You'll get to meet our guests and hear about the topics we're going to cover. If they're interesting to you, you can catch the full-length show when it drops on Wednesday. Okay. So special episode here today, we're here with Pete Hunt from Dagster. Pete is actually the fourth person from Dagster we have ever talked to on the show,

which is I think a show record. I think so. And also if you're like, hey, this is an unfamiliar voice, what's the deal? Eric is on a plane right now, so couldn't make the recording. I'm Brooks, producer of the show. You probably heard me here and there before. But here to kick things off today, and excited to connect with Pete. So Pete, what we always do first in our intros,

will you give us just the quick high-level version of your background? We'll get more in-depth later. But yeah, tell us where you started and what you're doing today. Yeah, it's great to be here. Thanks for having me. I'm Pete. I'm the CEO here at Dagster.

Come from an engineering background. So kind of the first big thing I worked on was React.js at Facebook, which was a large successful open source project. Then I really wanted to get an entrepreneurship. So I left and started a company called Smite, which did, that's really where I got into data, the large scale stream processing to try to find fake and compromised accounts on the internet. I ended up selling that to the company that was known as Twitter back then, stayed there

for a couple of years. And then my old buddy from Facebook, Nick Schrock, recruited me over to Dagster. And before I even knew what was happening, I was CEO. So that was, that was very exciting. It was very exciting. It was very cool. So Pete, just so many things to talk about. One of the things I want to talk about

in regards to data teams, which we talked about before the show, is this idea of data people starting, let's say kind of more from an analyst background, they're not from a development background. And we're seeing people kind of drifting that way. We're seeing data practices drift that way. So I want to dig into that with you. And then what are you excited to talk about? I mean, I love talking about that stuff. I'm, as you can probably guess, I'm very into

like dev tools and frameworks and infrastructure. And in many ways, that's about enabling different personas to participate in in an engineering process. And I think it's just a really exciting time to talk about that kind of thing, both because those practices are evolving, obviously, but also there's a lot of use of large language models to generate code. I think that changes the math a little bit on who can do what on the stack.

So we could talk about maybe how DevTools best practices impact that. Awesome. So good. Yeah, John and I have been talking a good bit about actually the kind of shifting ground underneath us all and excited to get your take on how GEN.AI is kind of changing the landscape here.

So let's dig in. Let's do it. All right. All right, that's a wrap for the prequel. The full-length episode will drop Wednesday morning. Subscribe now so you don't miss it.