The Data Stack Show - The PRQL: Data Marts Aren’t Just for the Enterprise

Episode Date: July 8, 2022

Eric and Kostas preview their upcoming conversation with Nick Hansel from Transform.

Welcome to the Data Stack Show prequel. This is where we talk about the show that we just recorded to give you a sneak preview and get you excited about the upcoming episode. Okay, Costas, the last time I remember hearing the word data mart on the show was when we talked with Alex from Intuit. Do you remember this? He migrated Intuit from an on-prem infrastructure to the cloud in, I think it was 16 months or something.

Maybe less, maybe it was 12 months. Just mind-blowing for a company of that size. And he talked a lot about data marts and how he was responsible for data marts. And what's interesting to me is that's a gigantic, Intuit's a gigantic organization. And so you kind of think about data marts as almost being more of an enterprise,

sort of an enterprise or a feature or characteristic of an enterprise organization. But Nick from Transform, who we just talked to, talked about this vision of being able to programmatically generate data marts, you know, really in any size of organization, which was really interesting. So we'd love your thoughts on that, on the data mart in general, and then Nick's vision for that with Metrix Flow, the open source project that they just launched. Yeah, yeah.

First of all, I'm super excited about the open source project to be on the next class. That's cool. Yeah, and as we discussed with Nick, I mean, we're talking about the project that's like very's say the engineering itself from the product opportunity there, just the engineering challenges, I think should make the project like a favorite for many people out there to like learn and try to help and to work with like challenging engineering problems. So having said that, I think what I find like very interesting with let's say this type of products and what Nick is very interesting with, let's say, this type of

products and what Nick is doing is that we are really crossing, let's say, the boundaries between the technology and the human factor, right? Many things have to do with how people can work, communicate, and organize their processes. And in some cases, it might be more important than, let's say, the technology itself. So we're talking about data marts, and data marts is a way to, let's say, organize and create some kind of silos in a good way for each team to go and solve specific problems, right? And not pollute this data with whatever another team might be doing somewhere else, right? But all these are the end of just ways that we try to organize our work and make it scale, right? Is this the right way to do it?

Is this what humans would want to do tomorrow? We cannot discount the importance of the human factor in that stuff. And that's, again, another reason that makes the work of Nick and the rest of the vendors in this space so fascinating to me. Yeah, it was absolutely a fascinating conversation. We talked a lot about what we defined the metrics layer. We defined a metric store.

And then we also got pretty nerdy talking about optimizing SQL queries, but maintaining legibility. So if any of that is interesting to you, you're definitely going to want to check out this episode. Tell a friend about the Data Stack Show if you like it. We always love new subscribers, and we will catch you on the next show with Nick.