The Data Stack Show - 251: Data Teams at the Crossroads: Proving Value in a Changing Business Landscape with Ben Rogojan

Episode Date: July 2, 2025

Highlights from this week’s conversation include:Technical Freelancer Academy & Consulting Community (1:21)Evolution of Data Teams and Technology (2:52)Data Team Growth and Output vs. Outcome (4:47)Internal Optimization vs. Client-Facing Data Work (7:23)Audience, Delivery Mechanisms, and Actionability (12:40)Proving ROI and Prioritizing Work (15:27)Practical Tips for Data Team-Business Alignment (18:31)Dealing with Vanity and Security Blanket Metrics (23:39)AI’s Impact on Data Workflows (27:07)BI Tools, AI Integration, and Dashboards (32:25)Top Skills for Data Professionals (37:27)Career Growth: Technical, Communication, and Business Skills (42:02)Show, Don’t Tell: Prototyping and Feedback (44:37)Taking Initiative and Risk in Data Roles (50:21)Parting Advice and Closing Thoughts (51:16)The Data Stack Show is a weekly podcast powered by RudderStack, customer data infrastructure that enables you to deliver real-time customer event data everywhere it’s needed to power smarter decisions and better customer experiences. Each week, we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.

Hi, I'm Eric Dotz. And I'm John Wessel. Welcome to the Data Stack Show. The Data Stack Show is a podcast where we talk about the technical, business, and human challenges involved in data work. Join our casual conversations with innovators and data professionals to learn about new data technologies and how data teams are run at top companies. How to Create a Data Team with RutterSack Before we dig into today's episode,

we want to give a huge thanks to our presenting sponsor, RutterSack. They give us the equipment and time to do this show week in, week out, and provide you the valuable content. RutterSack provides customer data infrastructure and is used by the world's most innovative companies to collect, transform, and deliver their event data

wherever it's needed, all in real time. You can learn more at ruddersack.com. All right, welcome back to the Data Stack show. We have Ben Rogajan here with us for the third time, I believe. So welcome back, Ben. Hey, yeah, thank you. Thank you so much.

What you been up to? I mean, you know, some of the same, some different, you know, continuing consulting, helping companies set up their data stacks, untangle messes, and also just trying to build some communities for both day leaders and consultants as well. Yeah, so I got it to you up for this. Tell us more about Technical Freelancer Academy.

I think you've started that since we last talked. So I'll give you a minute. Let's talk about that for a minute. And then we got a lot of topics to cover. Oh yeah, for sure. No, I just kept having people ask me about consulting and I was like, all right, let me create a space where people can go.

One, I can put content that's specifically around consulting people can learn from it, but also for people who maybe need some accountability or just want to see what other people are doing, it's a space you can go ask questions. People like to do like weekly updates where they're like, Hey, here's things I did well, here's things I did maybe poorly. Here's some things I'm learning about. And so it adds on accountability.

It has that information. So that's what I'm really trying to do is just create a space that again, if you're deciding to go consulting, you can kind of see what it's like, or if you haven't made the jump yet, you kind of see what people's problems are. There's no glitz and glamour there. No one's saying like, I'm making 10,000 dollars a year. People are like, hey, sometimes this is art. And so that's really the goal of the space. Yeah, it's a little bit different than a lot of the ones that I've seen where people, you know, you've got the screenshots of the Shopify store or the marketing, you know, campaign with the like however many million dollars. It's a little bit different space than that, but I can attest

to, you know, great community. I've actually met some people through the community. So it's a good place to hang out. It's been cool to see different people grow. I think we just saw, what is it, Jeff and Yuki both kind of go back into full-time but because sort of they were consulting. So it's kind of interesting to see that. That switched to it's like, oh, we're consulting and then we get pulled into some sort of full-time role because we were consulting.

And so it's interesting watching that now. Yeah, awesome. So one of the topics I want to dig into and I want to start with like where we've been. And that's like this data business impact, data leadership, let's just back up actually, let's call it data people's relationship to the business. I'd say that, I'd say it's been maybe a little bit

of a rocky couple of years, but why don't we rewind all the way to like 2019, the glory days, you know, 2018, 2019, 2020, Rewind where we started, cause I don't want to assume that all of our listeners, you know, were in the space even then, cause it has been, you know, over five years since then. So where did we start for people that maybe weren't in the space yet?

Yeah. I mean, I'm imagining years, I feel like the business data relationship has been rocky probably. I mean, decades. You know, since the first person to ask the question to an IT team and was like, Hey, can we answer this question? And they're like, ah, that's the first person to ask the question to an item team and was like, Hey, can we answer this question? And they're like, Oh, it's kind of hard. They're like, why we write data? Like, why is this so hard? I think, you know, going back to

like this far back is going not that far back to 2018. We just come off of like the Hadoop high. I always use this example where you boot up Google trends on Hadoop. 2015 was peak Hadoop. It's literally like peak, and then it just falls off from there. Right. And so that means we kind of come off of like Teradata and Netezza and people are like, this is expensive. Let's find cheaper solutions. We went into Hadoop.

We're like, that's hard. And we then got into this like snowflake cloud data warehouse kind of world. And obviously some people are still on post-prem. So a lot of people are still on-prem. I had a lot of projects recently that have been SQL Server to snowflake. So plenty of on-prem still. But I think a lot of the tooling that started coming out in 2018, 2020, removed a lot of the technical friction.

One that kept people from maybe going into data analytics as much. Like big data became less of an issue for a lot of companies because most companies don't have that much data. And so the tooling that kind of existed solved a lot of those problems. And then you solved a lot of other technical restrictions. Right? Like you had tools like 5Train that came out and like, hey, what if you didn't have to write as much code

and just extracted data? And so like all this stuff happened and it became cheaper. Right? Like something else became cheaper to like do this instead of signing a $10 million contract for Teradata. Hey, let's use Snowflake. Let's, you know, maybe spend $10,000 or $20,000. I'm sure now for a lot of people it's more, but you know, especially back then you're just getting started, it's not that expensive.

And so suddenly people are kind of starting to build up this data infrastructure that's trying to spend a little money on it and they're like, okay, we're getting this data warehouse. And I think at some point, like money was cheap. So like teams started building up. I remember, like, you talk to people, like startups that were relatively small. And you have a data team that's like 20 to 50 people. And you're like, what?

Like, what do you do? Like, what are you going to do? Like, even when I was at Facebook and this was something that's more post-hoc, you're like, yeah, our data team for what we did were like 30 people, but if you look at like, do we need 30 people? Right. Maybe not.

And so like teams started to explode, and really I think for a lot of teams, they were building a lot of stuff, and there was a lot of output, but there wasn't a lot of outcome, which didn't like, you know, as per most cases, didn't really become obvious, maybe until money started to dry out, then suddenly, right.

Oh, OK. What do you actually do? Like before starting to check, what do you actually like? You're building these pipelines, you're using DBT, you're building all these models. But do we use these dashboards? Are we getting any outcomes? I think a lot of things started to get questioned. And I don't think that happened before. Again, like you're going through the motions and you're like, OK, we're building the things we're supposed to build, you know,

based on the article that we've read, right? I think Lake House and Italian architecture was like 2020, 2021. Everyone's like, okay, let's just look, we're building that. And we got so caught up in the motions. And I think a lot of companies forgot there's a reason we're doing all this. But I think that's kind of where like, if you want to see like the trajectory of all of this, like starting somewhere in 2018 and coming till now, is why are we now here? And we're like, okay, what do we actually need? Why are we date meal?

I think you're seeing date teams start to shrink down as other things are going kind of up in other ways in terms of cost for tooling. Yeah, I think one of the things that's easy for me to forget and I actually think about this a lot when I'm talking to clients, is when you have all of your data efforts essentially focused for internal improvement,

that is pretty expendable, right? Because like, think about like lean manufacturing and like Six Sigma, right? That's like a really big popular, like, oh, we're gonna have all these like lean initiatives and Six Sigma initiatives. And some companies still do that but because it's like process optimization focused that's an easy thing when like maybe money's a

little tighter like eh like we're efficient enough we're just going to cut that for now and just roll with what we have and I'm afraid that some data teams like can kind of get caught up into that especially if they're really just focused on like internal metrics and like kind of driving driving cost up for example. I'm thinking like maybe manufacturing transportation like industries like that. Whereas, ironically enough, I IT teams or engineers that are working on apps. If customers use the app like there's going to be somebody that's they might reduce the a team size, but like there's a lot more like business perception of need if like, oh, our customers use this. Like we can't like get rid of that team. So that's the part where, and now there, and there's a lot of businesses that like have a lot of,

that I've actually worked with that have a lot of like client facing data stuff. And those I've seen have not had as drastic of a swing because they're like, well, this is literally like our clients trust us to help them optimize their business. Like we need these data people so that we can make sure that we're delivering value for our customers. That's that for me, at least my perception has been more stable,

but I feel like the internal optimization stuff is the part that's a little bit been more volatile. Yeah. Yeah. I think that's a good point. And it goes like, before we started this, I was talking about an article that I wrote recently talking about why data teams don't have a seat at the table. And one thing I think I really wanted to kind of hammer home there was the fact that, you know, if a business can afford a data team, they're already doing well, right? Like, if you're a data team of five people all costing $200,000,

and again, maybe you're only getting paid 150,000, but there's other costs that are associated, whether it's healthcare, et cetera. So you just rounded probably up to 200. So it's a million dollars there. You're spending another $300,000 on tools, whatever it is, you know, it ends up being 1.3, 1.5.

That's money that the business is spending somewhere and they're getting it from somewhere, right? Either they're funded well or they're successful business and are spinning off enough cap to invest in you. It's like that means they could just take that money as profit or they could invest it somewhere else. It's that they're investing in you. So the business is successful. It's kind of the bottom line. You can be an amplifier in the way I view it as a data team, at least if you're doing that internal metrics approach, but you're not really adding to the business

in that world, right? Like the business is already valued at one, you can maybe one point something it, if you add some efficiencies or something on that side. Yeah. Yeah, I mean, one of the things I've always thought about in that context, that is the, there's a finite amount of money,

and let's say they can invest it four different ways, or three different ways. in that context that is the, there's a finite amount of money. And let's say they can invest it four different ways or three different ways. One in the data team, two in scaling paid media, three in hiring more salespeople, four in some other thing. Like not that they're mutually exclusive, but if the answer is like,

we're gonna pick a few of those things, paid media, there's a perception. I'm not going to say reality. There's a perception of clear ROI, right? Like if you use, especially if you're using like Google or Facebook, there's a perception of like, we have this clear like measurement of ROI. We actually have a three episode series on attribution like back in the data stack show in our archives.

If you want to look at some old episodes, we went really deep on this. It's not always very accurate It's the TLDR on that but there's a perception there Salespeople if you have tight metrics you can clear out hire a new one Like here's what they should be able to hit, you know That can be pretty tight and then but a data team like it the ROI is hard Like it always has been and I think even if the value probably actually is there and maybe even is in reality,

pretty much equivalent to one of those other options, it's harder to prove. And if you don't have a data leader that's good at essentially selling what they're doing with the value is, then you, and there's limited budget, then it's probably gonna go to sales or paid media or another like cleaner, clearer decision. That's almost why I say comes out of hotel people like we want to work for a data team

where you're likely not going to get fired or if you want to like, in this environment so you see like consultants aim at these spaces marketing and sales are great places. 100%. Under if you're a team or if you're a consultant target, it's like there will be money there. There might not be money on the data side because again, it's hard to prove that ROI, but like salespeople want to know what they're, how they're tracking, you know, better, how to be better, go to market motions, all that stuff. Like there's always value there because if you can tie it to, hey, we were able to land contracts,

people are going to do more of those activities. And then going back to the efficiency argument, right? Like if what your data team is doing, like, oh, let's try to target sales better. Well, if it costs you for marketing better, if it costs you $300,000 to get that improved marketing, you know, return on investment, but you could just spend $300,000 in marketing and be a little bit accurate, then the question becomes, okay, how much are you actually worth if you're not actually improving it by that much more or just doing the same? right Another thing that I've seen is an is an audience problem, and I don't mean audience in the marketing sense

I mean like internal teams I think you actually touched on this maybe in your article but my view on this is like who are you making this for ultimately? And the easy answer is typically like some kind of like internal stakeholder that like you maybe interact with. The harder answer is like, who do they show that to? And then who do they show that to? And then who do they show that to?

And then how does it impact a customer? I think people miss that a lot of the time. And then the other one is the delivery mechanism. We like to pick on dashboards, right? Like could be a dashboard, could be a really valuable dashboard, I don't know. Could be a report or the delivery mechanisms that I find that can be easier to track ROI on

is like a reverse ETL to create an audience and a paid advertising tool or some kind of like audience you create for the email team like those are actually because I mean then to be honest you're not as dependent on a human to do something with it so like I always if I'm talking to data teams like as close as they can deliver the data like into the value layer and make it like as automated as possible so like if you're making the audiences better for marketing and you can just feed it in there and then their campaigns are better and you can go grab that number and say hey we contributed

to this that's amazing because but when you have like a human that's supposed to act on optimizing something based off of a report or dashboard that's tough yeah you have to have you really have to have the right humans, which is just not always the, you know. Yeah. That's a good way of viewing it. Just it makes me think of when I worked on one of the early companies I worked at, we built fraud detection models. And that's really what we did.

We just created lists of here are the providers in terms of healthcare. Here are the providers that we believe are providing fraudulent care. Here's how much we think it's costing you. Like here's the list of claims. Like, yeah, yeah, yes. Right.

Right. Versus like, here's a dashboard showing you like the cost of fraud is like, right. Well, what am I going to do with this number? Right. It's great for tracking the effort of that list or the effort of the marketing, like the chart and retargeting of audiences. Yeah.

You need a dashboard. You're like, are we getting better? If not, do we need to improve the segment somehow? That part makes sense. But yeah, like it is that like connecting to the business and that's then is where like, that's where the great for some of those tracking, but not always taking action off of much harder.

Right. Yeah. So I want to talk more, I'll throw out a hypothetical scenario. So say I work in a company, my'll throw out a hypothetical scenario. So say I work in a company, my team was downsized. We had like 10, now we've got like five in the last few years. They didn't take any work away. It's still a lot of work and this ability to sell, but say you're trying to guide me on how to sell better.

What are some things that you would tell people that are like, hey, things are not going great and I at least perceive that there's kind of a disconnect between business perceived value and what we're doing? How do you bridge that gap? Yeah, I think one area is to figure out, I know you in theory have the same amount of work that you had before.

I do think there's something to be said about like going through and figuring out, okay, is this work driving anything? Figuring out what work does and just being brutal and be like, we don't do this anymore, right? It does like whoever the stakeholder is that's gonna be upset, you know, figure out who the stakeholders are that matter

and which ones are not, right? Kind of going back to like in that article I referenced, like the stakeholder matrix, figure out who has influence, who drives impact in your business, figure out who you want to be aligned with, pick those, you know, one or two that you really think you can drive something with and start getting close to them and having conversations with them and see like what can we deliver for you that actually drives value. And then you start having that conversation about like, Hey, what are the two projects we can do? And then making sure you're actually one working with them to make it successful.

And then once you have, let's say successfully delivered these projects that drag some sort of impact for them, making sure you don't stop selling it. I think a lot of people deliver dashboard and then that's it. I'm done. No, they don't talk about it again. This is the same thing if you're selling a product or trying to sell a service. Once you've, let's say, written an article, made a product, you don't have to talk about it all the time. It's the things like, yeah, okay, are people engaging with this dashboard?

Are people engaging with these segments? Are we improving whatever it is that you've built? If not, is it because it's the wrong thing and do you want this product, so to speak, internally, or do we need to help people engage with it more? How do we get like engagement up so that people see that value? Because if you build something, if you go through all this effort and you went through the whole process of understanding what the business needed, built what they needed and now it's not getting engaged with, it really like, it doesn't matter how well that product is built or how well in theory it could help the

business. If the business isn't using it, not engaging with it, you know, you have to figure out, okay, what can we do? Again, starting with like talking about it more, if you need to have the more like one on one meetings with leadership to be like, hey, we built this, here's what we think we should be doing with it. Like, here's how we think people should be using it. We already maybe haven't even come up with some of our own strategies that we think the business should implement. I really think it's

about being willing to be proactive and go beyond just like, hey, here's the dashboard, now we're done. I think a lot of people just kind of drop the dashboard off and think like this is the end of me as the data person, right? Like I've delivered the thing, but there's this whole other side where it's like, you can be proactive, you can like, give recommendations that like go beyond just like here's the data, you can give business thoughts and to your leadership. Yeah, I totally agree. I've got two or two or three, like, I'm gonna call them hacks. But like two or three things that that I

have seen are just practical for me. And this isn't, sometimes you just can't do this. Like if the whole team's remote, like this first one I'm gonna say, it doesn't work. But one of the things for me, I supported a sales group for a couple of years actually doing analysis to help land clients essentially. So number one, I was positive it was high value. Because if it went well, so the analysis goes into a presentation, land clients, essentially.

So, number one, I was positive it was high value. Because if it went well, so the analysis goes into a presentation, goes to a pitch, and then you land a client and these average deal sizes were in the millions of dollars. So, great, value was absolutely there. But, a lot of ambiguity. Because you get the data and it wouldn't be exactly what we'd wanted and I don't know what, especially when you first start, I don't know what I didn't get, you know? So there's a high level of ambiguity. So one of the things I ended up doing,

and it wasn't even intentional, I don't think, but I would actually hang out a lot, because the space was kind of segmented, like this physical office space segmented by teams. So I just hung out a lot. They had this table in the sales area and I hung out a lot there and actually did some of the work like tried to get

like quick feedback loops with people on the sales team. I asked to be on the sales calls to see them present. I asked for the presentations that my data went into. And then the hardest part was internally dealing with cause people will essentially try to protect your feelings. So the hardest part was being on the sales calls, looking at the decks and seeing the 80% of my work that didn't make it into the deck and never saying anything about it, just dealing with it and understanding like, okay this is part of the process, not complain, not ever express it, because I never would want to

be cut out of the conversation just so they like don't let me know that I wasted 80% of my time. So that's the one that I think is challenging because a lot of times, especially salespeople, will intentionally cut. You won't actually get to see the final product and it's not typically because there's any reason you shouldn't see it, but sometimes it's because they use so little of what you did. it's because they use so little of what you did. They just want to like kind of like brush out under the rug So I would say and I know this is like specific but I think but it's happened to me in a couple Circumstances is like really fight to see the end result if your data is an input into something else Essentially. Yeah, I like that and I think it kind of leads to something else that I often think about This is more going towards like the IC level, especially when it comes to delivering an analysis

or something. One of the places I just see repeatedly is that people will often be like, hey, here's everything I did to get you these numbers. They'll literally put that all in a report. When honestly, like you just said, most people just need two numbers, right? Like, if you, every time I've talked with someone

in the C-suite or VP, they're always just like, they're constantly looking for a couple of things. They're looking for a narrative. So like you're in a project, they're like, where are we? What can we do now that we did yesterday? That's what I need to know. What's like the number I'm giving to the CEO say, hey, we're up or down?

They don't want to see like- Up or down. Yeah, literally. Yeah, maybe one, but not always. Not always, but are you fixing it? Fool. That's all we do, maybe one. But not always. Not always. Like, are you fixing it? Cool. Right, yeah. That's all we do, fixing it. It's bad. So, you know, but we want to show like all this work we did.

Right. Because you want like credit for it, you know? Yeah, exactly. Exactly. They just wanted two numbers. I know you did all this other work. I know you want to talk about it. I know you can get cool that you write, you know, DuckDV or numbers. I know you did all this other work. I know you want to talk about it. I know you can get cool that you, you know, ducked in your words or whatever.

Like you are the only person or you're a data-themed blog person that sadly cares about that. So yeah, I think that's great. Like bit of advice to you. It's like, yeah, look at the product. If they don't use it all, it's fine. They use something and it,

your process was required to get there. I think that's the big thing. It's like, because they only use one number. It doesn't mean you didn't require the whole process to get to that number. Right. I would just figure out that's the number that's important. Well, but yes, I think that's totally true. But there's also a feedback loop here of like, learn what the output was and optimize for that output the next time. And I do think there's a number of data teams that like do unintentionally waste time because they don't really know what the final output like should be as far as crucial, like crucial like like you said, upper like two numbers is a higher or lower than it was last quarter or whatever.

And I think there is like especially data teams that are feeling overwhelmed. Like I think there is a like for some of that are feeling overwhelmed. Like I think there is a, like for some of them, like we really just need to cut and get to like, all right, they just want these two numbers. What's kind of the minimum like behind the scenes work that we have to get to the two numbers and have some like confidence that the two numbers are directionally correct. Yeah. I'm saying that number. Yeah.

I do. Especially when it comes to final data. It's like, look, we can be direct. We can't be 100% accurate at counting every day because of a myriad of reasons. No, I think that's good. Going back to that initial question you asked, if you're a team that's been cut back. I think it's worth looking at the efforts you put in and being And you're like, is half of this stuff even doing anything? Because I wouldn't be surprised if you found out

that a lot of the work you're doing is just performative. Like, yeah, we're giving these numbers, nothing happens. And like, no one really makes any decisions off them anyways. But they like looking at them. You know, it's a vanity metric, whatever. But, you know. And the toughest part of this is what I, which vanity metrics or what I call security blanket metrics, where like there are people that like they need one or two numbers,

but they want 20 like just in case like somebody asked about it or this or that. And those are the toughest ones where the toughest stakeholders that just really want a ton of information They only actually need like a little bit of it But they're gonna continue to ask for all of it just in case and those sometimes there's nothing you can do But sometimes like really developing a relationship and having a level of trust of like hey I can get you that number quickly if you need it. But us producing that weekly port, whatever thing with like 20 of these metrics, actually like, here's what all goes into it.

And if it's automated, sure, like maybe not that big a deal, but some people are in contexts where like, that's not the current state, for example, and those will really eat it. If you have a small team and enough things that are non-automated then there is a little bit of a hopelessness hey we're stuck forever type of thing and you need to buy some time to automate some things to get to dig out for sure. Yeah it's kind of like when Starbucks the new CEO just came in the first thing he did was like

let's simplify the menu. there's a certain point where like, you know, and I think this happens with like every business, with every you start doing too many things because you're like, OK, we got to add more. We got to do more things. And you start realizing, hey, that's actually impacting our abilities to do the things that we were doing originally. Well, yeah, let's, you know, turn back a little. I think that's part of the process.

I mean, what should we keep? What should we not? Yeah, I think it's I mean, I do that with my life. I'll be like, oh, I'll get to a point where I'm like, okay, I'm doing these certain things and I'll start expanding. I'm like, oh, yeah, I'm doing too many things. What do I actually wanna be doing with my time now? You trim off some things.

So that's just part of the process of growing the business of the person. So yeah. We're gonna take a quick break from the episode to talk about our sponsor, RudderSec. Now I could say a bunch of nice things as if I found a fancy new tool, We're going to take a quick break from the episode to talk about our sponsor, RutterStack. Now, I could say a bunch of nice things as if I found a fancy new tool, but John has been implementing RutterStack for over half a decade. John, you work with customer event data every day and you know how hard it can be to make sure that data is clean and then to stream it everywhere it needs to go.

Yeah, Eric. As you know, customer data can get messy. it everywhere it needs to go. that you have implemented the longest-running production instance of RutterStack at six years and going? Yes, I can confirm that. And one of the reasons we picked RutterStack was that it does not store the data and we can live stream data to our downstream tools. One of the things about the implementation that has been so common over all the years and with so many RutterStack customers is that it wasn't a wholesale replacement of your stack. It fit right into your existing tool set. Yeah, and even with technical tools, Eric, things like Kafka or PubSub,

but you don't have to have all that complicated customer data infrastructure. Well, if you need to stream clean customer data to your entire stack, including your data infrastructure tools, head over to rudderstack.com to learn more. Yep okay so the same hypothetical data team talked about some strategies around sales. What's we have this fun game we do on the show like how many minutes in do we get without talking

about AI? We're doing good we're doing good but okay so here's the other thing all right we you know we caught the team, but like, and then, you know, your boss's boss is like, but AI is a thing. Like you guys are fine. Like just use AI. So maybe talk to that a little bit. Just any hot takes you have on that.

I'm trying to make my V specifically hot takes. So what I, I can't like, there are definitely benefits. I think anyone who's gone to the, start to use AI, definitely probably found some benefits. Like if you're migrating from one tool to another, like that is so much easier. Yeah. And so much just cleaner.

Like you'd always make weird mistakes. Right. In the past, AI still makes weird mistakes, but less of them. Like I remember I was like migrating some work from SQL servers or stuff. Like, and there was some things where it was like, oh, I need to make sure I always use,

I like and not just like. Right, yep, yep. Some uppercase one. So you just tell your, whatever you're using at the time to make sure you make those changes and then it's pretty easy and suddenly you're moving really quickly.

Especially when it comes to like code already existing and then asking me for it to change, I found that, hey, I was really good at that, making those changes. Yep. And so I think that'll kind of continue. I think that will then start to even to maybe some level of vendor lock-in.

It's like, if I can switch code from SQL Server Snowflake or from whatever it might be really quickly, the projects that used to maybe take a year is now three months, right? As long as you have everything well set up well. So I think there's that aspect. I think there's other aspects that are less ideal, right? Like can we stick AI on top

of your data warehouse? I haven't seen a great implementation of that. I've seen like decent basic implementation, better than they were definitely like five years ago with the whatever was back then was in terms of like English or natural language. Yeah, right, right. It's better than that but still heavily limited. So you can maybe do things a little faster, but that's a little clunky. I think another area I've seen that I like, is I've seen some tools that help you, maybe get you some of those initial metrics.

Like you can ask it like, hey, what are our sales currently? And then it can answer some of those basic questions. And then instead of trying to answer the harder questions, it will like send you, it'll look through your dashboards and try to set up a dashboard that maybe could answer these questions better.

Which then it features that engagement problem a little bit as well. It's like, yeah, can we get people to engage with these things a little more? How do we get them? Well, generally the process, for anyone who's worked in a data team knows, it's like, well, even if a dashboard exists,

the first place someone's gonna go is probably you and ask like, can you give me this number before even looking at the dashboard that you made for them a week ago. 100% So I do think there's something there where it's like, yeah, we can get that problem solved. It's like, look, there's a dashboard that exists,

let's get the number and then let's get you there versus like, you know, interacting with the data team who's like currently doing that at the moment. Well, I mean, that's one of the funniest things to me with the AI conversation is at least from my perspective in the data world, we've kind of skipped over the, hey, why don't we index and make the thing searchable? Catalogs are obviously a thing and they're out there and there's some good ones.

But as far as like first class, like, oh, there's some catalog that everybody uses that doesn't really exist. And there's a practical of A, and here's the funny part, like the business is probably never going to know what's out there. If once you hit the 50s, hundreds of reports, they're not going to know most BI tools I've used, like search is not great. And so, so there's that, but then honestly, I can't tell you how many times I've seen where like the analysts don't remember either and they just make another one. Like it's kind of crazy.

And to think like that maybe one of the like low-hanging fruit solves here is like AI is just kind of kind of index and search what you have and that's actually pretty valuable. I've even seen some of the approaches where there's like, obviously there's semantic layer approach, but there's some of these AI layers that they have essentially verified queries, where it really is like, talk to it and then ask it a question and it essentially will try to pull

from a library of queries, which that's pretty, in terms of the scope of text to SQL on one end, maybe terms of like the scope of like text to SQL on like one end, like maybe semantic layer in the middle. And then like, there is this other end of like, I don't know, just give me the queries and I'll pick between the queries that you have.

So I think, I think there's gonna be an AI, you know, as it improves, maybe like, there'll be like a broader spectrum of like what it can do. But I do think it can do pretty well if they'll just give it a library of queries and like tell it, have it find the right one. So. Yeah, I think one thing that remains true is searches hard.

Yeah. I think that's funny because for anyone who's tried to search anything on Reddit, on LinkedIn. Right. Right. How long before you go to Google and you're just like, LinkedIn. Yeah.

Yeah. Yes. Yes. Yes. It's like, right. It's like, I cannot find anything in Twitter search. Just have to Google it. So yeah, I think that's, that's an interesting idea there. Like we already have the queries.

We know what the queries have been run. Why not just try to base off that versus trying to custom query from scratch. And I know that's something that I saw with, I think Databricks is AI BI tool. I remember like you can kind of give it some pre-formatted. Yes, right. Like base layer. Right, yeah, I think Snowflake and Databricks both do that. And then a lot of the tools, I mean,

there are some good third party tools, you know, that have AI chat on the semantic layer. And I've seen a lot of progress in that world but I think for all those tools though they have this challenge of like we want people to work in a new way and we want to work this way but everybody also wants all the old functionality and you know I mean let's talk like Tableau like if you want the full functionality of like say a Tableau and you want this full new experience to have this wonderful like AI first thing,

like that's just, this is not practical yet, you know? Like it's either like Tableau is gonna be working backwards into like adding AI onto their stuff and they are. And then these new tools are gonna be working the other direction, like maybe starting AI native but they're trying to include every little thing that people want in a BI tool. And I think part of it depends on a bit.

Like if you're a startup, then like, yeah, like go find an AI like native tool and like you guys, and you're maybe pretty flexible on what it can do, great. And then, you know, established business is probably more likely to pick, you know, one of these like Power BI or Tableau or like one of the bigger names

and then like just go at their speed as they implement AI things, they'll try them. But it's just so hard to be in a spot and there are people trying to do it and I think I've seen some success but it's hard to be in that spot in the middle where you can like attract enterprise requirements where they want all this like rich feature stuff

around a BI tool and have like a really good native like AI experience. Yeah, no, I think that's it. I like, I have this article I haven't put out yet, but it's trying to think on the whole, why did we pick Dashboards in the first place? Because obviously, folks at Dashboards makes fun of it, but we got there for a reason, right?

There was a reason companies like this is what we want. And so I often think about that with AI. AI approach where it's like, okay, you know, I've seen ones where like you can ask the questions, it'll build charts to break, right? Like, oh, nice. Like your trends over time for this certain metric. But, you know, I still think there's some gaps where maybe it's like, maybe it's because people want filters. Maybe it's because people want all of these things. And then even there, I think about that. I'm like, sometimes you give people all the things they ask for, like filters. They still only want like one view of it.

And like, I remember I was talking to someone, they're like, yeah, one thing I learned when I deliver reports to leadership is just put it in the PowerPoint, because like one time they're like, yeah, I was trying to like show it to leadership and something didn't work. And someone's like, you know what?

I'm just gonna put this down, like they only care about this one view anyways. I'm gonna take a picture and put it in reverse. I'm so glad you brought this up because the one thing that I've never seen that I think would be a killer feature in a BI tool is download to PowerPoint. Oh yeah.

Like- I think they're not trying to do that so I'm not looking at the AI or something. There is a thing, yeah, there is a thing and maybe Power BI can do it. Or I mean, PowerPoint, generically, Google Slides PowerPoint. But PDF is a thing in a lot of them, but it is interesting that it hasn't been more of a

first party support where everybody decides to support some kind of slide format, because that is a practical, I mean, I say this, but Power BI, you can kind of do, and Tableau as well, you can kind of do the presentation mode. So I guess maybe, I say this, but like, you know, Power BI, you can like kind of do present, like in Tableau as well, you can kind of do the presentation mode. So I guess maybe I don't, yeah, this, I don't think so, but maybe, but so yeah, maybe the,

you know, and of course there's the, like, you want people engaged in using your tool and you don't actually want to encourage egress out of your tool because you wanted to use your tool. But all that to say, yeah, I think that is the practical destination of a lot of this data is some sort of presentation. No, and I imagine again there are some more operational workflows where it's

like, yeah, we want this dashboard. But it's something that I constantly ask myself, why do we end up on dashboards? I wish I knew where the first one started because it's just weird. We make fun of it because it's an easy output but there was a reason we got here and so I think AI fully answered that like you said like I think there's this functionality that dashboard survive that people want whether they use it or not this is a different question but that could be more again more of an issue where we talk about like are we engaging people correctly are we making sure they can find things? I honestly like at almost every company

I worked at, or at least two companies I've worked at, one was product self-heared services, one was Facebook. People were trying to build a website that could just make it easier to find all the dashboards. Yeah, put like a three company like whatever, like Tableau, whatever dashboards here. And we're trying to keep easy to search. And that is the goal of this website. I mean, typically Tableau has that too. I don't know if you're gonna do better than Tableau. Like it's funny to see everyone try to build something like-

So they're trying to build a dashboard to find their dashboards. All right, all right. But catalog of data. Right, right, sure. There's catalogs that do that. Yeah. And a lot of the modern, I think,

data catalogs try to allow you to not just do data pipeline and tables, but data. Right. Right. Yeah, that makes sense. Okay. So, we talked about AI. What, as far as like, we talked about this before the show, just, you know, you're working

with your community you formed and you talked to a lot of different people at different levels in their career, you know, in the data space. What do you think some of the top skills are that like, if I'm sitting here, I see, or maybe I'm like a team leader or something, like what are some of the top skills that people need to develop, you know, to get to that next level career wise? And really, whether it's, you know, they want to progress to be kind of an architect type role maybe they want to progress to be some kind of leader, they want to progress to be kind of an architect type role, maybe

they want to progress to be some kind of leader. They want to like do something on their own. What do you think, you know, or what are on your list of like top couple skills people need? Yeah, I'm trying to think of the specific skills there. I mean, there's the generic statement like on like, we gotta be better at communication. I mean, trying to work to make that clear on what I'm trying to improve the communication around that. Like I love it.

Often see better communication, which to me is like an engineering state. It is. Yeah. And so like I wrote, like in a recent article, I talked about like a few ways that you can do that, whether it's like if you are a data leader and you're trying to get buy-in for things, you know, talk about what the cost of inaction is, like, and figure out how to portray that in a way. So, hey, if we don't do the certain project, here's the cost to the businessaction is, like, and figure out how to portray that in a way. Say, hey, if we don't do this certain project, here's the cost to the business.

Talking about like impact and framing it in such a way that the business understands like, hey, what is the impact of us doing that? Not in the sense of like, hey, if we don't do this, like, you know, our dashboard will be slow. That's like one side of things. And it's very, it's starting to get in the right direction. It's more on the technical side,

but there are other things where it's like, hey, we're, you know, maybe last quarter we realized that, you know, sales teams are, you know, not getting their numbers fast enough. We talked about like numbers for sales team. And because of that, maybe we lost a few deals or maybe things like that. And having those conversations and trying to think about like, what does the business actually care about? Which then also ties to like the building up your ability to understand your business, which that is, you know, domain specific, right?

You're going to do supply chain, be really good at that and understand what people in the supply chain care about. I was talking to someone the other week who's kind of they've done supply chain their whole life. They did it like Home Depot, then Amazon and now Coupang. And I thought it was interesting. Like they are clearly they're not just technical, they clearly understand that space, right?

After going through it into so many different industries, like different versions of that. And there was another conversation I had with someone here in Denver, kind of what they were in, what I say it was like telecommunications. And then like, they're not just clearly, again, they're not just technical.

They understand the products, they understand like who the competitor and what their products are. And I think that helps you be better when it comes to, oh, what should we build in terms of maybe data products that help the business? Well, if you don't understand the threats currently that are opposing the business

or what your current business is really positioned as, you might not be as good to give that advice. Because you're just going to be generic dashboards that don't actually push me to certain direction. So I think that aspect, you know, if you're whatever your business is that you're in, understand it, if you want to be in it long-term, you gotta like it a little bit.

It's funny how much some people like the business that they're in. And so, yeah, you gotta like it a little bit. I think also just if you are going down, like let's say the routes, I mean, I guess this is kind of generically, whatever route you end up going, you kind of have to learn to be a little more proactive and kind of make things happen. And just to say, if you see an opportunity, don't wait and wait for permission.

I think that's kind of the skill. Yeah, that's a good one. People, maybe some people you have initially and they kind of get beat out of you a little bit. Nothing really matters if I do it like no one uses it, but there's a certain point where you have to come back and be willing to be like, Hey, I'm going to, I think we should be doing something here or I think I should build a product or I think, you know, I could go out and do consulting and things of that nature.

You kind of have to make it happen. And you have to be willing to reach out to people who will tell you you're wrong or you're dumb, you have to reach out to people who will ignore you. And then what happened, whether you're in or out of the business, right have to reach out to people who will ignore you and then what happened whether you're in or out of the business, right? You're going to give ideas to the business and they'll be like, yeah, man. I was talking to someone who was, they were having some challenges with some analysts who were like senior level and they wanted to be staff.

And one of the blockers that they kept running into was like, well, no one's buying into your ideas. And I was like, that's not fair because then I can't do those ideas and they can't prove that those ideas are good. It's like, well, but that's part of the process. If people think those ideas are good, what can you do? If you genuinely think those are good ideas, what is wrong with how you're presenting it to make that good? I recently had a chart that I put in talking about the senior plateau, which is where a lot of people I think get stuck. It was a good chart. I'm trying to

remember the newsletter I pulled it from, but listed in the article. And it just kind of has like different curves of growth. So like when you're early in your career, it's a lot of technical growth, right? A little bit later, kind of senior, it's like more toward communication skills and soft skills. And then finally, it's like the business skills. And it's like, you know, you kind of it's all little S curves, they're not built on top of each other, you know, side by side. And it's like, you know, you kind of, it's all little S curves, they're not built on top of each other, you

know, side by side. And you just eventually plateau on all those skills, really getting more technical, probably won't make you grow more at a certain point. At a certain point, getting more soft skills probably won't let you grow that much more. And really for a lot of people, last step is like, okay, let me add business skills and apply all this. And again, that means you're gonna do things where, and this is, you know, if you were to own a business,

you're going to put out ideas, people are going to think the same word, and connect and figure out, okay, you have to change our product, you have to change my messaging. What is it? And again, it's the same internal as it is external in many ways, if you would think it's you,

you should go a certain direction. So that's something that I think I've been trying to also just communicate more with people. It's like, you know, kind of to build up those those skills and you don't have to rush at maybe the other thing is like those skills will happen over time. Or do them all at once. Exactly, don't try to do them all at once. Like get really good technically.

And then once you've gotten really good technically then you can work on X and the next. And you'll keep picking up new technical skills along the way because you'll go for a new business and work on a new technology. Right. All that stuff happens. Yeah.

Yeah. And yeah. And what you just said, I think is really true. Typically, and not always the case, but typically the technical skills will get, there'll be some forcing functions in your role to pick them up. And the ones that there might never be a forcing function is typically business skills, right?

So I think if I had to pick for people often as the business skills that you probably have to be more proactive and seek out, and a lot of times the technical skills just happen because it's like what you do every day. Not always. There's some companies that you may be and where like you're just really limited scope and you really do need to like seek out more technical skills. So like there are people out there that are in that boat too. But two of the things that I really like that I've heard, one I've heard a lot over the last couple years is show, don't tell. You can get really stuck in these loops of like presentations and telling people about your ideas.

And I like, I still do this. I think I do it less than I used to, but I still grossly overestimate people's ability to abstract. Which what I mean by that practically is like, if you don't tactically show them something in like a high fidelity detail, they don't get it. Even if you're crystal clear with your communication, even if you try to use examples,

like it's really challenging for people to make that connection. And for me communication, like versus even doing this with clients, like especially, and the cool part is the AI part is like I will versus like trying to explain or like show you could use it in this way. If I'm saying words like, well, you could do that.

Like, no, like go make up some data with Faker, load it into a tool in your environment, and like actually tactically like show them something. That for me has just been a really like big like, okay, this is the way to do it. And it's actually way easier than it used to be. So that's a big one. And then the permission one you hit on,

like especially like early on in my career I had a phenomenal boss that Essentially like told me like early on is like don't go ask for permission. Just do it. I'll cover you like it'll be fine Which was just a really good you don't always have that as a boss, right? So you do have to be sensitive to your current situation if you have a very like cautious boss Like be careful with that. But if you've got a good boss, then that's huge. Because if you're constantly asking for permission

for things, there's just a lot of stuff that you're going to get answers or get nothing. You're never going to get clarity. And essentially, as long as you have a decent internal calibration of risk that matches the calibration of risk for your company Which can be wildly different. I've worked for people that were entrepreneurs were like I Could do anything like one of my jobs like working for an entrepreneur I got a credit card without a limit on day one

he was like spend it like it's your money and like go. Like any optimization from there was like, just move fast. Like I don't, within reason, I don't care what it costs. I don't care what it, you know, just go to like the opposite of working for multi-billion dollar company where there's like 12 layers of approval. So you have to know what context you're in, but even inside your context, like I think most people are apt to be waiting too long

for permission or for like, can I do this or whatever. Whereas if they just like tried to like inside their context, be creative with a prototype, use some open source tool that's free and then whatever to try to like have something to actually show like is a better route for people. I really like the show don't tell I think that's it's one of the things that also is easy to say but it's so easy to fall back into it is yeah yeah you will find yourself

telling more often than you have to be disciplined I was talking to another day with you recently it's like you know because I've been running some cohorts to help road day leaders and then kind of challenge them. And, you know, some of the advice I give, they're like, I know, it's all stuff. Like, in some cases, it's new. In some cases, like, you know, we know this, but it's like, when it comes to having the discipline to do it, it's just it's hard, right? Like, we know the right thing, right? Yeah, like with show, don't tell, it's like, okay. Am I

like, you almost have to ask yourself whenever you're talking, am I showing or am I telling right now? So I think, for example, I think about movies. It's like, if you write movie or script all the time, you probably know to show don't tell. And why do we still end up with so many movies that tell us versus get us in the game? It's just easier.

She could tell you, here's what I did or whatever, we're just trying to move you with like an actual succinct story, like you said, like take it out of this abstract and put it into something tangible. Well, and here's the other thing too. I don't know if this is an intentional decision for people, but showing is often more vulnerable

because you make more decisions that are more concrete and therefore are more likely to get a reaction of like, oh, that's not what I wanted at all. You know what I mean? Cause I've had that. I've done one recently I can think of, like show like very tangibly showing somebody like all the way through with like, like I said,

dummy, like all the thing. And I kind of got the reaction, that's not what I wanted at all. And it's like, oh man, like where if I just talked about it and like, hey, do you want, and then I gathered requirements, and like, kind of go on that route, it would have been like, I would have been more likely to be right the first time.

But I've also learned that's way slower. And people are horrible at giving, most people are horrible at giving requirements. But if you can show them something, and you can get a reaction of like, that's not what I want at all, that's golden. That's great. It's yeah, it's great. Yeah. Like I said, like people, people will tell you what they don't like if you give them something that's so wrong. Yeah. It's like the whole stuff overflow joke where it's like,

how do you get your question answered off stack overflow? Exactly. Yeah. Yeah. You make this counts one that asks the question one that gives an obviously wrong answer. Yeah, that's over there for right. It's kind of the same right there, right? It's like, yeah, when you give someone something so wrong, that they're, no, it's not right at all. Suddenly the brain's like, I know exactly what I'm doing. What do you have to say, what do you want?

They're like, it's like someone where you want to go to eat. Exactly, but if you mentioned something, you're like, no, not there, I don't want that. Yeah, I mean, I, yeah, I think that's, and the first time a couple of times it's kind of uncomfortable. It's like, Oh man, like I didn't get that right at all. But like when you get used, when you get used to the process, like it's so valuable

because you, you can even, I even mentally think to myself of like, cool, I'm going to give them something to react against or like just like even internally think through, like when you give them something to react to or to react against and even in my mind, like assume that it's wrong, but it give them something to react to or to react against, and even in my mind, like assume that it's wrong, but it really doesn't matter if it's right or wrong at this stage, we just want to move forward

and this is the best way. And again, go back to like, it's not necessarily a business skill, a big part of how the business are running the business is like, you gotta be willing to be wrong. Right. You're gonna be wrong. Right. Make mistakes. And it's okay, you just gotta improve.

Right. Yeah. And it's okay, you just gotta improve. Right, yeah. And even like having managed people, like typically like I said earlier, like people are gonna be more on the cautious side of like, I don't wanna do anything wrong. And I've like intentionally told people before of like, hey, like I would much rather like rein you in than have to like always push you essentially and

again there's probably some context that's not gonna be true of and you have to know your situation your manager if you're in some kind of like highly cautious environment like maybe some kind of like you know making this a medical research environment pharmaceutical environment or like highly regulated like nuclear or something like it's different but you know generic business which you know a lot of us work in you know in various generic businesses I find that people really like just need to like take more risk essentially. Yeah, yeah, yeah. Selling product, digital product. Nothing's the world. Right. Nobody's going to die and we're not going to like start a, yeah, some kind of nuclear event. Awesome.

Well, our time has flown by. Any parting words for us? I always like to give people a chance to like, kind of, you know, give any advice or anything that's been meaningful to you in the last couple months. I think maybe the, at least recent things I've been reading, I think it's Ego the Enemy, but in the open mouth. Oh, that's good. Yeah. So the one tidbit I took from that, especially since we lived in a reading, I think it's Ego with the Enemy, but in other words, I don't know. Yeah. So the one tidbit I took from that,

especially since we live in a very social world, it's one thing to look impressive, it's another thing to be impressive. Oh, nice. As you're going out and I guess, trying to figure out what to do. Sometimes it's okay to go through moments where you, maybe you feel less impressive and you're not,

like, I don't know, showing off online, just to actually, you know, do something do something worthwhile. Doing things worthwhile is hard. And again, it's going to have failure, you'd be wrong, and the process sucks. But I've learned that I love that you're doing it. Awesome. Great parting words. Alright, Ben, thanks for being on the show. Thank you, question. The Data Stack Show is brought to you by Rudder Stack.

Learn more at rudderstack.com.