The Data Stack Show - 225: The Stone Cold Truth About Data: False Hopes and Hard Truths with The Cynical Data Guy

Episode Date: January 22, 2025

Highlights from this week’s conversation include:False Hope in Data Roles (1:17)Naivety of Junior Data Analysts (4:27)The Challenge of Defining Data (6:41)Struggles with Enterprise BI Tools (9:43)Career Advice for Data Professionals (12:36)Generational Shifts in Data Roles (16:51)Self-Service Data Requests (18:17)The Importance of Analysis Skills (19:46)The Broader Context of Analysis (21:44)Boring Challenges in AI Deployment (23:29)Technology Development vs. Human Absorption (26:14)VC Resolutions for 2025 (27:00)Value Addition in Leadership (32:08)Final Thoughts and Wrap-Up (33:06)The Data Stack Show is a weekly podcast powered by RudderStack, the CDP for developers. Each week we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.

Hi, I'm Eric Dotz. And I'm John Wessel. Welcome to the Data Stack Show. The Data Stack Show is a podcast where we talk about the technical, business, and human challenges involved in data work. Join our casual conversations with innovators and data professionals to learn about new data technologies and how data teams are run at top companies. Welcome back to the Data Stack Show.

Today, we are welcoming back one of our favorite recurring guests, who is now referred to as the stone-cold Steve Austin of the data world. The cynical data guy. Matt, welcome back to the show. Or Steve, I guess I should say. Thanks for having me back. Eckhart Ricker, thank you for that little tidbit right there. Yes, we do have to thank Eckhart for actually showing us what we knew all along is that you know that the world of data is actually a wrestling ring where people act out elaborate you know elaborate violent scenes but no one actually gets hurt it's

all k-fab that's all it really is okay well actually stone cold steve austin of data we have a really good one to start out with one One of our good friends, Rogajan, is he's been on the show multiple times, just a fun guy and a longtime friend of the show. Man, he had such a great post about false hope. So I'm going to read this and then I have a question for each of you. So cynical data guy and agreeable data guy. And this is just so good. No one is more of false hope than a data analyst building a dashboard they believe the entire company will look at. I got to get this out laughing. A data engineer who just had someone tell them that the schema will never change. A data scientist analyzing a data source they thought was 100% accurate.

A data leader planning out how to build the single source for their company. Okay, so here's the question. I hope the listeners are laughing because all of those are just unbelievable. Which one of these is the highest level of false hope or the highest level of delusion? I'll let you go first, Matt, but I definitely have one. Maybe two. You can't have two.

It's which one's the most. So, I mean, this is probably also biased just because of my background. I would say the data scientist who thought that was a hundred percent accurate. Cause that's just you sweet summer child that does not exist. Oh, I have, can I ask one, one like question on that?

Cause I don't, I'm not super familiar with the, you know, I have, I don't have a lot of direct data science experience, but the little that I do have like a really good data scientist sort of assumes that's, they actually like little that I do have, like a really good data scientist sort of assumes that's, they actually like assume that going into the pod.

Right? Like that's. It's always wrong. The first step is usually to figure out where it's wrong. Yeah. So that you can correct for it or remove things or whatever there.

Okay. I have more questions, but reable data guys, I need you. Well, I mean, as jumping off from Well, I mean, jumping off from that, that's essentially the whole field of applied statistics. It's like, we know this was wrong.

We're going to do things to make it more representative. But I'm going to go dashboard. I think that, to me, is the largest amount of false hope for two reasons. One, because data analysts tend to be like that tends to be like a career starter like there's a lot of people that start their career

as a data analyst some as engineers or scientists but you know data analysts most of those jobs they're like a lot of companies have entry-level data analyst jobs and maybe don't have entry-level you know data science jobs so that there tends to be, at least for juniors, a higher level of naivete for that particular role if you're a junior data analyst. And it just makes me laugh because I've been that person. I've believed, like, this dashboard is going to be great. Even sometimes this executive wants me to build it. So you even have an executive that's a little delusional about it and like yeah we're gonna have this like wonderful thing it'll be our

north star for the company and everybody will look at it well i mean so yes i think there's naivety there because a lot of times they don't know any better if they're earlier in their career but i kind of view that as not the most delusional because of that now if you're and also because if you're like a it's more excusable yeah and if you're like a 10-year data analyst like your soul has been getting ground down for 10 years so the cynicalness of it is going to be pretty high sure there. You have a pretty clear view from the mountain of unused dashboards that you spend upon. Right. I mean, I think it was my first dashboard I built

that was, I had just moved in to a role from another part of the company and my boss was like, we're going to build this thing for operations. I was like, yeah, sure. And no one ever used it. And I was like, why? And then I looked at it and I realized that a meeting we had with operations, no one had used it and i was like why and then i looked at it and i was like i realized that a meeting we had with operations no one had asked for it and like right there i was like well

i'm never doing this again like yeah i was so excited one of my first dashboards i was so excited because it went on the tvs at the big operations center i was like man you know like i don't know 100 how many hundred people are in the operation center at this company. And then I find, and then, and it was a remote site. So I'm like physically located somewhere else. And I'd occasionally visit this like operation center. And I remember visiting and the TVs were off and then I inquired about it and it was like, oh yeah, like TV has been broken for like a month and for a month and they may fix it at one point. And it occurred to me, oh, they don't care that the TV doesn't work and they don't look at my dashboard

that's on the TV. Well, and one other thing I'll just say is I think on that kind of sliding scale, the data leader one would generally be the highest, I i feel like because you should know better except for the fact that i think a lot of them don't believe it that's just what they're telling management like yeah we're gonna work towards that but i don't think they really believe it and if you do believe it at that point you're the biggest sucker in the room then

that's a great point in that the single source of truth, the single source of truth conversation actually a lot of times comes from management or the business, right? Where there are all these problems from all these functional areas of the business where they say, oh, well, I need this data or I don't have this information or there's some sort of problem in me hitting my number because of this data, right? And so then someone from management says, okay, this is a technical problem, a data problem. We need to solve this at the root. And so the data leader, sort of their big project is,

okay, you need to go figure out how to build a single source of truth, right? Yeah, I feel like that comes up partially because you get the situation where you have an executive who every month they're doing reviews and it's always well our numbers say this but finances numbers say that but marketing's numbers say that and so they're like you know what if i could just have one place where like all the numbers were the same and everyone

had to use them but the problem with that is they don't realize is that one none of the business units want that because they've all skewed it towards what's best for them and they are going to fight you on that so you're just adding another number to the fight and also i think also there's this thing where they're like you data person go define this and a lot of this stuff is not like definable by the data it's like okay well we need to know what a sale is well finance has one definition. Marketing has another definition. And no one wants to be the person who says, everyone shut up, this is what the definition is.

I think I'm least cynical about this one because I have had some success with that. But at a large company, yeah, forget it. That's never going to happen. Smaller companies, you never get there 100%. But I think you can get closer like at a large company yeah forget it like that's never gonna happen yeah smaller companies you know you never get there a hundred percent but i think you can get closer and you just yeah there's a lot of things i have to go right to get close to that i think it's one of those if you can fit everyone around a table with a pizza yeah you have a chance of that exactly yeah once you get beyond that

yeah no yeah and especially once and especially once budget and comp is tied to any of this stuff, you're screwed. Okay, quick question because we have many more juicy morsels to move on to. One question on the data analysts in the dashboard. So I was meeting with a customer in person, actually, which was great. It seems to be more and more rare.

And we were talking about data and they were, you know, discussing sort of some of the things they wanted to do in terms of capturing product telemetry to understand onboarding better and, you know, just some basic things that they wanted to do. And so I was asking about what they currently do for analytics. And this is a startup, okay? So it's a, you know, a venture-backed startup company, not very big, but, you know, you know, sort of early stage, okay? So it's a, you know, a venture-backed startup company, not very big, but, you know, sort of early stage, right? So it's looking for product market fit. And they said, and this is a product manager, and they do some data stuff in their platform, actually.

And this is the product manager for their data, you know, features and functionality. Super smart. And he kind of laughed and he said, well, we have Looker and Tableau. And I'm thinking, wow, this is not a very big company. How does that happen?

Because to me, that gets at this false hope of someone is onboarding non-trivial enterprise grade BI tools at you know a small company and they're not dumb people right these are people and then you're talking head count of like 50 yeah yeah like like very small yeah yeah yeah somewhere yeah somewhere so how does that like that dynamic happen right because that's a that's fast and i guess like this stark contrast of like we have two enterprise grade bi tools and we're talking about how to get

basic product telemetry so we can optimize onboarding flow is that's just fascinating it's a speed optimization thing usually where like some person comes in they know tableau they like it would take them x amount of time to ramp up on said other tool the person or they just don't want to or they just don't want to sure yeah yeah the person before him used looker and like and and then they make the argument hey we're a startup like it would take me x amount of exaggerated time to ramp up on this other tool let Let's just use this tool I already know and then like tout some benefits of whatever tool they already know that may or may not be true compared to the other tool because they don't really know the other tool.

I think also one of the ways you can have that is the person comes in they want to use Tableau they don't even have that discussion first. They just download it, build stuff and then they're like it would take me so long to do this in Looker, but look, I've already got it in Tableau.

So now you have to buy Tableau for me. Yep. Probably also BigQuery or the Google team and what they did with Data Studio and Looker. I believe this company is running on BigQuery. We didn't talk about this specifically. But man, there's an easy on-ramp to go from data in BigQuery. You can get it in Looker Studio and then, you know,

all Looker. Like that pathway is super easy if you buy BigQuery. Yeah. And if you started with, there was like one or two people with like a Tableau license, then someone else needed to do something in another department. And it was like,

look, we can just turn it on. And we've already got it up there. And then once it's up, nobody wants to change it because nobody wants to actually consolidate it into one place. Okay, moving on to round number two.

This will be a double header here. So two posts. So one is from Tris J. Burns. And this is great. This is going to be a great topic. Short examine here. So two posts. So one is from Tris J. Burns. And this is great. This is going to be a great topic. Short statement here. Before starting a career in data, I highly recommend gaining experience in another field. And then I'll follow that up from another longtime friend of the show, multi-time guest, Ben Stancil, who continually just produced his unbelievable thoughts generally. But a quote from a recent post, a couple of quotes here. We can't just be analysts or analytics engineers. We have to decide that we want to be true experts in understanding how to build

consumer software first and product analysts second, or define ourselves as working in finance, then become an analytics engineer at a fintech company. Because there's a corollary to Dan Liu's theory of expertise, while it implies that we can become pretty good at stuff pretty quickly, it also implies that other people can become pretty good analysts. And in almost every field, that combination, a domain expert and 95th percentile analyst is almost always better than the inverse. And then I'll close it out with another, this is a paragraph down.

We probably can't get away with being good at asking questions. We need to know some specific things too. Cynical data guy. That one cuts a little bit that last line right there because I've used that before. I think, I mean, I think this is one of those that, I don't know, you kind of, like,

if you've been in the field, you can kind of go back and forth on it sometimes where it's like, yeah, we really need people who know the business and stuff like that. But if you don't have enough of the data working with it, then you run into a lot of problems with it. And then like you'd flip over to, no, I just need someone who's really good at data, right?

Like I just need a data engineer who can just step in here and doesn't care and can just put stuff in place. So, you know, I think there's, I think especially like 10 years ago, you could kind of be more of like, I'm a data person. I think that's shrinking over time. I don't think it'll get rid of it completely in the near future, but the idea of we're

going to have like a really, you know, like a data team that does like, okay, it's all the data analysts and they just are like data experts who help the other, the other parts of the business. I don't think that's going to last. I think that's already kind of gone in a lot of places. Agreeable data. I think, and this is, I think this is in the same article, he kind of, Ben does this like really

good comparison with data and science and says like, hey, science is a subject you can take in school. And then he has this like like great like science is what you take in fifth grade not what you win a nobel prize for so like like there's that like yeah yeah that's so good which is and that's especially what's becoming data like like what data is becoming for businesses it's like cool you do data that, great, you have a business degree? Like, that tells me almost nothing. Because it's so broad. And I think part of what the domain knowledge brings to it is it brings specificity and more clear application.

Like, oh, you know about marketing data or marketing data for law firms. You can continue to drill into specificity, which makes it more valuable. Well, I think if you look at the original kind of data people if you go back 10 plus years ago there was no like institutional educational infrastructure for it so you had to start at something else and then you saw this thing as they didn't get into it and then there was kind of the move towards semi-professionalizing it, I would say. It wasn't completely because the barrier centuries still weren't super high.

But there was this move towards like, oh, no, this is going to be a professional thing that you do. But I mean, I think where that's going to go away partially, I think that was partially a generational thing. You know, the idea of like, hey, we need a data person to do this is a little bit like going back to the 90s and having someone be like, I need an assistant to do my email because I don't do email. Like that whole, there's a generational aspect of it too. If you didn't grow up with it, you don't really want to do it, but there's going to be, you know, you've got people who are now in their 40s and 30s that like they've grown up with this basically in their corporate life. They're going to be more likely to be a data person, you know, be a finance and data person or a marketing and data person. Yeah, sure. I was, I worked with someone. Actually, this person exemplifies someone who had domain expertise in multiple areas and wasn't technically an analyst and had never worked as an analyst, but was probably

one of the best analytical people that I've ever worked with. They worked in supply chain, they worked in marketing disciplines, and were unbelievable at, you know, at like general math, right? And so you combine all those things. And it's like, wow, they're unbelievable at like, solving problems with data or like uncovering things. Anyways, coincidental that this person fits that exact profile. But returning to your point about like someone answering my email, they worked for someone, I want to say they were like, I don't remember the details. It was some sort of chief of staff type position where they were, you know, sort of assigned to an executive to solve problems, right? Go in and solve this problem, right? But one of the things they did was they had to print the executive's emails out for them, like figure out what the important ones are and print them out and put them on my desk. And so anyways, Matt, I was like, okay, what's the data corollary to that?

Like printing the email, you know? Well, I think, I think some of that is like self-service BI, right? Where you're like, Hey, look look you can go do it and they're like can you put that in a pdf and send it to me right like answer my question for me copy it over into an email and send it to me yeah that's what they want i don't want to go look through yeah can you make my google sheet update instead like i don't want to open the dashboard yeah can you i don't want a google sheet can you just like screenshot it and send it to me for me like that's what i want or put it back in salesforce or my marketing tool i don't want to like go anywhere just let me yeah just let me use my tools and shove the data in there so

i think it also depends on the job because there are those infrastructure jobs that like you know realistically you're not going to go from kind of necessarily i was in the business to now i'm doing what's becoming more and more a very like software engineering type role right but but i think for those analyst ones and i will say i think also if you come from like there's two ways you can kind of go about this. You can either be a person who really cares about data for some reason, and then you have to learn the business aspects. But I've found a lot of people who come in through that data one, especially in the last, I don't know, five years or so,

they don't really care about the business enough to like want to learn those things. So it might be easier to go the other way around. I do think though, we also need to start defining that a data person is not just someone who knows SQL and a little bit of Python. Like that tends to be,

if you look at like, hey, we're going to turn our business people into data people. It's we're going to teach you SQL. And it's like, okay, but to do this well, you have to learn how to think well. Yeah.

And that's the part that's missing in a lot of this. And if you don't do that, you're given, it's going to sound bad, but like you're given a monkey, you know, like a chainsaw and you're like, look, he can do this. Well, no, he can't do this. He's going to cause damage. Sounds a lot like, you know, WWE in the, you know, monkey, the monkey, the chainsaw in the wrestling ring. What could go wrong? I'm sure there's some ring in Japan where they've done that. Yeah, yeah, yeah.

No, I think my, as moderator, my, I don't even think it's a hot take on that. I agree, Matt, in that my, I believe that this has always been true, actually. And what I mean by that is when I think about people who are the people that I've worked with who are incredibly good analysts, they fall on an extremely broad spectrum of technical skill with data, to your point you know sql or python what they're really good at is understanding and breaking apart a problem into its component pieces yeah so they know what type of analysis even needs to be done right and right and even those people like we we kind of made joked about you know printing the emails or like putting the the thing in a PDF. But the thing is, some of those people may not have the technical skill to do it,

but they know the best way to solve the problem. And they may need to bring someone with the technical skills in to help solve a legitimate, really tricky statistical analysis problem because of some very outliers, underlying data issues or whatever, right? But to your point, Matt, they understand the best way to approach solving the problem because of the larger context. And that's actually the core of like good analysis.

Yeah. And I think that's the part that's missing when we talk, when like, if you make the comparison to, you know, data is like science. Well, science has a method to it that is then applied in all these other ways. We don't really have a codified method. I think there's people who are good at it,

and if you talk to them, they all kind of fall within this narrow range of how they do it, but no one's teaching you to do that. You have to kind of figure it out yourself. Yep. Okay, round three. And then if we have some time, we'll get to a bonus round.

This is a really great post. So this is Kurt Mumel. I think I'm pronouncing that correct. So sorry, Kurt. Please come on the show and correct me. We'd love to have you as a guest. Is AI democratization a myth? No, but it's a 20-year project.

This is a deeply insightful discussion with DataIQ. Am I saying that right? DataQ? DataIQ. DataIQ. I like read that in my mind all the time, but I never, I don't think I've read it. Yeah, it's like haiku.

It's like the haiku. Yeah, DataIQ. Yeah, yeah. Okay. Discussion with the DataIQ co-founder and CEO of Florian Duotto. Two main takeaways. One, best AI ML data applications will be built by multidisciplinary teams

that blend data and domain expertise. Two, the capabilities of LLMs are not what's holding back enterprise deployment of generative AI. It's all of the boring stuff. Security, data quality, monitoring. All right, cynical data guy. I mean, yeah, I think there's the truth in that is probably the, we overestimate all of this stuff in the short term. And then we run the risk of underestimating

it in the long term. So I mean, I think that part is true. I think the idea of you needing multidisciplinary teams, I think for a lot of this, for a lot of data and technology stuff, that's always been true. It's, it's not a one team thing. It's not a one person thing. And I think like, yeah, the thing that's holding a lot back is the boring stuff. But I also feel like that feels slightly hand wavy to me and it's not hand wavy it's hard it's hard there's a lot of details you know this is where it's this is that like you know it's almost like saying oh that's like the last 10 well that last 10 is going to take

one to two x as much energy as the previous percent yeah to the best analogy i can think of would be when viddy like when video first worked over the internet like you could stream a video like you're like wow this is really neat this is great this can revolution everything like to like netflix with like this massive streaming platform with like you know hundreds of thousands of videos that can be accessed instantly from anywhere in the world. That was not a short, trivial journey between those two things.

And I feel like that's a good, maybe even should be, that might even be under-representing the effort, but that feels like, okay, just because you know we can stream one person can stream a video at a college on a t1 connection or whatever like there's all these other things that need to happen you know to make this a reality but the interesting part is that the boring stuff is going to be like lagging and he says enterprise like I think that's almost any company. So I don't think it's just if you're thinking...

My mind went to big enterprise. No, I just replaced it with any company deploying some kind of generative AI. But the interesting part is the LLM curve I think is just going to keep going. And then are we going to just have a growing and growing span of like LLM capability from actual implementation capability if that makes sense um so that'll be interesting because essentially like I don't know I don't know what that gap is

going to be and if the LLMs keep developing at like x rate like like are there things we can do to speed up the Y, which is the boring stuff? Like, yes, but I don't, but it'll be interesting. Like how does that gap progress over time? Does it decrease or increase? That's almost kind of like the, you know, there's how quickly technology can develop and there's how fast people can actually absorb it

and integrate it. It sounds similar to that. I mean, I think also just learning, because I think we're slowly starting to come out of it, but for the first couple of years especially, there was this idea of LLMs were going to do everything, and we were just going to shove everything into the LLM instead of realizing it's got to have a,

it has a part and it's an important part, but it may not even be the central part of whatever system or agent or whatever it is you're using. You still need the deterministic elements in there. And that probably has to be the driver not the passenger.

It's going to be wild 20 years. Okay, Lightning, do we have time for a Lightning round? Yeah, I think so. Okay, Lightning, do we have time for a Lightning round? Yeah, I think so. Okay, bonus round. Sorry, bonus round. So Matt Turk has created so much great content over the years, and he had an amazing post about his VC resolutions for 2025. Okay, here's what we're going to do with this one.

Matt and John, can you pull this up and just pick your favorite resolution? Pick your favorite VC resolution. Yeah. And most of these, I think, are written kind of humorously. Oh, it's so tongue-in-cheek. But are also like, you know,

like the fun, like tongue-in-cheek, but have some like nice nuggets in them as well oh man I'll go honestly the first the first one just really got me laughing we made you pick one earlier in the show

and this is a bonus round we'll go back and forth for one or two of them the first one really got me laughing and so it's vc resolutions for 2025 shed the past is number one on here delete all my posts from last january about why the apple vision pro will be the big story of 2024 which is so great like the you know like it's i i think one of the things in like the age of social media the way that like media cycles work now is no one looks back like occasionally like posts will recirculate right like you know five ten years later or whatever but like pretty

much no one looks back so i think it's's actually really cool to pull something up and be like, hey, this was a projection or whatever I had from January that didn't work out as I expected. I will say, though, just one quick comment on that. And this is one anecdotal data point. But one of the best engineers that I know, very successful entrepreneur, moved to an Apple Vision Pro for their workstation and they said, I'm never going back. Are they still using it?

Do you know? Yeah, they use it exclusively. Really? They have a laptop and stuff. But they're a year into it then? Yeah. Wow. Yeah.

And then actually, someone who works here at Rudderstack is they went, I need to just call them and say, Hey, can I come try this out? But they went, they were at their house and they tried it out. And they said it is like absolutely unbelievable. The cost is super high.

Sure. And it's very dramatic change. But anyways, just one anecdotal data point, but like someone who I generally respect and we've talked a lot about workflow and tooling and all that sort of stuff. And so it was really shocking for me to hear them say,

like, this is it, I'm not going back. That's super interesting. And I've heard people say that and they do it for like a couple of weeks or a month and then they do go back. You know what, I'm going to text them today. You should. And on the next Stone Cold Steve Austin. But if they're

a year into it and they really have stuck with that, that's a huge win. That's a really interesting... We'll have him on the show. You should ask if he's now going to the chiropractor because the date of it was one of the biggest complaints. Oh, man.

Okay. All right. We're going gonna go with six focus add a filter to my inbox to automatically discard startup pitches that do not start that do not use the word agentic and say that i'm using ai in my deal flow process just leaning into that vcai obsession man costas the former co-host of the show when he was he has a startup so some point we need to ping him and have him come on the show i think they were getting close to being ready publicly but god this was maybe a year ago maybe a year ago we were messaging back and forth and just catching up on life. And I was asking him about, you know, are you talking with investors and all this sort of stuff? And I need to try to see if I can pull it up.

But he had the funniest statement about, you know, sort of just going up and down Sand Hill Road saying, you know, foundational model. And he's like, I mean, I don't know how much money you would walk away with. Sort of the digital version. Okay, one more in the bonus round. Okay, I've got it. This one, I'll choose between. Okay, I'm going to pick number nine on here. And it really got me laughing.

So each of these, like, thoughts are pre, there's like a little, like, summary. So each of these thoughts are pre... There's a little summary. One of them says inspire. One of them says anticipate. So this is the one under add value, which makes me laugh. And it says... I'll skip down.

Tell my founders to be more like Sam Altman, which I know they really appreciate, even though they often don't say anything in response oh man that's a good one like and he's like and that used to be like elon musk like i feel like you know like four or five years ago steve jobs there's always been a guy that that that's been that guy but that one made me laugh too. And that it's under add value. Yeah. That one's going to be related to,

I chose number eight, inspire. My CEOs should not forget about founder mode. Text them helpful reminders from the pool during my upcoming midwinter break in Cabo. He really just goes for the jugular on the stereotypes. So good. You know? Matt, if you're listening, love to have you on the show. Yeah, we'd love to have you on the show.

All right. Well, that concludes our bonus round. Stone Cold Steve Austin, thank you as always for joining us and sharing your war stories from deep in the bowels of corporate data America. And thank you to the listeners.

We'll catch you on the next one. Stay cynical. See you guys later. The Data Stack Show is brought to you by Rudderstack, the warehouse-native customer data platform. Rudderstack is purpose-built to help data teams turn customer data into competitive advantage. Learn more at ruddersack.com. Thank you.