The Data Stack Show - The PRQL: What’s Driving The Evolution of Data Operations? Featuring Kunal Agarwal of Unravel Data

Episode Date: March 4, 2024

In this bonus episode, Eric and Kostas preview their upcoming conversation with Kunal Agarwal of Unravel Data.

. Welcome to the Data Stack Show Prequel. This is a short bonus episode where we preview the upcoming show. You'll get to meet our guests and hear about the topics we're going to cover. If they're interesting to you, you can catch the full-length show when it drops on Wednesday. We are here with Kunal from Unravel Data. Kunal, thanks for spending a few minutes with us today. Eric Costas, thank you so much for having me here.

All right, give us your background. How did you get into data and what are you doing today at Unravel? Yeah, so Kunal Agarwal, founder and CEO of Unravel Data that I started with my co-founder Shivnath Babu, who is a professor of computer science at Duke University. We both started this company to simplify data operations. We feel data engineers and data teams spend too much time firefighting issues rather than being productive on the data stack. And we wanted to automate and simplify some of the daily activities that these teams have to wrangle with every day and simplify their lives a little bit and make the data work that they do more exciting. Yeah, and I'm pretty excited to be with you today, Malf, because I really would like to chat about how data operations have changed in the past like 10 years like it's extremely interesting that you've seen this whole like from the hadoop days up to today like all the changes that have happened

and i have a feeling that like the complexity around like data operations has like exploded right especially with like having pretty much like every one or two years like new use cases around data coming in, right? So even like, let's say, observability in data. What does it mean? What it meant like five years ago and what it means today when we have also like AI, for example, right? In the mix. So I'd love to get more into like this journey, how things have changed and what it means today.

I like to operate. I like to be an operator around data. And of course, learn about Unravel and how it helps in that. What about you? What are some topics that you're excited about? Yeah. No, of course, there's never a dull moment in the life of a data team member, especially for the last 10 years. So we've gone from doing things with Hadoop as a one Swiss army, if you may, to having

a multi-system stack now, primarily running, used to run on-prem, now primarily running on the cloud. That's a mega change that's happened. The other is we've gone from doing these batch workloads with ETL to now doing real-time or near real-time workloads in production and not just as a science project. And then we've gone from doing these BI, business intelligence, or just advanced analytic workloads to now doing machine learning and AI in production. So if you're part of a data team as a data engineer or a data scientist or a data analyst,

you've had to keep up with the demand of your business and also had to keep retooling and reskilling yourself on how do you work on a map-reduced base system to now a BigQuery and a Snowflake system. It's incredible the rate of, the pace of change and the rate at which things are getting involved in this ecosystem, right? So that's a very exciting part for us because what Unravel is ultimately helping to do is to simplify how these data engineers or data analysts are creating their applications, how they're making sure that these applications are

reliable, that they work on time every time, that they don't break, that they're making sure that these applications are reliable, that they work on time every time, that they don't break, that they get the right results. And then for the data leaders and for people who are managing these projects, making sure that they are able to scale in a very efficient manner. It's not a linear scale in dollars versus productivity. Can we bend the cost curve as these environments are scaling up so that they're starting to get more bang for their investments? And as we now see, the most exciting thing, it's actually here right now, not even the near future, is AI and how those workloads are changing businesses and turning industries upside down.

So it's a really powerful industry to be a part of and really exciting time to be a part of this industry. But it's not for the faint hearted. It's for people who are up for a challenge, who like change, who like evolving, who like to try out new things.

And that's what makes it exciting overall for everybody in this industry, and I'm sure you've had the same experience too, Costas. Yeah, 100%. I can't wait to get deeper into all that. Eric, what do you think? Let's dive in. Let's do it.

All right, that's a wrap for the prequel. The full-length episode will drop Wednesday morning. Subscribe now so you don't miss it.