The Data Stack Show - 28: Next Gen Data Governance with Stefania from Avo

Episode Date: March 10, 2021

On this week’s episode of The Data Stack Show, Eric and Kostas are joined by Stefanía Bjarney Ólafsdóttir, the CEO and co-founder of Avo. Avo, which started in 2018, provides data analytics governance as a service, helping organizations make data-driven decisions to improve their customer experience.Highlights from this week’s episode include:Stefania's background with mathematics, philosophy, bioinformatics and consumer mobile (2:39)Making pioneering decisions as head of data science at QuizUp (8:34)Is less more? Choosing fundamental parts of the customer experience and understanding them very well (16:56)Bringing data consumers closer to data producers (18:34)Avo mission to provide analytics governance as a service (25:09)Avo use cases (36:37)Focusing on event-based data (44:29)The Data Stack Show is a weekly podcast powered by RudderStack. Each week we’ll talk to data engineers, analysts, and data scientists about their experience around building and maintaining data infrastructure, delivering data and data products, and driving better outcomes across their businesses with data.RudderStack helps businesses make the most out of their customer data while ensuring data privacy and security. To learn more about RudderStack visit rudderstack.com.

Welcome to the Data Stack Show, where we talk with data engineers, data teams, data scientists, and the teams and people consuming data products. I'm Eric Dodds. And I'm Kostas Pardalis. Join us each week as we explore the world of data and meet the people shaping it. Okay, Steph from Avvo is our guest today. And the question that I want to ask her is, which this won't surprise our listeners who've been around for a while, but it's about her background. She studied mathematics, did work in bioinformatics, and then went from bioinformatics into consumer mobile. And whenever I see sort of a progression like that, where there's a significant

sort of change in, you know, the industry or business that you work in, it's just always fascinating to me how that happens. So that's what I'm, that's my So that's my main question I want among many others, but that's the main one. How about you, Kostas? I'm really interested to learn from her about data quality. She has an amazing background. And actually, she started like super early trying to solve these problems.

And this journey also ended up by building like AVO. So I really want to learn more about how she perceives data quality, why it's important, how it can be solved. And of course, learn more about how AVO addresses these issues and delivers value to their customers. So that's from my side. I think we are going to enjoy this conversation with Steph a lot. I agree. Well, let's go chat.

All right. We have a guest I'm really excited about. Steph from Avvo. Thank you for joining the show, Steph. Thank you for having me. Super excited to be here. Yes, yes. Well, so many things to talk about.

But as our listeners know, I really love hearing about people's backgrounds and stories and what they did before they ended up doing what they're doing today. And your background is fascinating. So you studied mathematics. I'd love to hear about, you know, hear about that journey. And did bioinformatics and then you jumped into consumer mobile. And so I'm just fascinated to hear how did you go mathematics to bioinformatics makes more sense, but bioinformatics to consumer mobile seems like a crazy jump. And I just would love to hear that story. Yeah. Well, thanks for highlighting it. I guess maybe one interesting aspect that might explain it a little bit. I didn't only study mathematics, I actually did a double and I studied philosophy as well, which makes me extremely curious about people and helping other people answer questions and also just ask good questions so you know that might explain a little bit about or yeah like how I moved into also being curious about just user experiences and people how they

experience games and things like that but well another aspect as well that I just highlighted recently in another interview I moved around a lot as a kid. I had very young parents and I actually had moved around 30 times before I finished high school. Wow. And that also I think makes me extremely curious about people. So I love asking questions. I'm extremely curious. I think people describe me that way. One question I just have to ask. So mathematics and philosophy, you would generally think about those as completely separate. But since you were studying both at the same time, were there any commonalities that stuck out to you in the two different disciplines? Yeah, I love that you brought that up because yes, absolutely. They are surprisingly similar. I feel like the world is sort of split into two groups,

the people who think it's obvious that you would study those two together. And then the people who think it's like, well, that's really weird then and then when you take a little bit into it both of them are very i guess logic oriented mathematics is rigorously logic oriented and has language to describe everything while and trains you up and really you know being logical and deducing logic and and you know finding the truth and philosophy does the same except it doesn't does it via you know written language and you are trained into taking texts of very complex words and vague statements and translating them into what are the things that are the actual assumptions behind this and is it true and what

are the first principles behind it and so I thoroughly enjoyed both but obviously also philosophy stretches a little bit into you know ethics and and sort of thinking a little bit more about people and behavior and society and how we should build our society. And so I really enjoyed that part of philosophy as well. Yeah, it is. It is interesting now that you say it that way and we can move on because I don't want to spend the entire time on this because I know we, I know I could too, but we have to talk about data because we named the show,

the data stack show. It's a, it's a legal requirement. But it is interesting, the concept of sort of having tools and mechanisms that you leverage in order to understand something or find a solution to something. And that's the same with philosophy and mathematics. Absolutely. Absolutely. It's just extremely oriented towards sitting for long times and thinking about very short statements and digging into them and like figuring out what the hell is true behind them. So yeah, I love that. Well, tell us about what did you do in consumer mobile? So you took that background into consumer mobile and what specifically how are you applying your skill sets there? Yeah, exactly. I'll go through them, jump through from, so I went from university to, well, first I traveled around Australia for half a year, which was amazing. And then I came back home and to Iceland, I'm from Iceland and applied for a couple of the sort of industry, which would

probably be called startups, but to me were like just huge corporate organizations. And one of them was a DNA analysis company, I guess, called Decode Genetics, where I had the opportunity to learn from amazing statisticians, world-class statisticians that have released fundamental discoveries

and statistics and imputation regarding how you calculate people's DNA based on sample data or based on their familial information and things like that. So in that role, I did a lot of distributed computing, setting up all sorts of calculations and merging data sets again. And, but I also worked with the people over there that were like doctors and biologists, and they were trying to, you know, their, their, their perspective was, you know, I'm, I'm, I'm curious about the mechanics of a physical trait of a human. And then it was my job to bring in the information about like, how could maybe the DNA statistics

and the data about their DNA have actually impacted that physical trait. So I was correlating physical traits with DNA mutations. And that was really fun. Yeah, I love that. And so I was one of the few people in Iceland at that point in time that had really worked with huge data sets for reals and not just, you know, what you study in school or like, you know, I don't know, because you can train yourself in, you know, writing software and working with big data sets, but nothing really compares to actually,

you know, having to go through the pain of fixing it in reality. Oh, sure. There's such a big difference between stubbed out data for, you know, sort of learning and exercises and like real world data. That's totally different. It's totally, yeah, absolutely. And that jump was really educational for me. And I had sort of started studying then bioinformatics after this job and was actually had gotten offered a position doing PhD also in statistics slash bioinformatics in Oxford and also in Berkeley, which was really exciting. I was excited to move to California as well. And then this mobile game called QuizUp blew up all of a sudden and reached 1 million users in its first week which

was the fastest growing app in the app store at the time and the founders were icelandic and i had some friends that were working in the company and i was like i can do a phd whenever but when will i have the opportunity to work with these really super fun people and building up a data team and like learning so many things about these people that are playing the game and I can build like a recommendation algorithm for what quizzes you should play if you like Harry Potter and things like that so I got super excited about being joining that team as a founding analyst that was sort of the shift basically so I was just like the Icelandic community type of thing it's a small society and just for example our the the coach of the national soccer team in in soccer he is also a dentist

yeah everyone does everything in Iceland so that's I love that. So that was the switch. And that was sort of the backstory behind me joining this company, QuizUp. And this was 2013. And so that was super early product analytics. I don't think the term even existed at that point. And just a couple of developers had thrown together analytics events using Mixpanel, if I remember correctly. Yeah, they were pretty early. Yeah, this was super early Mixpanel.

This was like Mixpanel was still, you know, we had a bunch of people that were just placing like Powered by Mixpanel on their website to get some free events. I don't think Amplitude even existed at this point in time. Segment maybe just launched their famous Hack News post where they released a wrapper around all of your analytics. Yeah, that was like the genesis that time period. Exactly. So when you were Googling stuff, I was just like, what is retention, you say?

What does that mean? And you couldn't really find definitions of it. And so when the board was asking our CFO, who I reported to, about retention after every launch that we made and for every board meeting, I was like, I don't know what they really want. You have millions of definitions of retention that we could work off. And, you know, I don't even know what the benchmark is. I don't even know what they would consider to be good, what they would consider to be bad. So I just spent my time at that point digging into data. But, you know, most of the data was terribly broken and inconsistent.

These few events that they threw together into Mixpanel. So I sort of spent my time just hacking together scripts and you know, you know, making replicas of the operational databases, which were like 16 shards of the user base and Postgres hosted on AWS or something. And it took maybe probably typically two weeks to answer some really basic question. Obviously that was not acceptable. So we ended up fixing that problem by basically throwing people at the problem.

I started scaling up the data team, hiring more people. So more people could be answering questions like these. But eventually, you know, or, and while we were doing that, we were always, for every single feature release, we were begging the mobile developers to add, you know, specific events for some things and questions that, you know, the board always wanted to answer or the CFO or the CEO always wanted to answer.

And it was so interesting the dynamics because we were in the position to have to report on these things, but we had absolutely no control over what data existed. Absolutely no control over that. But in some way, it was still our fault if we couldn't answer the questions, which is a bit of a tricky situation to be in. So that was sort of the birth of my sort of, you know, how I evolved this role, which was really amazing and a great opportunity to be head of data science for this company because it was, I guess, a twofold role. I got the opportunity to really build the data culture,

which is sort of, you know, getting people curious to use data to answer questions and go beyond their gut feeling. I say beyond because I know that gut feeling is extremely important in making product decisions and sort of you have to have some insights and sort of an understanding of the customers and the users. But, you know, it's really important to be able to back it up with data. And it also just makes the buy-in from anyone on the company just stronger

if you're able to do that. So just getting people curious to do that and basically doing some internal marketing on like, you know, hey, here are some cool things we know now because of data. What could you know? And, you know, having regular meetings

with everyone on the, you know, in all of the different departments in the company, trying to figure out what their needs were, trying to just like lobby, lobby for people using data. And that was really fun. So building up the data culture. So that was one aspect, data culture.

And then the other aspect is just everything around the technical infrastructure of data. So making sure we have the right tracking in place, making sure we had good quality of the data that we did have, making sure we had the infrastructure in place to both do self-serve analytics via tools like Amplitude and Mixpanel, but also extensive raw data analysis and building the infrastructure so that the data scientists were able to, you know, we built our own version control for Jupyter notebooks because no tools existed at that point to do that.

But everything is already built around that today. So nobody has to do that again. We scraped at SDK webpages to get some data from there because they didn't have their own APIs. And we built out like Redshift clusters and pipelines to pipe the raw data from all of our data sources into Redshift, which could have been done with Segment today, obviously, or on Particle or any of the CDPs, Rudderstack, all of the amazing tools that now exist.

But none of that existed then. But yeah, so it was hugely, it was a wonderful learning experience. You were pioneering so many things. It was fun, certainly. I learned a million things. Well, I'm interested. So one more question.

I have been very selfish with the time here. And I know Costas has so many questions to ask. And we want to hear about Avvo. But one more question. And I'll ask you to do the impossible task of distilling this down into just a couple quick lessons. But hearing about your experience pioneering tooling that almost a decade ago didn't exist and now operating in a world where product analytics is table stakes for any software

company or sort of high-performance software startup. product analytics is table stakes for any software company, you know, or sort of high performance software startup. And there's so much infrastructure. I mean, that's a, that has been a tectonic shift to not only sort of live through, but also have actively taken part in. So I'm just interested to know what are some of the, you know, the top maybe two things that stick out to you having seen and really been part of sort of pioneering that entire shift in the data space?

Oh, that's a really good question. So obviously I have made very strong opinions here, I guess loosely held because I love being questioned on them and having the opportunity to discuss them because I know that there are just so many people that have amazing experiences. And just like if anyone is listening that has anything amazing to add to what I'm about to say, I would really love to hear that. So reach

out to me at via Twitter or like my email or whatever. But I think there are probably two things that probably two things that are counterintuitive to anyone who is working in product analytics and this sort of fresh in product analytics and was very counterintuitive to me also when I started. And that is this on one hand, I would say that less is more is that that is sort of like a counterintuitive thing. And that's also a very debated point of view, I would say, in data. But from my perspective, it's more important to choose fundamental parts of the customer experience and understand those points very well, rather than having vague understandings of, I guess, a wider set of data. This is one of the things that I think matters a lot. And there are multiple reasons behind it. And I can definitely go deeper into them. And I would love to actually,

but less is more. It's a counterintuitive one. And I think then the other one that I really fight for and I'm very passionate about myself is, and I've had the opportunity to see through in QuizUp and then as a consultant after QuizUp, working with a few companies around the world and like just advising them what they should do next. And then obviously with our customers at Avvo. And that is, I have a deep passion for bringing data producers closer to data consumers. And when I say data producers, in this context, of course, I'm mostly thinking about customer experiences. And so that is often coined as product analytics. And so the data producers of those are the developers, the

product developers that write the code within the applications, the web apps, the mobile apps, or sometimes the backend that actually get events into other systems, into Roger stack, segment, mixed panel, amplitude, heap, Pendo, Firebase analytics, your BigQuery instance, wherever you want to put your user experience event data, the producers are the developers that write the code in the product that triggers an event when the user does something. So those are the data producers.

And the data consumers are the data analysts and the product managers that then use tools like Amplitude and Mixpanel or Firebase Analytics or Looker or Tableau or whatever to look at that data. And what I saw happening in QuizUp, for example, is the data producers and data consumers were entirely separated. And I was just sharing this in the story just before, which is I reported to the CFO and the CEO and the board of directors, and I made reports for them. And their expectations was that I would be able to answer behavioral questions. But the product team had no obligation to sort of let me know when they were about to release something or, you know, ask me for input or feedback

on how they should instrument some data or make sure the data existed. And that meant that when I was asked for information, I had just no way of answering those questions. And obviously, that was, I guess, a major driving force behind this in sort of bringing data producers closer to data consumers. And was the driving factor behind working closely with the CTO of QuizUp and building tools and processes that added way closer collaboration with the product managers, the data scientists, and the product developers for every single feature release in making sure we were really asking ourselves what was the goal of the feature before we even started implementing it so that we would also be able to design what are the metrics that we will use to measure the success of that feature so that we will then

in the third section be able to actually make sure we have the data to measure those metrics and my sort of so you know when we started that process the we had maybe one or two developers in the company that opened charts at some points in their sort of software development life cycle opened up you know and asked some questions but after we sort of made this a fundamental part of the software development life cycle or the product development life cycle i would say like 70 of the developers of the company they started you know they started asking the questions and they would be posting on slack like whoa check this out we just saw like because they would be posting on Slack, like, whoa, check this out. We just saw,

because they would know when the product was released and they would be just waiting, waiting for amplitude data to trickle in or waiting for mixed model data to trickle in. And so they would just go and look up their charts because they knew the definition of the metric because we had defined it together. So they would just be able to point and click a few things

and then open up the chart and share it with a team and ask some follow-up questions. And this again, like this change bringing producers closer to consumers that really impacted the quality because all of a sudden the producers were consumers and they've fundamentally understood how painful it was themselves when the data didn't work. Yeah. And it actually ties a little bit into the other sort of the other less is more angle as well, which is, you know, if you think about data and from this perspective that you plan it as a part of your product release cycle, you realize that it's never a good strategy to decide to track

everything type of thing because you know just like when you're releasing products you just have to say no to things it's really important to focus because you won't be able to do everything well if you try to do everything you will be able to do a lot of things really badly and it's way better to measure a single thing really well and then have the opportunity to ask some follow-up questions rather than having you know bad answers to a bunch of questions love it i could talk about both of those points all day but i have to hand it over to costas because i've been monopolizing the conversation thank you yeah. Take it away. It's okay. I mean, you had an

amazing conversation so far. So I really enjoyed listening to what you were saying, both of you. Maybe we should arrange another recording where each one of us can impersonate the philosopher or something and have a discussion this way. Yeah, yeah, let's do that. I mean, let's make a recording with Geddel, Russell, Wittgenstein, like this kind. Yeah, yeah, let's do that. I mean, let's make a recording with Gerdel, Russell, Wittgenstein, like this kind of question. I think it would be great. I do have to say, hearing a mathematician and philosopher say that your gut instinct is really important, I thought that was really cool. All right. I'm officially done now. I love that. Yeah. Thank you, Eric. All right. I'm going to ask, I'd like actually to start, Steph, by giving us a bit of more

background around the company itself, AVO. I mean, we talked a lot about what happened early in your career and what made you to do the things you are doing today, but it would be great to learn a little bit more about the company itself. So can you share some more information about the company? Happy to, yes. Well, I've given sort of a backstory, but basically, you know, the tooling that we built at QuizUp, the tooling that me as head of data science

and the CTO who was at that point, sort of his background was in mobile development. His complaints were from the developers who hated how frustrating it was to, and time-consuming and like inefficient it was to implement analytics events and particularly how often it failed and how often I would have to go back to them and be like, excuse me, I'm sorry, you did this all wrong. And now I can't answer my questions. And that was just so frustrating. And it was frustrating for everyone, obviously, basically, nobody, nobody likes, I mean, they don't want to spend a lot of time on this,

but they also don't want to disappoint. Like nobody wants to do that. Nobody wants to deliver something that, you know, is then in the end just useless. And obviously the board and the CEO and the CFO, they wanted us to be able to answer questions. So we, you know, being two very passionate people, me and the CTO, we worked really closely together in building tools that automated a bunch of this stuff. We built a central source of documentation for the schemas, basically, that had event descriptions and the types of the properties and all that stuff.

And then we built code generators so that it would just be super easy for Android and iOS developers to just implement analytics events without having to think too much about it. All of the processes that we also built around this design part, the data design part, we called it purpose meetings at QuizUp, was this thing about setting goals and then deciding the metrics and then designing the events based on that. This originally was so that data scientists would be involved in this process from start to finish.

And we would propose the event structures based on the needs of the data consumers. In the end, though, as the data producers became closer to the data consumers, we just saw this really clear shift where, you know, it was just about aligning on goals and maybe deciding together on the metrics. But in the end, everyone just was self-serve designing the data. So the developers were designing the event structures based on the code base and the data, like how the data sort of best practices that we had in place, a quiz of the naming conventions that we had in place at QuizUp, the naming conventions that we had, all those things. So eventually, sort of what I would like to say is, you know, we transitioned from a centralized analyst, you know, I was the person that answered the questions,

to an attempt to have self-serve analytics, where, you know, we wanted everyone to answer questions, but that was super throttled by lack of data literacy and lack of data quality. And so as we built out these processes of ensuring data literacy even before the feature was out, and then the tools to ensure the data quality before the feature was out, we basically also built like, we built, we empowered our self-serve analytics by building the tools and processes for self-serve analytics governance. And that is something that then, you know, I started taking for granted. But fast forward a couple of years later, I, QuizUp had been acquired and I had started a company with a couple of friends. And it was only about five months into

that company where we actually shipped a product update that was based on incorrect data and it was just like a punch in my gut and I was like oh my god I'm back to back to like five years ago or whatever and I will never be able to work in digital products because it's always gonna suck and I had this realization moment where, oh my God, you know, all the tools that we built at QuizUp, what are we going to do without them? And, you know, we're trying to take this product to market and it doesn't make sense for us to spend time on building those tools again. I can't believe it. And that's what I started sort of syncing with some of my colleagues from the industry,

people from Spotify and Twitch and Airbnb. Turn turned out that we were super early at QuizUp in building these tools. We were building them in 2014, while like Spotify built a compatible thing in 2016. And I was like, what? I can't believe that. That's super cool. And that was sort of a trigger for us to really build Avvo and start sort of bringing these solutions to the general business, the general digital product. And so that's the backstory. But what Avvo does today is Avvo is analytics governance as a service. We are here to pay down the analytics debt of Silicon Valley, which is like, there's a lot of debt going on. And we're here to really optimize this release process for the analytics workflow. And we're bringing data producers and data consumers together with our product managers and developers and data scientists collaborate to plan and to validate and to implement their analytics for every single feature release.

And it fits entirely with the sort of the Git product release workflow. the pioneers for building a branch workflow for your tracking plan, which typically a lot of schema management tools have had versions, semantic versionings for your tracking plans or for your schemas. But that's something that's scary if you're not a data engineer, basically. And one of our earliest learnings was we really need to support the data consumers well, and they're not necessarily data engineers. They can be product managers that effectively only understand what their goal is with the feature, but don't have a deep understanding in event structures and good event structures or semantic versioning. And so this is our mission, analytics governance as a service, bringing self-serve analytics governance to empower really that shift to self-serve analytics,

which is a fundamental piece for companies to succeed today, I would say. Absolutely, I totally agree with that. Can you also tell us a little bit about, I mean, when the company was started, when you launched the first version of the product, like your experience with the early traction, your experience with fundraising in this space? I think that would be amazing for people to hear about. Yes, absolutely.

I'm happy to share a little bit about that. We are very customer-centric as a group of people. So we care deeply about our customers. Like I was talking about earlier, we have strong opinions in what we're doing. However, it's really important for us to confirm that by talking to our customers and looking at the data for how they succeed. And that's how we have built the product from the beginning. We have had customers from effectively day one when we started building our prototypes. We also got into YC. So we are a Y Combinator company. And that was because Gustav Ahlströmer, who used to, or was, I think, one of the early growth persons, I guess, in Silicon Valley.

He was a growth lead or a growth product manager or product manager for growth. There are all of these variations of the growth titles at Airbnb. So he, you know, did like was a fundamental piece in the growth strategy for Airbnb. And he experienced this problem immensely. So when we did our Y Combinator interview, he was like, I've had that problem. And so we did Y Combinator, which was, of course, an amazing adventure. And following that, we raised funding around in Silicon Valley. Before that, we had also taken in some funding from

Icelandic venture capitalists, but also angels that sort of supported us also because we're, you know, it was a team that had a lot of, I guess, I'm lucky to be have been working with a great team in QuizUp, obviously. And so that was a reputation that we had built also in, in, in Iceland. And so even though this was a very pioneering idea at the time, and nobody was talking about this, when we started doing this,

nobody was talking about it. And so, but they sort of, you know, they, they, they funded the team, you know,

they, they, they believe that we were a team that would be able to ship stuff and build stuff. And so that's exciting because it turns out they were right. Yay. And then after Y Combinator, we raised a round also from a company or a fund called Heavybit. They are an accelerator and a fund for developer tools. They were founded by James Lindenbaum,

among other fantastic people, who was the founder of Heroku. And James's perspective on developer tooling has really helped a lot of fantastic developer tools go to market. For example, CircleCI, Stripe, just endless, go check them out, heavybit.com. Fantastic group of people that have been really helpful. And then GGB Capital, fantastic partners there as well.

They've funded giants as well, such as Slack. And our partner there, Glenn Solomon and Oren Younger, they've just been immensely helpful as well, and the entire GGB Capital team as well. And I think they're also very passionate about both developer tooling and the data space. And so that was sort of their drive. They also work pretty closely with the Heavybit team often. So I guess that was sort of a hand-in-hand type of thing.

And we're lucky to be working with these world-class backers of developer tooling and in the data space that have huge experiences in both developer adoption and enterprise sales. So I have them in my inbox all the time and I'm able to ask them for advice. So I'm really fortunate and lucky about that. Yeah, that's amazing. And I understand how you feel about feeling like fortunate, but I'm pretty sure that all these people got involved also because of the team and the passion that you have about solving the problem. So the reason that they are around is also because of you. I mean, you personally and the rest of the team. So congrats for that. That says a lot about the quality of the team, the company and the product.

Thank you for that. Those are kind words. Thank you. All right. So moving forward, let's discuss a little bit more about the product. So our audience can learn what AVO is about and how it works. And let's start by what's the value that someone gets from Avvo today? I know you have touched this previously in the questions that both Eric and me asked.

But yeah, let's try to make it a little bit more concrete about the value proposition that Avvo has as a product. Yeah, thank you. So ultimately, we see a lot of our customers talk about how much developer time they save. Obviously, they save a lot of data quality or like just they rescue their data quality and they start paying down their analytics steps, I would say. And that's the path for, for example, companies like Rappi, which is a huge Latin American company that is like a DoorDash and a Lyft slash Uber slash Instacart. All of those combined, basically just delivering whatever you need as a service. And obviously that exploded in sort of COVID and March, April.

And that is effectively sort of around the time when they reached out to us. And they had been trying to audit their data for months. They had been working on that. The BI team had been trying to sort of catch up on even just understanding what was currently wrong with their analytics before they wanted then even also to try to fix it. At the same time, the developers were just tearing their hair out because of the inefficiencies of implementing these analytics. So their journey was they installed Inspector,

which is our sort of data observability product. You install the Inspector SDK to inspect and observe your current state of tracking ongoing. And that's how you get sort of your, you get a snapshot of your analytics depth there and are able to fully prioritize what is wrong with it today

so that you can, so you can prioritize what to fix, like prioritize the top things that you really need to work, like three, four, five events that you really want to make sure are okay. And then you can just like gradually step by step pay down your analytics adapt. So that that's their first step. And then they team by team adopt the AVO workflow, as we call it. So the AVO workflow involves all

of these stakeholders that I've talked about. The product manager comes into Avvo, opens up a branch to plan the analytics alongside a feature release. The data scientist and the engineer, they sort of give it a review, request some changes potentially. The engineer from the perspective of like, yes, I can implement this analytics event in this format. And the data scientist brings in sort of like the bird's eye overview

of the company analytics and is like, yeah, these analytics structures make sense in comparison with the rest of the company. Great, let's move on. And once that's been approved, that's the third step where the developer pulls generated code. We generate type safe analytics functions. We call them functions where you can implement using, you know, avo dot, you know, item added to cart or whatever, instead of calling a generic analytics SDK, where you have to pass in a string for the

name of the event, and then a huge object blob with like a name of properties and a bunch of property values. Instead of doing all of that, you just call a type safe our function that you generated using your avo CLI, based on the tracking plan, which is all defined in a user friendly UI by the PM and the data person. And that's the third step. And then the fourth step is, you know, the developer, you know, treats the build just like they would for any other build. They just go through their product before they release the build, before they sort of release the feature update. And then they use the in-app Avvo debuggers, where you have like a bubble that shows you

when you're triggering the R analytics event. So you can confirm the timing is consistent across all of the platforms. You can confirm the data structures are exactly as you intended for them to be. The PMs and the analysts also love the NF debuggers and they use them even, you know, outside of the regular feature release process because they can really get a full understanding of like what, what really is being tracked in the product. And so after that

sort of is is sort of quickly checked off the branch basically gets merged the ava branch that happens just at the same time as you merge your git branch and your tracking plan update get out gets automatically published into whichever other source you need schemas in. So that could be your downstream, you know, your BigQuery table somewhere or your Redshift table or your protofile or your amplitude govern or your segment protocols or your mixed panel lexicon, just wherever you might need to manage your schema, you can then automatically publish that after that branch has been sort of finally reviewed and implemented and just ready for production. And that's when, yeah, you know, then just everything is released. And like this, this entire cycle, the workflow,

it's shortening the time that people spend on this by, you know, incredible amounts. So for example, Patreon, they reported, basically they are getting 90% of their time back that they used to spend on this for every single feature release. They used to spend one to four days on this back and forth and like a QA process and syncing between iOS and web or whatever and data. And now they consistently use maybe like 30 to 60 minutes on it. And so that's obviously a huge time saving. Yeah, that's amazing.

Actually, something that I want to add in what you said, Steph, is that my feeling based on my experience so far is that the most successful B2B products, they deliver innovation in two directions at the same time. One is, of course, like the technology and the product itself and how it solves technical problem, for example, in this case. But the other thing which is equally important, and not everyone is doing it, is that it's organizational innovation, as I would say, which means that it helps the company to reorganize

around more efficient processes or automating processes. And I think that when you combine these two together, you have at the end like an amazing both value proposition for the company and the product, but also you have a building an amazing mode for your company. I'd love to chat more about this. I mean, it's something that I'm a bit obsessed to be honest, and something that I'm always trying to figure out. And there are some companies that they do that, like they did that, like Slack did that.

Looker did that. That's exactly what the LookML plus the visualization layer was doing and creating like a very clear and efficient distinction between the people who do the modeling of the data and the people who are consuming the data, like exactly what you're saying about the data producers and consumers. But anyway, that's a very big topic that we can keep discussing. We'll bring it up in our Wittgenstein. Absolutely.

Absolutely. We can do that. So, all right, moving forward, let's talk a little bit about the focus around the data that you have. I mean, like a company generates a lot of different types of data. I mean, from logs that might be consumed, for security reasons, to clickstream data that are actually described like they are like breadcrumbs let's say of the customer behavior it looks like you are focusing more on the second case like you you are mainly working with product analytics and the clickstream data that customers are generating do you want to expand a little bit more on that like why you think that's more important

if it is more important and what are your plans general? Like as you move forward with the product, are you planning, for example, like to support something else? Yeah, these are really good questions. So I think the first part here is the why. Like why are we focusing on customer data? Why are we focusing on product analytics? And from my perspective,

I guess like obviously what impacted is I joined as an analyst applying for a role to report to the CFO. The CFO obviously thinks about finance and cares about finance. And that's really important to them. But at the same time, this was, you know, 2013, when, you know, companies like Zynga, Spotify, Facebook, these companies had just taken over the, they were the digital products, like, right, they were the companies that sort of showed us that we can have digital experiences, solving things that we used to solve in person, right? Games, listening to music, Airbnb, another example, where we used to call and book hotels, But all of a sudden, we had these really wonderful online experiences through going through a product, going through an experience, and just booking a place to stay. And this shift was really going straight. I mean, it started like 2005, 6, 7, when we started seeing all of the apps happening and like just more of the things that we needed to do as people, as humans, just it became more and more mainstream for us to be able to do that online as products, either on web or mobile. Obviously, like software development has been around for a while and we had the dot com boom.

But this sort of shift really started happening at that point in time, I would say. We were going into digital products, digital experiences. So even while I was reporting to the CFO, their biggest problem wasn't financial data. Their biggest problem was really impacting, like understanding and being able to report to the board of directors, the leading indicators for the revenue. Because revenue, I think most people have sort of agreed

to in this space right now already, revenue is a lagging indicator. It's not a leading indicator for your success. While your customer experience can act as a really important leading indicator for your key performing indicator, which is typically revenue or some sort of other way of measuring that you are bringing value to your customers and that your business will continue to be able to thrive and sustain itself. And so, I mean, that's obviously my background into this and why, you know, I thought that this was one of the most difficult things to report on was this experience analytics. So what we focus on at Avvo is we basically focus on event-based data. It's typically event data

that describe user experiences, because those are the things that really sort of are the leading indicators for companies' successes today, for digital product successes today. And I would say today, if you're a company and you don't have a strategy for your digital presence or for your digital product it's likely that you won't survive and so I think this is just a fundamental piece of business strategies for right now already and we'll see like the laggards just falling behind the train in the coming years if if they won't catch up and keep up so that's really one of the really main drivers and just also to take an experience like an example you know one of our customers like patreon they've been with us for a long time they

were one of the sort of one of the earliest sort of like one of the earliest larger organizations to take a back pat on us and we have a fantastic relationship with them they're fantastic mora and jason and everyone over there at patreon it's been fantastic to work with them and they know so many things about this space um and one of the things that mora has already shared with me she talked so much about this like obviously revenue matters to them but the performance of the user experience in the customer journey that also matters a lot to the financial teams and they already use that in their modeling to be able to sort of have some leading indicators for their

success so and then if i tie this to like how do we see this evolve for Alvo I mean there's no limit to what we can support customers in you know fully structuring their event-based data however I think the biggest challenge and touching it like tying it back to what you were saying in like change management and organizational shifts and how people build their organizations and their their sort of focus i think that is really around like the customer experience so that's a deep passion for me and in helping people sort of do that better and so i think while we haven't fully solved that problem we're going to focus on fully solving that problem i would say that's that's some great points, Steph. And yeah, I mean, I think we are going through like a major cultural change inside companies

when it comes like to how data and KPIs and all that stuff are perceived. And I think it's going to be super interesting to see how these things will change in the next couple of years. One last question from my side, then I'll hand it to Eric. You mentioned at the beginning that one of the lessons learned for you was that when it comes to data, less is more at the end.

Can you expand a little bit more on that? Because I think it's going to be super interesting. Yes. So, again, I think like what I mean by that. So I touched on it a little bit before with like when I was talking about, you know, how important it is to bring data producers and data consumers closer together and sort of bring this whole process really into the product development lifecycle. What the mistakes that I've seen people do all over, like again and again and again, is when they decide to track everything quote unquote and you

know this mandate can come from like it can come from the developer that is just building the product and they have a mandate to add some analytics for it and so they're like yeah let's just track everything it can also be something from like a data person or or or a cfo type of person that really just doesn't understand or or isn't haven't they haven't just taken the time to think through what are the fundamental steps in the journey that we really need to understand the friction point for so they give this mandate to track everything and this is just the worst track that i that i see people go through again because of the same reasons why we can't just build everything, right? So building everything, building good products means focus. It means

saying no to things. It's difficult to say no to things, but we really need to do it because if we don't, we will either never ship anything because it just takes so long and we'll never have anything ready. Or we will just do a lot of things really badly, right? They just won't work and they won't have a delightful experience. And you know what it is today in building product. It's like delightful experiences are the fundamental piece in you succeeding and bringing your product to market. So the same thing really applies to data. Like if you just try to track everything, you won't have time to do it. And you just won't, you won't, there's no such thing as tracking everything. And instead, what you'll end up with is like, you'll track random, the random things that you were able to squeeze in before you had to press the release

button. And that's really bad, then you definitely, or it's highly unlikely that you have the most important things tracked if you go that route. It's really highly unlikely. And obviously here, I'm talking about sort of custom tracking. I'm talking about manually adding events. And that brings me a little bit into the subject of like the auto tracking. And I know Rudderstack supports auto tracking. We have some fantastic products that do a lot of auto tracking like Pendo and Heap and Rudderstack and Full Story, a bunch of auto tracking products. The challenge behind this, like I often put, like I often sometimes, sometimes at least I recommend to like PMs or, or, or people who just don't have time yet to think about their data. I'm like, yeah, sure. Go ahead.

Install this. It might give you some insights and it's like a safety net. I understand that you'd rather have something than having nothing. However, it really just postpones the problem of really thinking through what is the information that you need to answer. It postpone it so that you will have to be digging through a lot of noisy data after the fact. And it's really difficult for non-experts or even just like non-developers to understand what data they are looking at, because typically the data points are sort of like they're named after some IDs that the autottracker found through the DOM. And it's a lot of work to figure out what this means. And even if you do, it's not even guaranteed that you have the tracking that you need because good analytics events,

they typically have event properties and segmentations in place that you won't have automatically. And that's that thing like designing good data structures. And so even if you go through the auto tracking process and you have your safety net, it's just, it's going to be difficult to sift through it. It's like looking for a needle in a haystack. Certainly not the way to go if you want to build yourself your analytics and sort of increase the company's trust in data and sort of their trust in themselves to be able to go into a point and click UI and just like figure out what they want to answer. And

also it's not even a guarantee. It's not even a safety net. It's not even a guarantee that you will have the information that you need. So these are my two hot takes, like tracking everything, the tracking everything perspective. So, and because this is really difficult to do, like tracking a lot and tracking everything is really difficult to do. My general recommendation is just start small. Like if you haven't started adding tracking yet, choose three events and stop there. Like just absolutely stop there. And I know you want to track everything, but absolutely stop there and i know you want to

track everything but just stop there because it's way better to to ship something and ship a small slice and be able to start building out that sort of like oh interesting oh that was fun it was fun how we were able to make a decision based on that rather than just like turning like tracking into accidentally for two sprints you didn't ship any tracking because you just decided to track too much and you didn't really have time to finish it so that that's a little bit on the sort of less is more perspective I have way many way way more thoughts on that as well but I think that sort of captures a couple of the essences of that, I think. Well, unfortunately, I think we're close to time. But also, I think we definitely wouldn't have enough time if I asked more questions.

But I have many, many more. But we can save all of that for our philosophy role-playing episode, our special philosophy role-playing episode, our special philosophy role-playing episode. I learned so much stuff. And I think the, I just think our listeners will really appreciate the lessons that you have brought us on this episode based on your vast experience. It's just really helpful. And I think there are lessons that apply to companies or data professionals that are just starting out and really large organizations. And I just always appreciate wisdom that time-tested wisdom that's sort of applicable to all of our listeners. So really, really appreciate you being on the show.

Sounds like you're doing amazing things with Avvo. So excited for you to continue there. And we'll schedule a time to have you back on the show in the next three or six months. Thank you. Thank you for having me. It was super fun. I can't wait for our role play. Yes. Okay, thank you all. Well, that was an absolutely fascinating conversation. I think one of the things that really stuck out to me was how balanced of a perspective that Steph has. And I think the statement that just really, really stuck with me was, she said this almost in passing, but she mentioned how important your sort of gut sense is in making decisions about a product. And just hearing about her background and how much work she has done in data and data

tooling and just the sort of the massive amount of understanding and capability she's developed, yet she has just seems to have such a mature intuition about about how to do things well and not necessarily just get lost in the data or put that above absolutely everything else so i just really appreciated the perspective what was what was your big takeaway costas well we should for sure need to do this role-playing with philosophers role-playing episode. That's going to be a lot of fun. Out of this, what I keep from our conversation with Steph is how important culture is when we are dealing with data. And as we said during our conversation with here, I think we are at the beginning of a tectonic move or change

what is happening inside companies in terms of how they perceive data and their value. With a very good example that Steph gave about how growth is perceived even from the CFOs and how even the CFO's perception

has changed and non-standard KPIs like retention and customer satisfaction is becoming like a leading KPI even for them. And yeah, I'm really looking forward to chat with her again in the next couple of months. Avvo is growing really fast and I'm pretty sure that like in a couple of months from now,

we will have even more to learn from her and her team. I agree. Well, thanks again for joining us on the Data Stack Show. Be sure to subscribe on your favorite podcast tool to get notified of new episodes every week. We have some really, really exciting shows coming up in the next few weeks that you'll want to make sure to listen to. Until next time, that you'll want to make sure to listen to until next time.

Catch you later.