How I AI - “Vibe analysis”: How Faire’s data team uses AI to investigate conversion drops, analyze experiment results, and convert raw data into executive-ready insights
Episode Date: November 3, 2025Tim Trueman and Alexa Cerf from Faire’s data team demonstrate how AI tools are revolutionizing data analysis workflows. They show how data teams, product managers, and engineers can use tools like C...ursor, ChatGPT, and custom agents to investigate business metrics, analyze experiment results, and extract insights from user surveys—all while dramatically reducing the time and technical expertise required.What you’ll learn:1. How to use AI to investigate sudden drops in business metrics by searching documentation and codebases2. Techniques for creating a semantic layer that helps AI understand your business data3. How to build end-to-end analytics workflows using Cursor and Model Context Protocols (MCPs)4. Ways to automate experiment analysis and create standardized reports5. How AI can help design and analyze customer surveys6. Strategies for creating executive-ready documents from raw data analysis7. Why every team member should have access to code repositories—not just engineers—Brought to you by:Zapier—The most connected AI orchestration platformBrex—The intelligent finance platform built for founders—Where to find Tim Trueman:LinkedIn: https://www.linkedin.com/in/tim-trueman-99788592/—Where to find Alexa Cerf:LinkedIn: https://www.linkedin.com/in/alexandra-cerf/—Where to find Claire Vo:ChatPRD: https://www.chatprd.ai/Website: https://clairevo.com/LinkedIn: https://www.linkedin.com/in/clairevo/X: https://x.com/clairevo—In this episode, we cover:(00:00) Introduction to Tim and Alexa from Faire(02:53) The challenge of analyzing product quality and usage(04:14) Breaking down what analytics actually involves beyond data manipulation(05:46) Demo: Investigating a conversion rate drop using enterprise AI search(09:05) Using ChatGPT Deep Research to analyze code changes(12:40) Leveraging Cursor as the ultimate context engine for code analysis(18:55) Analyzing a new product feature’s performance with Cursor(26:27) How semantic layers make AI tools more effective for data analysis(30:00) Using Model Context Protocols (MCPs) to connect AI with data tools(34:17) Creating visualizations and dashboards with Mode integration(37:04) Generating structured analysis documents with Notion integration(44:39) Building custom agents to automate experiment result documentation(53:10) Designing and analyzing customer surveys(59:40) Lightning round and final thoughts—Tools referenced:• Cursor: https://cursor.com/• ChatGPT: https://chat.openai.com/• Notion: https://www.notion.so/• Snowflake: https://www.snowflake.com/• Mode: https://mode.com• Qualtrics: https://www.qualtrics.com/• GitHub: https://github.com/—Other references:• Model Context Protocol (MCP): https://www.anthropic.com/news/model-context-protocol• Faire Careers: https://www.faire.com/careers—Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.
Transcript
Discussion (0)
How do we start at the very beginning of analyzing a product and its quality and its usage through analyzing conversion rates?
The new AI tools have just absolutely transformed the process of just getting all that context.
You can go as broad as you like, self-serve, into an unfamiliar topic just incredibly quickly.
And that means you can not only deliver quicker analysis, you can just deliver a much better analysis to.
I'm going to start just by doing an enterprise AI search.
So I'm just going to start very simply by asking a notion, what experiments when new features launched between September.
to December, 24, that could have added friction to the checkout process for new retailers in Europe or North America.
And I've just said, focus on XP docs, PRDs, and launch announcements.
I've got straight away a really interesting list of hypotheses to dig into it.
No work.
You can see it's searched across Slack, Notion, Jira, and everything else very, very quickly.
So Alexa, how do we do actual analysis of data when we've identified a problem or an opportunity we want to go after?
Without AI, especially the context gathering, would mean hours spent digging through all the specs and PRDs, writing SQL queries from scratch, and then, you know, spending a lot of time writing and editing a dock using cursor to actually create, edit, write, sequel has been pretty game-changing.
Welcome back to How IAI. I'm Clairevow, product leader and AI obsessive, here on a mission to help you build better with these new tools.
Today, I have a great episode with Tim and Alexa from the data team at Faire.
They're going to show us how you can use cursor, MCPs, chatGBT, and even write your own agents to do data analysis.
We're going to see everything from decomposing that scary question, what went wrong in September, to doing detailed funnel analysis on experiments and surveys.
Let's get to it.
AI is supposed to make work easier, but I've been there.
Weeks of setup, endless back and forth with engineering, and yet another tool the team never really adopts.
That's why I use Zapier's AI orchestration platform.
It connects with nearly 8,000 apps, so I can finally put AI to work without the drama, without
the delays, and without pulling engineering in every time I want to automate something.
With Zapier, you can roll out AI-powered workflows that do real work across your whole company
in days, not weeks.
I use Zapier every single day.
It automatically responds to leads with enrich personalized data.
It checks my calendar weekly and offers smarter ways to manage my time, and it even drafts
emails for every new request that lands in my inbox. All of that running quietly in the background,
so I can focus on the work that matters. And Zapier's built for scale. With enterprise-grade security,
compliance, and governance, it's trusted by teams at Dropbox, Airbnb, Open Door, and thousands more.
Go to try.zapier.com slash how IAI to learn more about how Zapier can bring the power of AI
orchestration to your entire org.
Alexa, Tim, thank you for joining How IAI.
Great to be here.
Thanks having us.
Thank you so much.
One of the things that we can do now that I am probably personally causing in the internet
world is we can just build a lot of, a lot of product.
I am always out there like, I was thinking the other day, I was like, I'm going to tweet
something where I tell PMs that they should just spend a month saying yes instead of saying no.
Like, let's ship some features.
And I think AI has really accelerated product development, software engineering, getting
innovation to the hands of customers.
But the problem it is created is we don't know if those products are any good.
So the perennial product problem, which is you can ship things and they can not make the
difference that you hope they would make.
And so I'm really excited about this conversation because you are going to show us how to use
AI and even some of these tools that software engineers are.
product managers might be familiar with to do really deep, meaningful product analysis.
And I spent a lot of time in experimentation. And so I love a good conversion rate optimization.
So Tim, we're going to kick it to you to start with. How do we start at the very beginning
of analyzing kind of a product and its quality and its usage through analyzing conversion rates?
Yeah, I love this. I think everyone's talking about vibe coding, but no one's really talking about
vibe analysis. And we're heading in that direction very quickly.
so let's get into it. So before we do anything to technical, I think we want to share a really broad
range of examples here, from the really complicated to the like actually incredibly simple.
I think everyone knows PMs are going to have to become engineers. And then we've got a lot of
issues where all of you guys are going to have to come and our analysts as well. So I think
there's a lot we can show here. So we want to start off with just a really simple use case
that should be familiar to, I think, everyone listening. But I think it illustrates the point.
There's often the most simple AI tools that can actually have the biggest impact here.
I think before we get into the actual demo, I think it's useful just to pause very quickly for a second on the question of what analytics actually is.
Because I think once you break that down, you get a much clearer view of where these current tools can be most valuable.
I think most people jump straight to the nuts and bolts from actually manipulating and crunching data.
But actually, it's really just a small part of the O4 process.
And the most important, often most difficult thing is actually just getting the right context in the first place.
because that's what separates good analysis from bad.
You need to know it to ask the right questions,
to come up with the right hypotheses,
to know what analysis is even worth doing in the first place.
You need to know where the data lives
and you need to be able to interpret it all very well.
And the new AI tools have just absolutely transformed the process
of just getting all that context.
You can go as broad as you like self-serve
into an unfamiliar topic just incredibly quickly.
And that means you can not only deliver quicker analysis,
you can just deliver a much better analysis too.
So to illustrate the point, I want to talk through what, sadly, I'm guessing, is a very familiar
situation where a business metric suddenly drops off a cliff and no one's got a clue what to do with it.
So I'm going to use a real example from Fair for this.
And this happened to our new customer conversion funnel at the end of last year.
So if you've ever worked in growth, everyone's going to know new customers.
They're just extremely sensitive to even the tiniest little friction.
So almost anything anyone does in the business,
anywhere can affect these kind of things, whether it's a sign up flow, a search algorithm,
shipping policy, like this all can affect these things. And if you're not careful, you're going
to have to decomp the entire business. So let me show you how these things can just be done so
much quicker. So imagine this problem lands on my desk. I might look at a couple of just
existing dashboards that exist to say what's going on here. And you can see very quickly
the issues started in September and there was another drop in December. And it seems to be
concentrated in the checkout stage. But beyond that, I've really got no idea what could have actually
caused that. So let's start really bored. I'm just going to share my screen. I'm going to start
just by doing an enterprise AI search. And we use Notion, but frankly, every document system now is
going to have an AI system. If they haven't gone yet, it's coming and they are just game changes.
So I'm just going to start very simply by asking Notion what happened. Okay? So the only thing I'm
going to do, I'm going to just make this more realistic. I'm going to filter the day range.
I don't want it cheating and looking at the answer.
It's only going to have access to the things I had access to when I actually did this.
So I'm going to put it up to the end of April last year.
It was fine right now.
And then we're just going to get that running.
So if you think this, all I've asked is what experiments or new features launched
between September to December, 2024 that could have added friction to the checkout process
for new retailers in Europe or North America?
And I've just said, focus on XB docs, PRDs and Nord announcements.
Okay?
So if you think about what I'd have done in the past, I'd have to be crawling through a million documents, doing a load of searches, going through a ton of different Slack channels, trying to work out what's going on.
And instead, look, I've got straight away a really interesting list of hypotheses to dig into it.
And you can see it's searched across Slack, Notion, Jira, and everything else very, very quickly.
And if you let's just pull out a couple of these.
So what's happening?
So let's go.
So you've got, clearly we launched some kind of challenge.
checkout experiment around this time. That's definitely worth looking in. We've done something with a
checkout blocker in Europe. Okay, lots of interesting things to dig into. Now, with a couple of clicks,
I've got a good long list, but I don't really know what these things are. So I've got all the
links of extra documents I could go click into. But let's just ask as a starting point,
what is Eori? Let's pick a one of them. What is Eori? So we'll just ask that. It's going to run
another little search, give us more things. Now, you've got a little bit here, but it's going to
for bring up a little bit more information, just get a bit more, a bit more detail on this thing.
So let's see where that goes. Okay, so very quickly, it's saying, give me the term of what it is.
And you can kind of see it's, okay, it's a regulation that's involved Europe and someone's done
something to start asking for more details, clearly trying to improve checkout conversion rates,
and they're trying to bring that one in. But I think this is a great starting point. I've got some detail.
But I think what's really interesting here is everyone knows, like a POD is one part of the story.
But between a PRD being written and something going into the code base, a lot can happen.
So if you actually understand what's going on, you usually need to go one layer data into the actual technical implementation.
And I want to show you a quick trick of how I do that.
So I think one of the best things about these AI tools is just the ability of someone who's like non-technical to access things that they couldn't previously access.
And a great example of that is just being able to talk to the product codebase.
I'm not an engineer. I can't like Kotlin or Swift.
I used to be a lawyer for God's sake.
Instead, I can run a deep research against our code base to find out exactly what got implemented
for our particular feature and when.
Now, I'm going to do this in two different ways.
I'm going to do it on Chat 2BT, which I think is very simple and anyone can replicate
incredibly quickly. Everyone's familiar with it.
And I'm going to do it on cursor, which is a bit more specialized but just incredibly powerful.
So I'm going to open up a new chat and I'm going to put it into deep research mode and make sure my GitHub is connected.
So we do. It's not technical to do that. You just need to say yes a few times to get your GitHub connected.
The only reason you have done on deep research is just because it's the only way you can actually access it.
It's going to search our code base now in exactly the same way.
It would normally search the web on a deep research.
So I'm just going to put in a prompt.
Let's just copy that in.
Now, let me talk a little bit about what this prompt is doing.
I've given it a role. I've said you're a senior staff engineer and you've got expertise in all these different
co-based as Cotlin Swift type script and you're working at fair. And I've given it a task to say,
please conduct a forensic investigation of the co-base to produce a comprehensive time sequence report
of all changes to the Yorri collection process at checkout between June 24 and February 25. So just making
sure we don't miss anything. And the rest is just a bit of detail as to what I want this to look like.
So I've said, I want an exact sum.
I want a table with all the different PRs and commits, what they've gone into.
And I really want it to focus in on the actual impact these commits had on the retailer experience.
Like, explain it to me in layman's terms.
And then I've just put a few requirements in here just to give it a bit more context.
So be precise, simple clear language, only use GitHub sources.
I want to call out here.
You're using this prompt in the context of sort of what I would call like a business.
incident, right? New user signups just drop. But this is a prompt that I want the engineers
watching or listening to the podcast to really pay attention to because if you're in the middle
of a, you know, Sev 1 incident and you need to trace who did what. I know so many of our
engineering teams are looking either manually looking through code, looking at these specialized kind
of code gen tools to do this, but probably aren't reaching for something like chat GPT,
deep research to just go ahead and do this for you. And if you're a product manager,
looking to be helpful during an incident, this is maybe a task you can take on on behalf of your
engineering team just to provide some additional contexts in the background.
100%. I mean, this is great for engineers. I think it's great for just getting people to talk
better to engineers. I think there's just so much you can do here. So as always, deep research is asking
for you questions. So use discretion. We'll just answer a few of those to make sure we're on it.
Use discretion. And yes, so that'll get it going. But now you prompt just like I do. I just say you
pick, you decide, you go, I don't care. I think the fact that the pro doesn't ask you these questions
make me think it's more to make you feel like it's doing it rather than anything else. So that's going
to take a bit of time. So while that's running, I want to show you how to do this in cursory.
Because I think cursor is one of those tools that everyone thinks of for vibe coders, they think of it
for engineers, they're not really thinking about what else it can do. And I think for both
analysts and non-analysts alike, it's an incredible tool. So I think more more people are talking about
phrase context engineering well and content engineering. I love that. It sort of actually explains
what we're trying to do here. And for me, just cursor is the ultimate context engine. You can hook it up
to MCPs. So basically, I can hook it up to every single system in our business to get all the
data I need. And that just makes it such an incredibly good accelerator for getting context and doing
analysis. So I actually find increasingly this is getting better results than deep research on
TPD. So both are good. Both are game changes. But I think this is just a little bit quick.
quicker and better. So I'm just going to make sure my MCPs are all hooked up. And then all I'm going to do is I'm going to drop exactly the same prompt into cursor. And we'll see the two running. So exactly the same prompt. So just for context, we are not even started on our, it hasn't even got off to the races at all on the chat of tea. And straight away in cursor, we're going and finding it's got a nice to do list. It's saying it's going to search all the right things in GitHub. It's going to then forensically analysing.
and we'll just let this run for a little bit.
You can see it's already starting to pull in the code and the pull request that we won.
One of the things that I think is interesting to call out is, you know, I've run a lot of product
engineering data orgs before.
Engineering, certainly, day one, what are you doing?
You're getting access to all the repos.
You're getting set up with GitHub.
You're pulling your local environment together.
I know that data teams often have a similar onboarding because they're working so closely with
production data.
One of the things I think is going to change, or if it hasn't already should change,
now is I think product managers and designer onboarding first seven days has to include access,
at least read access to GitHub, getting your local repository pulled down, getting all your
MCPs set up because it just code has become now a data source for anybody doing work,
not just people writing code. So I look at this and I think leaders out there need to pay
attention and rethink basically their onboarding process because you don't want to be in a
situation like this and go like, can something give me GitHub? Like, can I, can I get access?
You go, you go beyond that. Like, everyone should have access to every system. And it should be
from day one. These tools are just the best onboarding accelerators. We've seen it for analysts.
We've seen it for engineers. Suddenly people get the contact very quickly. Okay, so it's already
is summarized everything. It's written some natural. We're actually starting to write things
out here. So straight away, you can see I've got a nice summary. It's given a few things.
But this is what I was most interested in. So I'm getting a table here for those that can't see my
screen. I'm getting a table with every single.
PR that affected this part of the flow from like it starts in July 24 all the way to
it's still going but it'll probably go to somewhere like December or February if anyone's going
to go with all of these things now let's just call out what this is doing so it's given me
an exact link to the specific PR that actually pushed this into the code base it's giving me
the name of it and it's giving me a summary of what it did it's saying who was affected
and it's saying what was the impact on a retail experience?
Now, if anyone's done this kind of thing,
it's so difficult to do
and actually pick through all the codes
and actually understand what's going on on this.
And it can just be incredibly quickly.
And so very quickly, knowing nothing about this feature,
I can already start to get really smart on what happened.
And I can see if I dive down here, yeah,
you can see there was an experiment launched in mid-September
right in the sweet spot of when this drop first happened.
And if I scroll through, getting through to looking at December,
yeah you can see it launched all treatment to all users that much bandwidth right now so this now looks like a really interesting potentially smoking gun that we can deep button into and so instead of spending days talking to people about all the potential hypotheses I can now speak to exactly the right colleagues and have a really targeted conversation and informed conversation right from the off with them to crunch through this problem in a matter of like hours rather than weeks here so even before we've done any data crunching this can just be absolutely game-churching
changing for us. Yeah, and it allows you to go a lot deeper than, you know, I've been able to do
historically on these kinds of analyses. You know, when you're running these high velocity
experimentation programs, you have so many concurrent experiments, you have experiments, colliding
with rollouts, colliding with just plain launches, and just try to decompose what was the state
of your app on any single day is really challenging. And even if you can do the manual
research to get this at a feature level, like, yeah, today we launched the one, one page
check out. I think the real challenge is, well, did we implement it well? Is there anything in there
that we should like worry about? Did we exclude any users from that? And so I do think the ability
to use code as a detailed source of truth when doing these kinds of forensic analyses really
makes the difference in figuring out what's going on with your business. And they're getting
smart enough to go one level deep as well. You can ask follow up questions to say, how did it differ
for different segments? Are there other ones interested? Like you can get so much detail just by asking questions.
on these kind of things without speaking to any engineers.
And this gives me a little bit of some inspiration on other use cases for querying your codebase
in GitHub history for events.
One of the things that I do very frequently is I do a very similar analysis to this.
But I say, what is everything that shipped in the last week from the context of a customer?
And then I use it to write my newsletter.
So again, like, I'm starting to use our code base as a source of truth for our marketing materials.
I don't have to proxy through like what was in the PRU.
or what did a PM right or any of that stuff.
I'm just like, just tell me what was in the code in the code commits,
because that's what I know went live.
It can interpret what the customer facing experience and intention would be.
And then you can create these really interesting business and market facing assets out of that.
So I just think the ability to query your code base and your GitHub history for any use case,
including this one, is really useful.
Yeah, I love that.
Great.
Now, what do we do after this?
So you've identified, you have a conversion rate problem, you've identified maybe a couple
sources of the issue. You're going to go talk to your colleagues. You're going to look at the code.
How do we actually do some analysis? Or I know we said we were going to do some vibe analysis
and we have seen very few numbers. So Alexa, how do we do actual analysis of data when we've identified
a problem or an opportunity we want to go after? Yeah. So obviously like a quite classic analytics
task. I'm going to take us through, you know, we launched a new product feature and we actually
want to understand how it did. So I'll take us end to end from understanding how the feature was
built, analyzing its performance, and then producing a summary that could eventually go to our
exec team. Like Tim kind of touched on without AI, especially the context gathering would mean
hours spent digging through all the specs and PRDs, writing SQL queries from scratch,
and then, you know, spending a lot of time writing and editing a dock. So with AI, I can pull
context similar to what Tim just did directly from the code base. I can generate queries and I can
draft a synthesized doc. And so I am going to start sharing my screen. And while you pull that up,
I have to say people think that why I got into AI in a deep way was because I thought it was so
fun to code. And it was actually, it made my sequel so much less ugly than it used to be. It was like
my number one use case, however many years ago. I was like, thank God. Now I don't have to bother
my colleague with my disgusting sequel. I can bother AI with my horrifying sequel and it can make it
a little bit more efficient. Yeah, I mean, even just chat GPT for the last couple months has been a
game changer for SQL queries. The problem with chatybt is you had to spend a good amount of
time giving context, like the exact table names, the exact field names. And so using, I mean,
it's not its sort of most marketed use case, but using cursor, which is what I'm going to show
today, to actually create edit, write sequel has been pretty game-changing, especially because it's
so context-aware, and I will talk about that. So cursor can take like three to four minutes to run
some queries, so I'm going to just kick off this prompt, and then I'll explain the context and what
I have done. So while it's running, I will set the stage.
Last month in July, we redesigned the sign-up flow for a new payment method that we have been piloting.
And this process of sign-up is successful when a customer links their bank account for the payments.
And our old flow had been live for a few months.
We had a hypothesis that we could improve it.
So we redesigned the flow.
Because this is a pilot, we actually didn't have enough retailers or users to run an AB test.
So I just needed to do a pretty straightforward.
you know, how is this performing before? How is it performing after? Historically, again,
that would have meant a lot of digging through documentation or more realistically just pinging an
engineer to ask questions like, okay, what did we build, who sees it and why, what front-end events
are admitted that I can use to analyze this. And while I do work closely with our engineers during
the end spec phase, like figure this out, those details are easy to lose track of, especially.
like we're often coming back to analyze things, you know, weeks or even months after the feature
launched. I will say that I probably would start with Notion AI context building similar to Tim,
but we already showed that. So I'm skipping straight to the codebase. And if we go up to this
prompt, my prompts are way less pretty than Tim's. I don't like spend a lot of time on them. I feel like
with cursor you can always iterate. And so I wanted to understand the setup wizard, which is what we
called this new flow, I told it to research our code base, and I essentially asked who, what,
where, when, why. And so if we go to this answer, we can see, okay, it is, you know, looking
into the code base and, you know, I'm not an engineer. I don't really know what this means,
but it, you know, we called this in our code the first run user experience. And it tells me about
some flags, cannot be sub users. There's just like a lot of detail here. And it's telling me when
users see this flow, what happens during the flow, the order of steps that happen. That's like
pretty important. If I'm going to analyze a funnel, I need to know, like, in what order did
things happen? And then if there is a success event, like when the setup is complete. And then it
gives me a bunch of events that I can use to analyze it. So this is already such a game changer.
Like in the past, I would have leaned on secondhand sources like Notion to piece together how it was
built with cursor, like you were saying, I can go straight to the source and have it translated
into natural language. And that just gives me a lot more confidence because it reflects what's
actually live and not what someone remembered to write down. One thing I want to call out while you're
going to your next step is one of the steps that I see skipped by engineering teams is good event
tracking when they release a feature. Because, you know, you start up front in the PRD and you like
define a tracking plan. And then it gets to do.
implementation and people for again should be a front end event should be back in event.
And one of my favorite follow up AI tasks after something has been released or it's in code
review is I do a quick prompt and I go, is this is everything appropriately tracked in this
feature?
And I get either cursor or Devon to go in and put in all the right events and make sure that
the schemas are normalized.
So for all the data analysts out there, be annoying and do a PR for your own events on new
features.
So you're not, you know, stuck with what the engineer is built for you.
you. That inspires me to, I can take the end spec and just put it into any AI tool and say,
what front end events do I, or what events do I need to ask for to be able to measure the success
of this effectively. Because right now I'm just doing that in my head. That is not something that I
have, yeah, don't do it in your head. That's the, that's the, the title of how I AI. How I AI.
Yes. Don't do it in your head. So, with this next prompt, I, again, not the most like,
sophisticated prompt.
I'm just saying, I want to understand at a high level how this feature has been performing.
And I give the quick context of, you know, our goal is to make it better.
That's pretty obvious.
I just want to spell that out.
And I like Tim, I'm giving a fair amount of discretion to the cursor agent.
I'm saying, okay, come up with the ideal output fields.
I have some ideas, but like, you know, it's up to you.
And then two, I do find that telling it explicitly to create a
it sometimes forgets to do that and just writes the sequel directly in the conversation
sidebar.
Use the MCP connection.
Like I went through all this trouble to set it up.
I want it to use the Snowflake MCP connection.
And then actually QA the file.
And that's what's so powerful about this cursor agent and the Snowflake MCP is not only
it's writing the sequel, which is what ChatGBTBT has been doing for me for the last year.
It is running it, looking at the output, and then making like its own.
and sniff test sense check decisions, which is just so cool.
Okay.
And then another thing I want to call out as we are running this, the reason why I have
a fair amount of confidence that this is going to work relatively quickly is because I and
our data team have done a fair amount of work to create what's called a semantic layer.
And so first, our amazing data engineering team, like six months ago decided we were going to
create like a general company semantic layer. And a semantic layer is essentially just a translation
for an LLM of like our business terms, tables, fields, filters, metrics, et cetera. And AI can look at
those files and understand what our tables mean. This general one covered like our most used
generic tables, orders, items, users, et cetera. And so they connected it to a custom GPT and anyone in
the company can go ask pretty basic questions like, what?
was the average order size in Europe last year and get an answer really quickly. And so that's been a
huge unlock to save our analytics team time of like, we're not answering these questions for people.
They can self-serve. It's just democratizing data and, you know, saving us a lot of time so that we
can focus on more deep analysis. And for deeper analysis, like, we needed something more than just
these basic tables. And so I, with a lot of help from one of our data engineers, she said,
built a specialized semantic layer just for like my scope as a test.
I was, you know, I was the first one in the company to do this,
but we're planning on kind of rolling it out to all of the areas of scope.
And, you know, basically this semantic layer just defines the tables that I use the most,
the joins, the filters, the metrics.
And because it lives in our code base, it's like in our data science repo.
Curser can just tap into it and it just makes the zero shot ability like insane.
of running steeple.
I've seen a couple of these,
and I don't know what yours looks like,
but they really just look like
defined terms, tables.
This table means this.
This field means that if you're trying to query
average order value, this is how you do it.
And it's almost your documentation
in a little bit more of a structured form
around common queries.
And what I think is nice about this
is its ability to be managed by code.
You can change it.
You can update it.
You can add new things.
I also think for the data engineers out there, it reduces a little bit of needed complexity
on the data warehouse setup because previously you were creating these aggregate tables
and these like defined metrics and you're hoping people were writing queries the right way.
And now you can define these canonical queries and know that no matter kind of like what your
tables look like, they're going to get to the right answer, which I think is quite nice on the
data engineering side.
Yeah.
So this is an example of like what you were talking about.
It's just a very structured JSON file.
And from what I understand, I did not do this, but I had the engineer explain the process
to me.
And honestly, LLMs helped a lot with creating this.
He fed in details about our data warehouse and just a million queries that I had previously
written.
And it kind of helped spit out this type of thing.
He also used Langchain to like change the names of a bunch of the reports that we had into
question form.
Because obviously, when I'm queried.
this, whether it's through a custom GPT or cursor, I'm often asking a question. And so I thought
that was pretty cool. Like translating it to a question makes the semantic layer work so much better.
Oh, this is going to be my next project. This is so fun. Amazing. Glad to inspire. So to go back to
the actual sequel that was run, and I will actually just run this. Let's see, hopefully this.
And just in case people miss this, you did call out
the Snowflake MCP, which is what we're seeing right now, which is a programmatic way to hook
into running queries in your Snowflake data warehouse. So you can not only generate the SQL here,
but instead of like copying and pasting it and going into like Snowflake Cloud and running it
or whatever your visualization tool is, you can just run it right here. You're getting your tables right
here. So again, like you're eliminating that contact switching. You're eliminating the copy and paste
and you're getting your data right here. Yep, exactly. And so I am, oh, this is interesting. This
actually I am looking at this and it's I think it showed a mistake. But, you know, I asked it to
QA itself. Normally, this has done, does a very good job. But one of the quick QA's that I do
for something like this is I want to see no skip steps. Oh, actually, you know what? I remember
from the context, this is a temporary, this is a step that only some people see. But usually when
I'm looking through this, you know, in, if we were not doing, you know, if we were not doing,
this demo, I would spend probably a lot longer QAing this. But I just want to see drop off
that makes sense, right? Like, I don't want to see zero, zero and then one or then zero. And so that's
just a quick QA that I can do. You know, it's not the AI's name on this analysis. It's mine.
So I hope I do that. The other thing that I have done to really make sure that I can QA this
effectively is I, in my cursor rules, I tell it to comment every single CTE so that I
I know what the, and sorry, CTs are like sections of SQL that often are created when you're
writing SQL. And I just want to know each step of what is happening so that as I'm looking
at the sequel, I can say, okay, the agent said it's doing this and like looking at this code,
I can actually tell that it's doing this. So engineers cover your ears because engineers hate,
hate, hate, hate, hate when I say this, they hate it. I love overcommented AI code. And let me tell you
Why? Because when you are not writing this code, you really need to understand the thought process behind how the code was designed.
And having AI comment the code that it writes gives you a natural language way to understand if your understanding of the implementation matches the actual technical implementation of the code itself based on the AI's own reasoning.
Fine, delete it if you want to. I don't care. I know all the arguments against overcommented code.
and I think there's a lot of benefits for human review,
and it's also great context for AI when they go back and work on it.
So engineers, you can now uncover your ears.
You can yell at me on Twitter if you want to or an X if you want to,
but I do the same thing where I say go ahead and comment in the code
so I can understand how you decompose these step by step.
Yeah, it's pretty awesome.
I even have a custom GPT in chat, GBT, to comment code I've written before.
I just insert code and then, you know, if I'm ever,
handing off dashwords to someone, I really don't want anyone to be so confused that they have to
bother me. You know, my goal is to have it be quite self-serve. Look, those lines of code are not going to
expand themselves. Let's get some comically. This episode is brought to you by Brex. If you're
listening to this show, you already know AI is changing how we work in real, practical ways.
Brex is bringing that same power to finance. Brex is the intelligent finance platform built for
founders. With autonomous agents running in the background, your finance stack basically runs itself.
Cards are issues, expenses are filed, and fraud is stopped in real time without you having to
think about it. Add Brex's banking solution with a high-yield treasury account and you've got a system
that helps you spend smarter, move faster, and scale with confidence. One in three startups in the
US already runs on Brex. You can too at brex.com slash how I.A.I.I. So I'm going to take off my next
prompt, but basically like we're going to skip ahead a couple hours here because up until this
point, like my goal was to get this kind of clean base query that I could use for dashboards in
mode, which is Fares BI tool. You know, a lot of what we are doing as the strategy and
analytics team is creating, creating tables that then can be used for pretty charts to tell a story.
And so let's pretend that I spent a few hours with cursor, like refining queries.
I actually did one for the old flow and the new flow.
I actually did do this.
This is also a real use case like Tim's.
And then I built some visualizations in mode.
What's really cool is that there is actually a mode MCP and I can't.
tell it to view a dashboard directly for those who are listening.
Here we have on the old, on the left hand side, our legacy flow and on the right hand side,
our new flow.
You'll see that there's one step that is only present in some of the,
some of the entry points.
This is a split by entry point.
And basically it's just showing, you know, like what is the overall success rate
and success rate by step for each of these flows?
And so this is what I have pointed the mode MCP towards in this in this prompt.
So if we go back to this prompt, and I'm just going to tell it to run this tool.
Okay.
So I'm telling it, again, like, hey, go look at this mode dashboard and use this MCP.
I also give it the direct sequel that I wrote with cursor that's powering that dashboard.
I was asking it for some detailed takeaways and next steps.
I give it a little bit of context.
And I tell it to ask parifying questions and use the MCPs if necessary.
The MCPs, I think I'm not sure if we've defined it yet,
but model context protocol, I believe, is what it stands for,
are like so powerful.
I think that that's when this has felt like magic the most.
Like at first I assumed that they were similar to APIs
where everything needs to be defined.
Like some engineer on, you know, both sides needs to go to find endpoints that there's a very specific structure.
It seemed like a lot of work.
These models just like know what to do.
It's just wild to me.
I will say that there's a lot of work on our data engineering side to get some of these MCPs set up.
So I think Ben on our analytics platform team has just spent a lot of time on this.
Like I don't want to minimize that step.
But as the end user of them, it is like it just feels magical every time.
it can just access something.
And so if we go into the results over here,
next key takeaways and next steps.
Cool.
So we looks like we did a good job.
Yay, fair.
And it gives like a pretty detailed list of, you know,
the funnel analysis, insights and concerns, actionable next steps, et cetera.
Like this is already a pretty good sort of output to start with.
but at the end of the day,
like analysis like this only matters
if you can communicate it clearly, right?
Like you need to sort of convince people
of whatever you are trying to communicate.
So we also have a notion MCP
and I'm going to ask the cursor to create a doc
that captures our findings in a structured way.
And I want to pause really quickly
because we have done this in maybe 15 minutes
where you have taken a problem,
kind of like a pre-imposed analysis of a feature change.
You have written SQL,
have not used a Wizzywig analytics tool.
You have written straight up good sequel,
traceable sequel, to do a funnel analysis of that on a daily basis.
Very interesting.
You have made a dashboard for it so that your business users can use it.
You have then done a meta-analysis of that dashboard using the MCP
to actually read the dashboard, do a first-fast analysis,
create a summary not only of the results,
but have recommended next steps,
and then you are going to publish that to your business using Notion.
Now, I have to say,
I have worked with a lot of data teams.
And most of them spending their time saying,
what is the priority of this analysis?
We have a backlog.
I need data engineering.
And fine, here's the dashboard.
Like, it's like the ones that, like,
get promoted three times in a year.
That go the extra step where they're like,
and here's the analysis.
and here are my recommended next steps and I made it pretty so you can share it with your boss.
And I just think like, I was watching this and I was like, oh man, I'm going to promote this data
analyst. Like they're pretty, they're pretty good. And so I just think the ability to level up
the quality of your work and think through the interesting things. The interesting thing isn't like,
did I write this sequel join correctly? The interesting thing is like, have I thought through all the
edge cases? Do I have any creative ideas on what we could do next? Can we improve this analysis for
the future? And so I really like this.
end-to-end flow because it just shows how you are leveraging up into higher strategic tasks
as opposed to spending your time sort of in the tactics.
Yeah.
I mean, I totally agree.
And we are almost done.
But like you said, you know, we need to communicate this.
And so one thing that we have done on strategy and analytics is our chief strategy officer, Dan,
like he really cares about synthesized writing and all the leaders on his team care about
synthesized writing.
And so we worked with him a couple months ago to actually create some guidance on how to write at Fair.
Like, fair is very much a vertical doc culture, you know, pre-read culture.
We're not creating a lot of slides.
We are writing a lot of docs.
And so we have this sort of like use answer for structure key principles doc.
And then we also have a template for what docs should look like.
And so actually in this prompt, you'll see like, I tell it to,
to follow these rules that are in these docs.
And that's like another thing that I love about SQL,
or sorry, about cursor is you can just tell it what rules to follow in a variety of ways.
Okay, Alexa, I'm going to give you an upgrade here,
which is you should reference these files and your cursor rules so you don't always have the answer.
That's a great, I should.
I mean, I wanted to, you know, show the full flow.
But the reason I don't is because it would have actually done it in the previous step.
Oh, yeah.
Because it would have, it would have known, and then I wouldn't have gotten to talk about it.
But yes, I will.
I will do that once we are done.
It's showbiz, folks.
That's what this is.
And so the last thing is I am going to pull over the dock.
This is one that created from a previous time I did this just because I wanted to highlight in yellow.
I gave instructions in this prompt to tell me what to add.
I think one thing I want to get across is this, I don't think that cursor,
yet or AI can zero shot like an executive ready doc yet like there's that is where I think that
we still need to do three to four revs of um of sort of editing adding analysis making sure this makes
sense like these tools have so much context but we have we still have some context that is just
this like genus sequa like humans are still valuable and so this is like a pretty good start and I think
what's cool about cursor is like I cut out some of the middlemen. I got to this point like really,
really quickly. But we're not just creating like AI slop docs all over the place. We are,
you know, just accelerating how fast analysts can do things like this. We, you know, and the other
thing that's really helpful about, I would run this through that guidance three or four times.
It can be hard when you're been so in the weeds of an analysis to like take a step back and
make sure your story makes sense. And so that's,
that's what LLMs are really good for.
So it can cover my blind spots.
Well, you know what's more painful than running this three times through your guidance
is sitting three times with your SVP of strategy and having them tell you this makes no sense
and you need to go back and edit stuff.
So again, I think what a nicer way to get to a higher quality output than having to.
It saves me time and it saves the leaders on my team time.
And hopefully improves the quality.
you know, it's fundamentally improving how, you know, we are doing work on analytics team.
And one thing I want to call out for folks that are maybe listening and not watching is Alexa,
my friend here is smiling. This is fun. This is like interesting and it's fun. You're not sitting
here saying, I have no role to play anymore. The machines are going to take over. You're saying,
man, it was really boring to like dig through tables and write all this sequel that I know how to
Right. And I've done it a couple times. So let's let the machines do it. And now you're able to focus on interfacing with the business, having impact. And it's just, I think it's fun every time I get in these tools. I feel like it's magical. I feel like it's really fun. And so I want to call out we got smiles across the board here on how I AI. I didn't show this, but the type ahead, like if when you're actually editing the sequel, that's also so fun. It's just fun. It knows what you want to do. So yeah, this whole.
process is very fun. I think what's so powerful this is not just like making the good
analysts just incredible. It's also democratizing data. So this is something that can be done.
SQL can be written by people all over our business, whether we're in sales, designers,
anyone else can write this. So the people with the context can do analysis just like this,
and then the analysts can do the really complicated stuff where these tools could help them get
really into the weeds. For people early in their career, I've said this before, and I mean it to be
true. If you want to know the inflection point of Clairvo's career, it is when she learned sequel.
I mean, truly, I became unstoppable at that point. And so lowering the barrier to entry on data
analysis is just going to create a whole bunch of really high, high impact folks. Awesome. Okay,
Alexis, so we just saw how cursor can do end-to-end funnel analysis all the way to the proverbial
front door of your SVP strategy. Tim, let's talk about another kind of analysis, which is
experimentation analysis, my favorite.
Yeah, you should as close to your heart.
So, look, we've talked about the big picture.
We've talked about a really detailed sort of actual analysts of how they do their day job.
But I think one of the other things these AI tools are just so good is just accelerating
process, like automating away some of those routine, lower impact steps in the analytics
journey.
And so as a good example, we want to show you a quick agent rebuild, which automates the process
of writing up experiment results.
So across Fair, we might be running, I don't know, hundreds of AB tests on the product a month.
And each of those experiments needs to be monitored, assessed, documented.
And that just takes up so much time for our analysts.
So if we don't stay on top of this very quickly, it's our team that can become the bottom neck
and slow down our launch velocity, which is the last thing anyone wants.
And I know this is something that's happening up and down the country around every single tech company.
So we thought it would be a good example just to demonstrate.
So let me show you how about this.
one thing I want to really, really stress here is just how straightforward these things are to build.
Once you've gone through the pain of setting up cursor, getting your MCPs in place,
actually spinning up any new agent you can think about, it's just so quick and so non-technical for anyone to do.
So it all runs off a cursor rules file.
So if you don't know what these are, they're literally just a type of file, an MDC file that these agents know to look for and know they're likely to contain instructions.
they're really easy to set up.
It's basically plain English.
So you just write a simple one line
and just from description of what it is.
So format for writing experiment result
using Epo data.
Epo is just the experiment tool that we use.
It basically takes our data,
does a bit of analysis,
slaps a UI around it and writes it out for us.
So you then select when you want to apply.
I've just selected Applied Intelligent.
I trust the model to work out when it needs to use it
and they do a pretty good job.
And then other than that, it literally is just writing out what you want the agent to do.
Now, this might look a bit complicated.
I'll generally write this in a few minutes in plain text, what I wanted to write.
I'll ask cursor to then, tear the thing down, and I'll rewrite it a couple of times, and just get it right in the format I want.
But ultimately, it's just a step-by-step guide of what I want this thing to do.
So I've just said, for those who are listening, I've said, if you're asked to write up experiment results, do the following thing.
So ask the experiment name, if you haven't already got it.
and then go collect the data you're going to need.
So use the Epo MCP we've set up.
So go talk to our experiments base,
pull in the actual results of the experiment,
and then use our Notion MCP that we've already talked about
to go pull in all the other contexts that you might need.
So any other documentation is going to help it interpret that data
and write up this report.
And I've got a little bit down here you can see telling it
exactly what kinds of documents to look for.
So PRDs, experiment, docs, technical specifications,
that's what's going to help it look for.
And then I ask it to basically write out those results in the format I give it.
And then I'm pretty prescriptive about the format I want,
because I want this to do it really consistently in the format we want
with really tight takeaways.
So actually, I've asked it to create it in just a local file on my cursor, on my computer.
And that just means I can actually look at it before it goes to create to the notion ducts.
You can take a peek, refine the prompt if I need to.
But that's just a fallback.
and then ultimately it's going to turn it into another notion doc so everyone else in the business can see it.
And it's going to do all this incredibly quickly. And let's actually just see what this thing looks like in
reality. So let's just run it on an experiment result. So I've just said, please write up the experiment
results for, and I've given it the name of the experiment, which is vertical product tile images.
And straight off, it's gone off and it's found, it's written a nice to-do list. It's found the
EPO results. So it's just called the results. And it's found its results. Great. It's found the
the rules and now it's going to start working this all up for me which is great to see and then while it's
doing all that we'll just have a look so the format we've gone through uh we can just show here so basically
the rest of this is all just showing exactly what the format the thing's going to look like so i've asked
it to give me the document links exactly uh what i want to if i click into more context the brief summary
of the experiment and then the key bit the actual metrics that has got from EPO so it's going to show me
the actual results the confidence intervals it's going to pull out
the most important ones and it'll give me a nice little color coding for it.
And then I just want the actual answer from this. So I actually want it to do the work of
interpreting what we should do next. And so it's written the takeaway section. So I want to clear,
should we roll this out, should we roll it back? What should we do? And give me the reasons why,
like why are we doing this and are there any other interesting insights that you found that we
should call out from this? So let's see. Right. So let's have a look at what it's doing here.
It has found everything we need.
It's starting to write out the dock, which is nice to see in this little thing.
I'm just going to go ahead and queue up.
So turn this into a notion.
So as soon as I've read it, while we look at the actual results, it will start writing the notion dock.
And then have a look.
So straight away in a second, while it's running that, I have got a write up with all the right context I need.
So it's got the links I needed.
It's got the context.
It's pulled the right data. Good. The nice thing is this was also, this was just literally sharing
vertical images, rather square images, like a really standard growth experiment, like which one performs better.
And you can see a nice stat sig lift of about three and a half percent for the treatment.
And then it's pulled out some other interesting business metrics. And let's have look at these takeaways.
So it's saying, great, roll it out, the right answer because of that lift.
and it's also called out some interesting things.
So it said our data science prediction models are also actually positive.
So it's saying not only if we've got more retailers,
they're actually higher quality retailers, the ones we've got.
So this looked good.
As a first fast, this looks great.
And just to call out one thing here personally,
like we have a standard format for doing these
where you have to type the confidence interval and type the emojis.
And that is like work that is not valuable for our team.
And so it's pretty awesome that like it came up with takeaways, but also saved us five minutes of like fiddling around with emojis and decimal points.
Yeah, I mean, AI as a translation layer between a SaaS interface or a sequel query into natural language in the format that you like, that your boss likes.
That's just a time saver in and of itself.
So I love using AI as like the universal format translator.
So as you can see, I've just asked the notion,
link. It should produce the motion. So let's just open that up. And let's put it on the screen.
And look, straight away, I've got a nice document I can share around with everyone with all the
right color codes, the takeaways. And even as a little bonus, let's see it's done it.
Always has struggle getting things in a little toggle. But right in the bottom here, I've even
asked it to spit at a slack with an even more summarized version. So I can just drop this
into the right review channels. And straight away, this can go and get approved. Now, are we going
to do this for every copy cage experiment? Probably not. There might need to be a bit of analysis.
But for the simple ones, straight one shot, even the complicated ones, this accelerates you,
but also anyone in the business can start doing this, which means we can pass more and more
of these things down to engineers, PMs, other people to write this kind of stuff and do the
analysis for them, which again can just massively accelerate our launch velocity affair, which we're
really excited for.
Yeah, I'm sorry, and I know this is my brand, but I feel like AI is just accruing to every
task.
Sorry, PM, it's your job now.
So I do like that little trend that's happening.
This is amazing.
Love it, have done these kinds of analyses before.
They have not been this easy to read,
and they certainly haven't been generated in 90 seconds.
Really useful tool for experimentation analysis,
a call out to the experimentation tools out there that I know and love.
If you have not made an MCP for access to your data,
you are limiting your customers.
And so I do think sort of AI integration of SaaS tools
is going to be a way that teams start to evaluate the quality of tools that they're working with.
So it's just something to think about if you're out there building data analysis tools.
Okay, we are going to wrap up very quickly with a final.
We're going to do a bonus. We usually only do three use cases, but yours are all so good.
We're going to do a speed run through a bonus use case, which is actually designing and analyzing
kind of unstructured data in a user survey.
So Tim, you're going to whip us through how you could use AI to make service.
in survey analysis a lot better.
Yeah, I'm going to do this really quickly.
We're going to spend time with this, but let's just show it.
I think it's just another one of those incredibly common analytics use cases that everyone has to do,
and they are just so time-consuming.
You've got to design the survey correctly, code it into a survey platform,
then analyze all those results.
It's really time-consuming.
But end-to-end AI can just, like, transform the whole process.
Let's show another one.
I'm just going to stop out.
I'm not going to run these.
I'm just going to go straight to my backup.
So let's just start on design.
So what I love doing this, I think you can do it on many things.
I think chat-GV-T projects is really.
good for this and again incredibly accessible everyone knows how these work it's just a great way of
giving context so if we switch over to this one which chat to petite it's lovely and taking a bit
times a load you can see in files what I did was give it a bit of background information so
what is our bit of business so this was a survey we want to design on fair direct tools so that's our
tools that we give all our brands to help them accelerate their sales with their own customers
and so I've given a ton of information to the model that just says like what actually is
say direct what are these tools what's the strategy and then whenever i do a survey like this um
whether i'm doing AI or not i'll start with hypotheses that that's ultimately what you want to test
and so this is the nice way if i just open up those hypotheses so this is what i fed it into
i just gave it a list of simple hypotheses on what um what we want to learn we do line we've got
everyone aligned on some hypotheses there's 14 in here and then really simple i'll just call that one like um
Higher sales on fare leads to more usage of these tools.
Things like that that we asked.
Now, I've just given that into it.
And all I did, if I just look at this prompt that we ran,
so this was a simple prompt.
All I did was drop it in saying,
you're a specialist of doing this customer insight surveys.
Design me a 10-minute survey for the 1,000 brands to test those hypotheses.
I said, these are the inputs I've given you.
Here's a bit of design requirements that we want.
And I asked the three things.
I said, turn those hypotheses into a full.
questionnaire that we can go ask our customers, but also don't just do that. Give me the coding file
that turns that questionnaire into the actual, in this case, Qualterix, the platform we use to
actually run these things, can actually design that straight away in one click and give me an
analysis plan for something to do there. I have to pause you really quickly because this whole
episode has been Tim saying, I just did this really simple prompt and then you see this like
1,000 word, hyperstructured, very organized prompt.
And Alex was like, oh man, I would just go in there and be like, maybe a nice survey, please.
I love it.
So I'm a big believer that 99% of my prompts are going to be one line.
And then if I'm going to send a model, a big model is going to do work for 15 minutes.
I'll probably ask another model just to turn my one line into something more detailed.
I want the AB test of Alexi.
You run this exact same GBT with a tinier prompt.
And you tell me if you get the same quality.
See what happens. See what happens. Baby I'm just, I don't trust it quite as much as Electra does just yet.
Okay, so what do we get from that? So very quickly, from a list of hypotheses, I've got straight
away a really nice first pass of a survey. Now it's going to ask loaded questions, it's about the
right length. Like this can just massively accelerate the process. And then once we've got that
right, it's also giving me that coding file, which I just scroll and screen, these things are painful
to write. So just having this, a one-liner to tell exactly how the system should prompt this
and write it out is just like saves hours of time for our research operations team
and it even then translates that into an analysis plan that says this is what the outputs
from that are going to look like. So straight away, this whole thing can go from a list of
hypotheses into something we could probably get out to our customers by the end of the day. Now
that's like shortens this enormously. But what happens when you get the results back?
That's the other thing this can do. And so again, I'll do this incredibly quickly and just show
you the final result. But I did a very similar prompt as well. So all I did, I'm going to
show you the file I dropped into this, just show you how painful this is. So I just gave the same
hypotheses and look how bad this. Like it's the raw output from Quarotrix. Like these usually take
a lot of cleaning. It's one line for every respondent and then one column, not just for every question,
but for every possible answer to every question. So these things are incredibly dense for anyone's
worth them. And they take a bit of time, a bit of playing with. So the only other thing I gave it was
a sub helper file
which was
basically that
sort of coding file
that I just showed you
so it's the
what's the question ID
what's the question language
what's the answers
and then is it
I just add these two columns
which is like
is it a demographic question
or an answer
and is it a single choice
or is it a multiple choice
that's all I gave it
and then I've written
another one of my
fun and simple prompts
so
roll
task here just
analyze the survey result
find the most interesting things in this data
and then judge the predefined hypotheses
so I want a table that basically says
like for those hypotheses was it right or was it wrong
and then again I always end on little quad check rest
I don't want it to go away 15 minutes before
and come back with something that isn't very useful
and let's have a look at this just very quickly
so I've got a nice sort of summary out front
and then there are my 14 hypotheses
and it's got a nice table that says proved
neutral, disproved for each of them
and it's even because I asked it to
giving me a nice confidence score
so I said one, it's really confident in this
five, it's not very confident at all
and you can kind of see the different levels
throughout this and then beneath it I've got
for each of these, actually the specific analysis
that I asked to do so just throw all the insights
it found to back up those findings
so like is this the only analysis
we're going to do on this survey? Like almost certainly not
but day one I've got the results
I've thrown it into this and within a matter
of minutes I've got
a much, much better intuition of what all that day is showing.
So while I might go and do some analysis on this,
I can be so much more targeted on exactly what we want to look into
and where I want to spend my time.
And straight away, we can start sort of sharing some of these findings out with people very, very quickly.
Oh, no.
So I'm reflecting now after this episode.
Like, okay, I've told everybody to ship a bunch of features,
and now I'm going to be like, do a bunch of analysis.
Like in my mind, I'm like, oh my gosh, I'm underusing AI to actually understand my business.
And it's so accessible.
And if I can just write 17 point prompts like Tim, I can get really high quality insights.
But I do want to call out just reflecting on this whole episode in your four workflows.
What I love about what you're showing us is so many people think that AI is an input to producing a thing,
but haven't done that full circle back to analyzing the thing.
sharing the thing, communicating about the thing.
And I think you're showing both sides.
You can create with AI and you can analyze and communicate with AI.
And I think looking at both sides of that coin is really useful.
Okay, we are going to do the one and only lightning around question because we have gotten
long on this episode.
And I want to get you all back to all of your agents and MCPs and analysis.
We're going to go back to prompts.
One last time, we're going to figure out your personality around prompts.
Alexa, Tim.
when AI is not listening, when your MCP will not call the tool, what is your prompting
technique? Alexa, what do you do? I think mine's pretty straightforward where I think the problem
that I run into most frequently is that I'm clearly running out of context. Like a conversation
has gone so long that it's starting to be wonky. And so while I think, you know, level one is
just starting over, what AI is best at is summarizing. So I'll also, I'll
say, hey, summarize like what we've done so far in this, you know, 30 turn conversation and then
use that to start over because, you know, like, like I've heard other episodes people say you
want to figure out like where it got off track. Clearly I'm a pretty efficient person. I don't,
you know, I'm not Tim. I'm not like writing out the entire prompt for 20 minutes. Like I don't have
time for that. I just want to say, hey, summarize what happened? We're going to start over, but I'm going
to give it that summary. So at least the new conversation can get some.
context from the old. Great. And Tim, what about you? So much stick for my prompts. It's all AI. It's all
AI. Work my chat. So I generally will go and open up three windows on cursor and I'll go three
chats with three different models and put the same prompt in and get up a cup of tea and see what
comes back. That's the British stereotype in me and getting my cup of tea while I do that.
Yeah, you run the AB test is what you do. Okay. I love this. Tim, Alexa, where can we find you
and what can we be helpful with?
You can find me on LinkedIn.
My full name is Alexandra.
And ways to be helpful.
Our strategy and analytics team is hiring across the board.
Our team partners super closely with PMs and our go-to-market team.
We make strategic data-driven decisions, super fun.
We have tons of open roles.
So if you like experimenting with AI, we are very AI forward.
So you can learn more at fair.com slash careers.
And you can find me on LinkedIn as well.
and I echo that as well, like come join us. If you love AI, come join us and show us how we can do it more here.
Okay, we will link to your careers page in the show notes. Alexa Tim, this has been so fun. Thank you for joining How IAI.
Thank you for having us. We're having us. Thanks so much for watching. If you enjoyed this show,
please like and subscribe here on YouTube or even better, leave us a comment with your thoughts. You can also
find this podcast on Apple Podcasts, Spotify or your favorite podcast app. Please consider leaving
us a rating and review, which will help others find the show. You can see all our episodes
and learn more about the show at how IAIIPod.com. See you next time.
