Everyday AI Podcast – An AI and ChatGPT Podcast - EP 142: AI is Changing Data Analysis: Insider Tips

Episode Date: November 10, 2023

How is AI going to change data analysis? Will spreadsheets and databases become irrelevant? Zain Hoda, Co-Founder of Vanna AI, joins us to discuss what the future of data analysis and data teams will look like with AI. Newsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageJoin the discussion: Ask Zain and Jordan questions about AI and data analysisUpcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: [e-mail] with Jordan on LinkedInTimestamps:[00:01:12] Daily AI news[00:03:30] About Zain and Vanna AI[00:06:00] How AI is changing data analysis[00:12:20] Impact on data teams in companies[00:15:00] LLMs and data misconceptions[00:21:12] Enterprise LLMs and data safety[00:27:00] GPT-4 is best for data analysis[00:29:20] Zain's final takeawayTopics Covered in This Episode:1. The Future of AI in Data Analysis2. Use of Generative AI in Data Analysis3. Concerns and Considerations Around Generative AI4.  Accelerating Business User Questions and Data AnalysisKeywords:data analysis, AI, ChatGPT, GPT-4, GenAI, enterprise, LLMs, dataSend Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info) Start Here ▶️Not sure where to start when it comes to AI? Start with our Start Here Series. You can listen to the first drop -- Episode 691 -- or get free access to our Inner Cricle community and all episodes: StartHereSeries.com Also, here's a link to the entire series on a Spotify playlist. 

This is the Everyday AI Show, the Everyday Podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life. Meet Firefly AI Assistant, now live and Adobe Firefly, the All In One Creative AI Studio. Just describe what you want to create and the assistant handles the rest, orchestrating multi-step workflows across Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome. The assistant accelerates execution. How is AI going to change data analysis?

I love data. I love spreadsheets, but I also love AI. So today's conversation should be a fun one. I'm excited. Are we still going to be using spreadsheets and databases and dashboards in a couple of years? Or is AI just going to take care of all of that for us? And we're just making sure that it's doing its job. I'm not sure.

That's why I bring smart people on this. show. So welcome today to Everyday AI. My name is Jordan Wilson. I'm your host. And Everyday AI is it's for you. It's for all of us, actually. But it's a daily live stream podcast, free daily newsletter, helping us all learn and leverage generative AI. So if you haven't already, make sure you go to your EverydayAI.com. Sign up for that free daily newsletter because it's awesome. I'm biased because I write it. Yes, I actually write it a human right to newsletter. I know, crazy concept. All right. But before we talk about AI and how it's changing data analysis.

Let's first go over like we do every single day, the AI news. All right. So first, pretty soon, AI won't even need us, according to a recent report. So singularity is less than eight years away, according to a scientist who's an expert in the field. Singularity is the moment when artificial intelligence surpasses human control. And this report is coming out of popular mechanics. So this is Ben Gortzel, who is the CEO of SingularityNet. And he recently said that artificial general intelligence and singularity is three to eight years away.

All right. So apparently, everyday AI might be run by AI. Next piece of news. So Gen AI is kind of coming to our banks. Well, so J.P. Morgan is collaborating right now with U.S. regulators to implement some generative AI models to ensure proper risk. management and control measures. So this new JP Morgan, Gen A.I. Model, if approved, would be used for their help desk service that helps solve problems and also track earnings summaries for all the companies that J.P. Morgan tracks. All right. Last but not least, Bill Gates says we're all

getting personal assistance. All right, Bill, does that mean you're paying for mine? Paying for everyone's? I don't know. So billionaire Microsoft co-founder Bill Gates predicts that AI-powered agents will revolutionize the way that we work with technology and carry out tasks, and it'll be in the next five years. So Bill Gates recently said in a very new report, about 30 minutes old, said that they will utterly change the way we live, in quotes. And he said, we'll all have them even if we don't work in an office, right? I think we assume these personal AI agents are just going to be for those people who are on the computer all day. but, hey, according to Bill Gates, we're all going to have one.

Just following us around in the home. That sounds fun. All right. Also, if you all didn't know, GPs, custom GPs from Open AI, they should be out in the wild. If you haven't seen them yet, go hit refresh. Let us know. All right. But we're not here to talk about all the AI news that's happening on this beautiful Friday morning.

We're here to talk about how AI is changing data analysts. Data analysis. Sorry. Data analysis. So I'm excited for our guest today because I love data. I love AI. So let's bring on to the show. And please help me welcome. Here we go. Got him. Zane. Zane Hoda, the CTO of Vana AI. Zane, thank you for joining the show. I appreciate it. Hi, Jordan. Thanks for having me. Yeah, absolutely. Hey, and everyone, if you're joining us live, what questions do you have about data? You know, all those numbers, all those databases floating

around. Zane's going to help us figure it all out. But Zane, tell us a little bit about what you do at BANA AI. Sure. Some contacts may be helpful. So I used to work in finance and then I actually ran a data company that I sold to a larger data analytics company. And so I think what we generally saw when we were delivering data was that we, like, if you're delivering data to business users, you're typically delivering that data in the context of dashboards. So you're making like a fixed dashboard to answer like a limited set of questions. And if any sort of business user wants to dig deeper, they need to get an analyst involved in order to query that database. Because really only the data analyst is going to understand like, oh, where is this data stored? How does that

data join together? And so what we saw was that actually generative AI can solve a lot of this problem because particularly if you have SQL databases, you can just use the information that you have about the database and ask questions and have it generate SQL. And then we have systems to then go and execute the SQL, generate charts and things like that. And so that's kind of like the genesis for VNA AI. And so like how do we help data analysts answer these questions from business users like much faster? And then for, you know, the simpler questions that business users have, like when they

have, when they're in the dashboard and they want to go slightly deeper, they could potentially use the AIS system themselves. So, so let's see. start at the end, Zane. Let's start at the end and we'll circle our way around. How are you seeing already how AI is changing data analysis? I've seen it myself. But I want to hear from you first, like from your vantage point, how is it already changing? Absolutely. So I think like the time to insight is getting shorter and shorter. So, you know, from when you have a question or when you have a hypothesis to actually getting an answer,

I think AI is substantially accelerating that process. And then on the job side, so if you're a data analyst using AI right now, generative AI as part of your workflow, I think what you'll quickly realize is that documentation becomes much more important as part of your job. And I think if you think about like, you know,

If you are onboarding a data analyst and you're training them on like, you know, where the data lives, like how it's collected, how it joins together, what feels to use, what feels not to use, like how this company functions, what's important, things like that. All that now, if you document that in a text format, that is all information that you can pass on to the AI and have the AI do a lot of that heavy lifting for you. You know, it's interesting here. And, you know, I'm actually just going to jump straight into Jay's question because this is where my mind was going to. So Jay, thank you for the question. And if you do have questions for Zane, make sure to get them in. So Jay asking, I assume no more spreadsheets needed.

You just feed AI the data and ask questions you want answered and the AI will derive unseen insights. Is that where we're headed toward? Are we headed toward maybe a dashboard list, spreadsheet list? future of data? I think that that's one possibility. And so right now with chat GPT's like advanced data analysis, you can already do some of that. You can actually upload CSVs or Excel files and have it begin doing analysis for you. I think that is a definite direction that we're going. At VAN AII, we're actually not doing exactly that because what what we're doing is like we are just accelerating the process from question to and answer.

And like it's the data analyst and business users jobs to come up with the questions and to come up with like the actual insight. But that, you know, as we see, you know, GPT5 come out, GPT6 come out, I think this questioner is probably right, that more and more of that will shift directly into the AI system. Yeah, Zan, I'm happy you brought up ADA. So if you're new to data or maybe you're newer to, you know, even chat GBT, so advanced data analysis is actually a fantastic mode inside of chat GBT. I think early on, Open AI and chat GBT kind of got a bad wrap because it's not good

with numbers by default, right? It's not good at math. So people kind of wrote it off. But the advanced data analysis mode inside of chat GBT is actually extremely impressive. I've uploaded spreadsheets with tens of thousands of data points that would normally take me. You know, I'm not a professional data person, but it would normally take me many hours to make, make sense or make use of this data. And to be able to have a conversation with it. It really, I think, Zane, and I'd like your take on this.

It seems like it's also bringing data to more people that maybe think they either didn't need to, you know, analyze data before. Maybe they think they didn't need it for their role. Is that something you see kind of for the future is so many more people are going to start using and leveraging data that didn't before because of the accessibility? Absolutely. I think that's actually one of the major reasons why people come to us because, you know, typically what we'll see is like at large organizations. you know, the business users have tons and tons of questions. And the way they get them answered is they need to get a data analyst involved. And typically, you know, they're going to be asking them either via Slack or like, you know,

if it's like a structured team, they'll put in like a Jira ticket. And like in the ticket, you'll be like, oh, can you like answer this question? Then like eventually weeks later some data analyst like picks up the ticket, writes the sequel, runs it, you know, and then sends back the results. But then the results actually prompt like potentially five additional follow-up questions. And so now you have to begin that process all over again where you need to like submit additional tickets. And for the data analysis team, they're just like inundated with all these tickets. And so, and you know, they're not really providing that much value add as part of this process.

They're literally mechanically taking in questions and then, you know, writing SQL queries and getting the results back. And so that is the perfect place for AI to accelerate that process. Because if you can ask a question and just get like the answer in like a couple of seconds and then have additional follow-up questions, it just allows you to dig deeper and deeper into the data and just basically make organizations a lot more data-driven. And we've seen typically, you know, a lot of organizations have already done the heavy lifting of like the data collection and getting a lot of that data into data warehouse.

into databases. And so now that you have all this data in the database, a lot of that is just like sitting there. And really the key question is how do you unlock that for like really anybody in the organization? Yeah. Are we going to see something as an example? Because I know all companies work differently and it depends on the size. But it seems like to me sometimes in, you know, medium organizations, let's say, you know,

you have your data team. They're doing great work. they have their dashboards. And it almost seems like that data kind of lives with that team. You're right. And you almost have to go to them or they have to run you some reports. Or maybe if you're, you know, a highly technical person in a different department, department, you might be able to make sense of those dashboards.

But how is even the role of those types of people going to change as generative AI becomes more commonplace? Are we going to see like as an example, the majority of companies? have a in-house large language model that you can log into an internal system and just being like, hey, what's going on with my sales, with my KPIs? And are they just going to be having conversations with in-house data? And then those data teams are just making sure they work. Yeah, I think that's a potential direction. I think one of the things that we're seeing a lot of is that there's a lot of skepticism around using open AI because a lot of companies like don't want to risk leakage of

their confidential information through open AI to other parties. And so, you know, as you, as you said in your talk, you know, J.P. Morgan, for example, is building up their own large language model. And it has some potential, but generally from what we've seen, GPT4 is still the best at being able to do data-related tasks. In particular, be able to generate accurate SQL queries for databases. And so I think until some of these open-source models, like, you know, get a little bit better at some of these tasks, I think for now what we'll probably see, is organizations use OpenAI in one way or another. So what we're seeing for like for organizations that have particularly strict confidentiality requirements, they're sort of okay with using OpenAI through Azure because Microsoft via Azure is providing some SLAs

and guarantees that any information that you provide it won't be used to train the foundational model so that you don't have that risk of leakage. Yeah. It is interesting, right? Because I think maybe it's a common misconception or it depends on your level of technical expertise. But it seems, at least to me, there's a divide on OpenAI and chat GPT and even advance data analysis.

I think people either think of it and they think, oh, that's to write a blog post, right? Or it's someone like yourself. You're saying, oh, no, like that's the future. People should be using this. Why do you think there's so much misconception around, you know, such a powerful tool like GPT4, OpenAI and advanced ad analysis even? Why do you think that maybe more people aren't even leveraging it? I think a lot of it is just exposure. So I think people haven't seen. the capabilities and that really hasn't like propagated. So, you know, what people typically do is like they go to chat GPT and they use it for like text-based stuff.

So they use it, you know, for blog posts. But actually, you know, there's a lot of nuance in blog posts like, you know, people have different writing styles. Like, in fact, I would actually say that large language models are probably better at doing data analysis than they are at, you know, generating copy and things like that. Okay. Okay. We've got, we've got our first hot take of the Friday morning.

I like that because I think people always assume, right, that, oh, chat chb-t is not good at math. It's not good at computing. So they think they kind of couple that in with data analysis, whereas it is kind of almost a different thing altogether. You know, people always try to stump, oh, you know, chat GBT can't handle this math problem. So it probably can't analyze data. It's like, yeah, it can, right?

Like, it's two different things. Yeah, I think in particular, if it's connected to other systems. Yep. So if what you're doing is you're using it to generate SQL queries. Now, one of the biggest things has to do with hallucination. So, you know, if hallucination is your biggest issue and what you're doing is generating SQL, there's actually ways to then verify that, okay, is the SQL output correct? Is it formatted correctly? And then does it run? And then once you get the results, like, do the results like make sense? It's a lot easier to kind of verify that than to ask chat GPT about facts. Because like, facts need some sort of like external references. But in the case of data analysis and generating SQL queries, the facts live in the database. And so all it has to do is generate the correct SQL to

access it from the database. You know, and just, just hey, everyone, just in case you're listening, we're dropping some buzzwords. Don't worry. If you're here in SQL, that's a structured query language. It's kind of like a standard language for database creation, manipulation, right? I'm like, I'm pretty sure I've got that right, but I want to explain that as well, because also a great question here from Tanya. So Tanya asking, what does the future look like for SQL engineers? Like, yeah, Zane is, I mean, we kind of touch on the future, but what does that look like specifically for SQL engineers? Absolutely. I think that I think people will still need to learn SQL just in the same way that you learn, like long division, for example.

but the question is how much of that are you going to be manually writing in the future? And I think we're going to see less and less SQL in particular manually written. And I think that role changes a little bit in terms of, instead of like writing SQL, I think the thing that you're going to be doing is writing documentation, getting really good at documenting everything so that you can get AI to write your sequel for you. Yeah, and I'm even wondering, right? So many, and it's, it's hard, Zane, too. It's hard to think, like, with how quickly things change. It's hard to think, you know, multiple years in the future. But I mean, is there specifically when we're talking about data analysts, is there a point where mainly data analysts in the near future are just overseeing AI models exclusively, right? Maybe they're not even writing the documentation. Maybe they're just making sure the Gen AI models are writing the documentation correctly and that they're running all the models correctly.

Is that an actual thing? Because it seems like data is very finite, zeros and ones, bytes and bits, right? Are there always going to be the need for human intervention in the actual analysis process and not just the overarching making sure that all of the AI systems are doing it correctly? Yeah, that's a great question. And I think a lot of that is going to come down to, like, organizations are very, like, dynamic. They change over time. You know, companies buy other companies that are, like, integrating different systems.

You know, you were using a particular data collection method that changed or broke somehow. So there's a lot of this knowledge that gets built up inside of an organization. And I think, like, as part of the data analyst's role, is going to be. be keeping on top of that and basically making sure that that information is then transferred into the AI system so that it understands what's going on in the larger business. And so I think it's going to be even more important for data analysts to be kind of that bridge. So be in touch with company leadership, follow what's actually going on inside the company and basically like doctor document, so document, document, like all of this stuff that is happening at the organization.

Yeah, and I'm curious about this one because, you know, I know people who are in data fields, and it seemed like even up until, you know, a month or two ago, it seemed like so many of them weren't even using generative AI at the time. And I know a lot of it because when we talk about data, data is gold, right? Data is everything and you have to protect it. You have to be secure with your data. But I guess now that there's these kind of more enterprise models coming out or enterprise systems, you know, chatGBT enterprise is rolling out.

Microsoft 365 co-pilot, you know, you mentioned Azure. Is like, is there still going to be that point where companies or data teams, are still going to be like, oh, no, we can't use these models. It's our data. Like, will that wall ever, I guess, crumble down? Yeah, I think once people realize how much more efficient you can be, I think we'll start to see some of these barriers come down. In particular, I think where we're at in the cycle is that AI is largely being used

as like an experiment or a toy. but I think very quickly we're going to start to see it become like an integral part of company workflows. And the way that that's manifesting itself, where we see it, is that we see a lot of companies doing hackathons. And so like there are a lot of companies are doing these hackathons where it's like, oh, there's like a bunch of AI. How can AI like help us and integrate into our workflow? And so, you know, the vast majority of the winners of these hackathons end up being something related to data in particular, because I think that's one of the easiest ways for an organization to just become a lot more efficient. Adobe just introduced an entirely new way to create, bringing the power and precision of its creative suite into one conversational experience. Meet Firefly AI assistant, now live.

in the Adobe Firefly app, the all-in-one creative AI studio. Powered by Adobe's creative agent, Firefly AI assistant lets you start with your vision, just describe what you want, and shape the outcome as it takes form with the assistant. The assistant orchestrates multi-step workflows, drawing on 60 plus pro-grade tools across Adobe Creative Cloud apps, including Photoshop, Illustrator, Premiere, Lightroom Express, and more to help bring your ideas to life. You can also get started with creative skills, a growing library of pre-built workflows for common creative tasks, like batch editing photos, creating mood boards, portrait retouching, and creating social variations. Every step the assistant takes is visible so you can refine, redirect, or take over at any time.

You stay in the driver's seat as the creative director. Adobe Firefly AI assistant now in public beta. See it today at firefly.adobie.com. Yeah. And as we, you know, as we, you know, move forward in this, right, because I literally just had this conversation with someone the other day, they, they said, yeah, we have all of our data, you know, already in, you know, Microsoft's cloud or Google Cloud or whatever, specifically when it comes to data, if companies are already storing it, right, in a cloud, I guess from a, from your perspective, you're the, you're the, you're the, you're, you're the, data expert. I'm definitely not. What's the difference then? What's the difference, you know, if let's say if they're using, you know, Amazon's cloud for, so using, you know, if their data's already in the cloud, why wouldn't they just use that same cloud providers, generative AI? Is there, is there a difference between something already being in the cloud versus tapping into the generative

AI model that they may be, that the same company may be offering? I think that, yeah, that's an excellent point because you're right that a lot of data that used to be stored in-house is going more and more into the cloud. And if you're already trusting the cloud provider, then what's the difference between trusting the cloud provider and trusting an AI system in the cloud? I think that's very accurate. I think the one thing I would say about that is that there's still an apprehension about like leakage. So it's like the fundamental question is like is the data that you give it, being used to train the system, the same system that will be used for a different user. And I think, like, you know, we're going to see tighter guarantees around that.

Because as long as we can guarantee that, like, you know, when you give data to the AI, that it doesn't, it will never end up in the hands of another, another user of that same system. If you can isolate them much better, I think that'll drive additional enterprise adoption. Yeah, it's, it is something it's, for me, just head scratching that people are always so, so open and willing to use cloud services or to, you know, send big documents. But when it comes to, you know, using a generative AI model specifically with data, it just seemed to me that there was always some sort of hesitation that didn't quite add up. It's like, all right, well, if you're going to do it anywhere else, but it seems like maybe the big,

maybe in the data analyst world, Zane, is what you're saying, is it's more of the how they do it. Like how they use their data sets, how they train it. That's necessarily what they don't want the generative AI models to, quote, unquote, learn or use. And it's not necessarily the data itself. Is that right? Yeah, I think that's part of it. I think the other thing that's happening is like regulatory. So as an example, like if you are in healthcare and like you're uploading like potentially confidential patient information, now the question is, is that even allowed?

And I think like we haven't even answered that question and, you know, probably beginning with like the recent executive order and additional regulation that comes down the pipe. Like we'll probably see a lot more clarification around potentially like what's allowed, how you're allowed, to use that, particularly as it relates to like these regulated or heavily regulated industries like healthcare and potentially finance and things like that. Yeah. And it's that that is an interesting point. And it's funny you bring up those two industries because I feel those two industries before Gen AI, right?

Traditional AI, I mean, financial and health are two of the sectors that have been using kind of like quote unquote old school AI and machine learning for the longest. you know, it's been for many decades. You know, Zane, I want to transition to this. What's maybe one thing that other people in your field, that maybe they're getting wrong when they're thinking about data analysis and AI, maybe everyone's saying, oh, this is the direction. What's one thing that maybe you're seeing people get wrong or maybe one opinion that you have

that some of your colleagues might not really agree with when it comes to data analysis and AI? Yeah, so I think the biggest one there is that GPT4 is still far and away the best foundational model to use for data analysis. And so if you're going to be using AI right now with data analysis, you pretty much have to be using GPT4. If you're trying to train up or fine-tune one of the existing open source models, then I think you're going to see that you're going to fall a little bit short potentially. And, you know, it costs potentially like hundreds of thousands of dollars to, or potentially millions of dollars to train a model from scratch.

So I think what we're going to see at the end of this is like probably a number of companies will try to train their own large language models and then realize that actually, you know, GPT4 and then like whatever like succeeds it, GPD5 like a year from now, potentially is going to be way smarter than the one that the LLM that they have right now. So they're always going to be trying to play catch up. So I think that'll be an interesting, interesting thing to keep an eye on. You bring up such a good point because, you know, and I mean, we just got GPT4 turbo, you know, in the last like 48 hours.

I do think that, you know, smaller companies, medium companies think that they can build, yeah, they'll build their own model. But yeah, like you said, sometimes I think if you fill out the form on Open AIs website, I think you have to click and you have to click like you understand that it could be a $2 to $3 million investment to build your own model. So I guess moving forward, Zane, maybe there's a small business owner or someone that works in a medium, you know, size company on a data team. What kind of parting pieces of advice can you have for them as we, you know, as we wrap

up today's episode, because we've covered a lot. But maybe what's, what's that takeaway that at least where you see data analysis heading in the world of AI? What do people need to get right? What is that one direction that they should be focusing on? Yeah, I think the main thing is like, think about all of your hidden assumptions. Like when you're doing analysis, think about like, you know, if some, if business users are asking questions, like, oh, what was like, you know, our daily active users, like, over the last week, just think about, like, all the assumptions that go into that. Like, what is the actual formula?

Like, are there any restrictions? Like, are you excluding certain things? Like, I think being able to, like, articulate that and document that is going to be, like, a key skill over the next, you know, probably year or so. Yeah. Absolutely. The next year or so is going to be a wild ride. But at least I think we're all a little more secure now in our data. Thanks. Thanks to you, Zane. So Zane, thank you so much for coming on the show. We really appreciate having you on. Thank you, Jordan, for having me. Hey, as a reminder, we covered a lot. There's a lot more. We always break down each and every interview in much more detail. So make sure if you haven't already, go sign up for the newsletter. Go to your everyday AI.com. Sign it for. for that free daily newsletter.

We're going to have a lot more from Zane. A little more information, too, about BANA AI. I know there's a couple of question or two. We didn't get to it. So make sure we'll get that in the newsletter. So we hope to see you for that. And we hope to see you back for more everyday AI. Thanks, y'all.

Meet Firefly AI assistant. Now live in Adobe Firefly, the Allman One Creative AI Studio. Just describe what you want to create in your own words and the assistant handles the rest, orchestrating multi-step workflows across Adobe Creative Cloud apps, including Photoshop, Premiere Express, and more. more in one conversational interface. You direct the outcome while the assistant accelerates execution. Stand control with the ability to step in and refine at any time.

See it today at firefly.adobie.com. And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going. For a little more AI magic, visit Your EverydayAI.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.