Everyday AI Podcast – An AI and ChatGPT Podcast - EP 372: Maximize the Power of AI With Data Streaming

Episode Date: October 3, 2024

Are you missing out on the power of streaming data? Want to know how to make your AI faster, smarter, and more relevant? Join us for a deep dive into how data streaming can transform AI from a predictive tool into a real-time decision-maker with Will LaForest, Global Field CTO of Confluent.Newsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageJoin the discussion: Ask Jordan and Will questions on AI and data streaming.Upcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: [e-mail] with Jordan on LinkedInTopics Covered in This Episode:1. Internal Use of AI2. Data Integration3. Risks and Importance of Data Streaming4. Data Governance and Traceability5. Future of Data and AITimestamps:01:30 Daily AI news05:15 About Will and Confluent09:15 Real-time data is crucial for current accuracy.12:16 Data streaming enables AI use for midsize businesses.15:11 Data streaming enables real-time customer data updates.19:36 Data streaming's stability is crucial for finance.20:27 Data governance ensures safe, accurate data delivery.25:16 Services simplify data streaming integration for companies.Keywords:AI internal use, Data integration, Risks of data streaming, Data governance, Provenance and traceability, Ethical AI use, Future of data and AI, Industry outlook, Data streaming services, Notion AI, Industry-specific integration, Quality data for AI, Real-time data importance, Confluent, Data streaming, Generative AI, Mid-market companies and data streaming, Data streaming use cases, Confluent's mission, Customer testimonial, NVIDIA NVLM, OpenAI competition, Google AI development, OpenAI funding, Will LaForest, Uber and data streaming, Generative AI context, AI in business, Enhanced productivity, Data privacySend Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info) Start Here ▶️Not sure where to start when it comes to AI? Start with our Start Here Series. You can listen to the first drop -- Episode 691 -- or get free access to our Inner Cricle community and all episodes: StartHereSeries.com Also, here's a link to the entire series on a Spotify playlist. 

This is the Everyday AI Show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life. Meet Firefly AI Assistant, now live and Adobe Firefly, the All In One Creative AI Studio. Just describe what you want to create and the assistant handles the rest, orchestrating multi-step workflows across Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome. The assistant accelerates execution. When it comes to generative AI, sometimes it seems like yesterday's data may as well be last year's data, right?

I think for especially enterprise companies and for all of us interacting with them, right? If you're chatting with a large language model on a big company's website, you need today's data. No, you need this hour's data. You need this minutes data. And that's where data streaming and the intersection of data streaming and AI become so important. So that's what we're going to be talking about today and doing a deep dive into not just what data streaming is, but kind of how it helps power AI that we all need and we all use. All right.

So that's what we're going to be talking about. I'm excited. If that sounds like something you want to know, well, you're definitely in the right place. So if you're new here, thank you for joining us. My name is Jordan Wilson. and this is Everyday AI. This is a daily live stream podcast and free daily newsletter, helping us all learning leverage, generative AI,

to grow our companies and careers. So if you haven't already, please go to Your EverydayAI.com, sign up for the free daily newsletter. We will be recapping today's conversation and a whole lot more, everything you need to know. Speaking of everything you need to know, let's start off as we do every single day with a quick recap of the most important AI news.

So Nvidia has launched an open source AI model called NVLM challenging industry giants. So, Invidia's release of NVLM, the family of multimodal language models, is a significant development in the AI landscape as Nvidia will now be competing with established proprietary models from companies like OpenAI and Google and the open source or open weights leader in meta. So the NVLM D72B, that's a mouthful. It showcases exceptional performance across both vision and language tasks, enhancing its

capabilities in text-only tasks, which is a notable achievement in AI development. So, Nvidia's decision to make the model weights publicly available, along with plans to release training code, mark a pretty significant shift toward open source practices in a field dominated by closed systems. So benchmarking comparisons, early ones for Nvidia's new model, show it performing competitively against leading models such as GPT4, Claude 3.5, and Lama 3. All right. Speaking of models, yeah, tons of model news.

So Google is working on a reasoning-based model similar to OpenAIs 01 or Strawberry. So according to reports from Bloomberg, Google is making significant strides in the development of artificial intelligence software that mimics human reasoning, a move that intensifies its rivalry with Open AI, and they're recently released, O1 preview model, what a lot of people refer to as strawberry. So multiple teams at Google are reportedly making progress in AI reasoning software, which excels at solving complex, multi-step problems in areas like math and programming.

So the company's efforts come in response to OpenAI's recent advancements, particularly the launch of its O1 model known internally as Strawberry, previously as QSTAR, which has raised concerns within Google DeepMind's team about falling behind. All right. Last but not least, speaking of Open AI, they secured their $6.6 billion funding round while reportedly restricting investment in competitors. So Open AI has successfully closed their $6.6 billion funding round from some pretty prominent investors, including Thrive Capital and Tiger Global, marking the largest round of funding ever at $6.6 billion. So Open AI has requested, though, according to reports, that investors refrained from funding five of its competitors,

including Anthropic XAI from Elon Musk and company, Safe Super Intelligence, SSI, as well as two AI application firms in perplexity and glean. So the exclusivity agreements could reshape the VC landscape, concentrating funding around fewer larger companies, and potentially stifling innovation among smaller startups. However, this ambitious strategy from OpenAI may open it up and attract some increased regulatory scrutiny and could push competitors to accelerate their innovation efforts to remain relevant in the market. All right, that's a lot in a very short amount of time.

Don't worry, we're going to have all of that in today's newsletter and a lot more. But today we're here to talk about data streaming. Like, what the heck is it and why is it so important for both big entrepreneurs? companies and for all of us, right, functioning in our daily lives. Cause now it's like we need, you know, all big companies to have data and to have chat botches like we need them to, you know, have a website. It's kind of the same thing, right? So enough of me, chatting.

I'm excited for today's guests. So let me bring on to the stage. There we have him. Will LaForrest, the global field CTO of Confluence. Will, thank you so much for joining the everyday AI show. My pleasure. That's, that's, it's not a trillion dollars, but it's a lot of money. Yeah.

gosh, right? But yeah, or seven trillion, right? Those, those, those numbers that were floating around originally. I knew it was in the trillions. Yeah, it's wild. So let's let's start here. Well, first of all, Will, thank you so much for joining in, you know, for our live stream audience. Hey, it looks like LinkedIn streaming's back. That's that's great. So if you have questions for Will, please get them in now. But let's start at the top, Will. What the heck is data streaming? Yeah, yeah, absolutely. Sometimes people call it event streaming as well, but essentially it's a, it's a, it's a category of data technology that's really created to handle the continuous flow of data, right? So as data is generated, it changes occur, it's published, and it can be continuously

processed, enriched, filtered, and acted on. And so this data or the derivatives of it then can flow to any number of downstream consumers that are subscribed and at massive scale. So like, Great example of one of those consumers that we'll be talking about today, I'm sure, is things like a vector database to support RAG architectures, which are really critical. I think the easiest way to explain this technology of people who are not like data nerds, not data infrastructure people, is to use an example, right? So if you think about Uber, for instance, if I'm a driver, you know, I have this phone

and I'm moving around. Like it's constantly sending these events, this data. It's like I'm Will, I'm in a Mercedes, I'm at this specific location, and I may be with the passenger or maybe not, right? A second later, I'm a new place, a second later, I'm someplace different, okay? If I'm a writer, same thing's happening. Like, I'm walking around, trying to get to the front of my hotel, or, you know, walking in a terminal of an airport.

And so the most obvious thing is with these constant, streams of data is like if I ask for a ride, it needs to match me with someone, you know, Uber and pretty much every ride share company does the same thing. You need to match riders with drivers, right? If you don't do it fast enough, like I'm going to get pissed off and say, okay, let me check a lift or let me just take a taxi. I mean, we all know like consumers now are like super temperamental. I said, time is money. Okay. The other thing I'd say is like those streams of data, like if you're a data driven company and you're applying machine learning, like that data needs the flow to all these other systems as well. So like Uber was did a lot of early work in traditional

machine learning. They had this platform called Michelangelo, but like that data is flowing there. They're looking at like how do I affect surge pricing to like squeeze as much money out of us as possible. You know, how do I look at driver behavior ratings? You know, when do I add more drivers? Like that all that stuff has to be done on the data. And the faster they can do it, the better the business. So same thing with I would say AI, like the fresher of the data, the better the results. Yeah. And Will, I do want to kind of for our more non-technical audience, I want to put this into perspective because you, you know, you mentioned vector databases and embeddings and in rag, right, retrieval augmented generation. Why are these things so important, right? When when people think,

oh, you know, I can just slap, you know, use an open AI API or I can use a Google API, I throw a chatbot on my company's website and we're good to go, right? But you need that layer of real-time data about your company or what your customers need. So can you explain this concept of rag and vector embeddings and why this is ultimately important when we're talking about generative AI in large language models? Yeah. I like how you started the conversation with saying like, you know, a data an hour late is super late.

I think that was the time increment we're talking about. I would argue seconds late is too late. So, I mean, a great example is imagine the best way to think about this is you have these amazing, you know, general purpose, large language models that are trained to really effectively understand all these facts and communicate with you, the consumer. But if I am an airline, this is a great example, if I'm an airline industry and I'm providing a chat bot, okay, it's not possible at the moment to train a model. so that's constantly up to date to the second. So if I log into the map and say, oh, what's the status of my flight? Well, there's no large language model

that was trained a year ago, a month ago, a day ago, or even an hour ago, that's going to have the relevant information, right? So you need to actually provide that to these models that you were describing, whether it's GPT or LOM or whatever. It doesn't matter. It all works the same way.

You need to provide the information so that I can reason with it. It will, like, for instance, say, okay, well, this person is Willa Forrest. He was on this flight. Here are all the bookings. And this is the status of the airplane. That all has to be provided when you ask the question, right? So that's sort of the criticality of real-time data is that businesses that are looking to use these large language models.

They have to bring it to the table, if you will, when you ask the questions. Because they're not trained on all this up-to-date information. And we kind of just got straight to the heart of today's conversation, but maybe Will, if you could explain a little bit, like, what the heck does Confluent do? And, you know, we all assume it's in the data streaming, but maybe just describe real quickly what it is that you all do. Yeah, so I would say data streaming about 14 years old now. So relatively new in, I would say, data infrastructure world compared to like databases and mainframes. They're still out there people. I know this is an AI conversation. But, um, And it was created by our founders at LinkedIn just to solve this problem. How do we deal with massive scale data and do it in real time? Okay. Confluent was founded about 10 years ago and really the idea is how do we take this technology that the super giant technical companies are all using the Ubers and Netflixes and LinkedIn's

of the world and how we make it possible for mere mortals to use it for their businesses. And so, you know, that's, that's really been our mission to make it as easy as possible to people, for people to do things with real-time data. And this has never been, I would say, never been more important than before with generative AI or for, you know, just the everyday person, if you will, the everyday business. So that's really our mission. We've had all these pieces on top of it. But that's the general idea.

Yeah. And so, you know, obviously you all work with with enterprise companies. and most enterprise companies have had their kind of data game strong for many decades, right? And many companies have been using artificial intelligence for many decades. But how does this kind of, you know, even you said quote unquote newer concept of data streaming, right, been around for a decade and a half? How does this change what's possible for everyone else, right? Even for, you know, medium-sized businesses, mid-market companies, right?

Like, can you talk a little bit about just how much data is truly a valid? available and you know how even those medium-sized companies that are growing can start to leverage this data streaming concept and pair it with AI. Adobe just introduced an entirely new way to create, bringing the power and precision of its creative suite into one conversational experience. Meet Firefly AI Assistant now live in the Adobe Firefly app, the all-in-one creative AI studio. Powered by Adobe's creative agent, Firefly AI Assistant lets you start with your vision, just describe what you want and shape the outcome as it takes form with the assistant.

The assistant orchestrates multi-step workflows drawing on 60 plus pro-grade tools across Adobe Creative Cloud apps, including Photoshop, Illustrator, Premiere, Lightroom Express, and more to help bring your ideas to life. You can also get started with creative skills, a growing library of pre-built workflows for common creative tasks like batch editing photos, creating mood boards, portrait retouching, and creating social variations. Every step the assistant takes is visible so you can refine, redirect, or take over at any time.

You stay in the driver's seat as the creative director. Adobe Firefly AI assistant now in public beta. See it today at firefly.adobie.com. Yeah, it's actually a great question because oftentimes when people hear about like these like massive scale projects going on with data streaming, I like, well, clearly this is not for me. Because like my data just doesn't change that fast.

I don't have that much data. I'm not Uber, right? But if you're doing generative AI, it all gets back to time, right? Time is absolutely critical to how quickly you can make use of data that's in these databases. So let me just step back. Traditionally, the way data worked is it's really been done the same way for, I don't know, five decades, six decades.

Take your data, you stuff it into some place, and then you ask a question and it gives you back a batch of data. Okay. And then you do something with that data. And really nothing's changed. I mean, we've added like new ways to do it faster and do more data and make it cheaper. But it's the same principle. The problem is that doing things in these batches immediately introduces this time it takes to actually make use of the data. So if I'm a, let's just say I'm a business and I have a customer set of data that lives in a database, right? and you know, I'm constantly updating, oh, I have a new customer. Like, here's his address or whatever.

Like, you want that data as soon as it's changed to be available to these downstream systems. And if you use the traditional approaches, you're basically, you have to do these batches. And usually you do them like on an hourly basis. Every hour, I'm going to take all my customer data and I'm going to move it to my, to this vector database that I'm using to provide context for my, generative AI. So now you move to this model that every time I update my customer user base, you know, that that is immediately sent to my destinations and is immediately enriched and provide

context, et cetera. So so that's that's really where I say data streaming helps mid-level. It's not it's not necessarily about the skill. Yes, it can be used for massive skill. So you can grow like open AI, the customer of ours. They do massive amounts of data. Right, but it's about the time. So. So, so, Will, you kind of gave us some great examples. And I love those, by the way, right? Like Uber needs to know whether I'm in the front of my building or the back of my building, right?

You know, we need to know if that Uber has taken a left yet. Right. So some great external use cases or use cases that the everyday person might, you know, need or rely on data streaming. But I like that example that you just gave there, right? Like a customer, you know, changes their address, right? But I want to, you know, maybe kind of explain how data streaming is not just, you know, an external thing, right? Not just for me to know about Uber, but also for internal teams to know, right? Because I think when we think of large language models, I think, unfortunately, we just think of, oh, I'm a consumer using this on the company's website.

So how does data streaming in AI help internal teams work with more real-time data? because I can see in those instances, yeah, last hours data could be very bad to have right now. Yeah. Well, not only that, but generally speaking, if you're using a public model, which almost every, you know, medium-sized business, they're not training up their own large language models. They typically aren't even using the open source models and trying to operationalize it. You talked about like Lama, et cetera. So those models don't even have your business information.

They know nothing about your customers. and you don't want them to know about your customers either. Like, you don't want to send that stuff. There's, like, sensitive information. So you have to provide that to the model when you ask the question. So, you know, actually, it's kind of interesting because if you think about businesses, like startups, maybe not as much, but medium-sized, large businesses, I would say the first use case for Gen.

I. is just increasing the efficiency of employees, right? And this means like, and I do this all the time. Like internally, we have all this information. I want to be able to ask a question rather than like search all these different silos of data, right? And it is, and the only way that works is if it, if that information is provided to the large language model at the time that I ask the question. And yeah, these are internal sets of data. This is like customer information, employee information, product information.

So that's a very, I would say a very common use case, especially even for large businesses. If you're a bank or your healthcare company, it's actually kind of hard to start using these generative AI stuff for your customer facing stuff. There's just so much risk. but you can use it for internal information relatively risk-free, right? Because there's always going to be a human in the loop, right? Yeah, and that's exactly what I was just getting to, right? Because we just talked about all this, this promise of data streaming and marrying it with, with generative AI and how that can be so impactful, both for, you know, real front-end users

and for internal teams. But maybe we'll talk to us a little bit about those risks and challenges, because It all sounds great, but what happens when data streaming goes wrong? But data stream never goes wrong. Numbers don't lie, right? Well, I do like to say, and I don't think, again, if you're not a data person, you're not familiar with the category, like it's not, it's not an exaggeration to say that unless if data streaming one day just woke up and had a bad, bad day, like it didn't drink

its coffee, that literally the global financial service system would just, fail. Like, because like all these banks, the way money transfer, behind all these things, there's data streaming. Like it's just everywhere. It's ubiquitous. So like data streaming is super rock solid, right? So, but of course, anything can fail. But like I think the bigger concern is especially if you're a big business where it's always a risk reward. Like if you're a small business, you don't care that much about potentially what happens if it goes wrong. You're like like, okay, well, maybe I'll just shutter my doors and just start up a new company, right? But like, if you're a bank and you provide false information to Wall Street or you're,

or you leak patient information, that's a pretty big deal. And so I think where it could go wrong, I think this is where I, one of the things that Conflin does is really important on top of data streaming. Data streaming fundamentally is just about delivering data really, really fast and acting on it. But the question then is, if I'm using J.R. AI and I get some sort of response, how did it achieve that response? How do we ensure the wrong data doesn't get to the model and then get sent to the wrong people? Right. Those are really important things. And so that's where there's this category called data governance, which is all about as this data is being produced, I need to look at the data and make sure that like PII information doesn't get. sent through. I need to make sure that, you know, PCI information, that's a banking standard for payments like your personal information doesn't get through. So data governance is important. And the second thing there, I think, is if something does go wrong, and this is where, like,

all the big generative AI players have to look at this. The medium guys do too. Like, how do I know what went wrong and went why? So let's just say I enter and enter. some prompt, I get some crazy ass hallucination that leads people to make bad business decisions, my customers. How do I, if all this data is in real time, it's not in the model, how do I know what that was? So you need this ability to trace what's called the provenance, the source of the data that was made in that decision. And I will say where I think we're going to be going in large language.

Like right now, that doesn't really exist very effectively in the large language models themselves. Like essentially they're trained on the entire internet, right? And it doesn't really track like where the pieces of information are coming from. They kind of don't want to, to be honest, because it opens up a can of worms, which I'm sure you've talked about in podcast. Yeah. So that's another place where it can go wrong. Like you need to know how these answers were provided. Yeah. And speaking of that, a great, great question here from our audience from Cecilia. So she's asking, Will, how do you ensure that data is truly coming from diverse sources so that are getting complete information, how do you discern who is providing the data or maybe what

is that data actually telling the full story? Yeah, yeah. I mean, that is a great question. And it is, it is a challenge that big challenge we have right now in terms of applying Genervéi in an ethical manner. So, I mean, first of all, I wish I had the magic incantation to solve that problem. I think I'm going to speak to how do you know you're getting the complete information and who is providing that data. So again, I think the data that's provided to the large language models, these generative AI models, you can actually track what that is. So this is basically, I'll just simplify the way these things typically work.

Someone enters a prompt. You take that prompt, you get a bunch of other data, you throw it on top of the prompt, then you send it off to the model and you get a response, right? All that extra data that, you can. you're sending to the model, you can track that and you can know what source it's from and you can know when it was created. All that is what's called lineage information. So to have ethical generative AI, we have to do a very good job of tracking that to ensure that you're not getting biased results and that when you do get biased results, they can figure out what the problem is. So hopefully

that kind of answers the question. No, yeah, it does, absolutely. So, you know, one thing that I kind of want to, you know, shift toward, and I, I never put guests really in the hot seat or ask you to tell the future. But, you know, one thing I'm always talking, talking about and wondering about is just this availability of data. And even for, you know, smaller businesses, which we kind of already referenced, you know, but, you know, five years ago, small businesses weren't thinking about using AI until generative AI came a lot. and kind of democratize that. You know, might there be a similar, you know, realization when it comes to even data streaming, right? So where do you kind of see your industry headed as this generative AI landscape is changing so, so quickly, which brings more and more people in using models and wanting data? So where do you see it headed? I mean, I think what there's a couple things we're going to see.

One is, to be honest, like the average company is probably not going to be directly be dealing with data streaming themselves. Because data streaming has become so, so, I would say, inextricably linked with how generative AI is being done, that you're going to be using a service that makes it incredibly easy for you to do this without ever having to touch data streaming. Underneath the covers, like, that service will be doing for you. I mean, there's lots of great examples. It takes something like a notion, AI. I don't know if your listeners, probably some of them are familiar. Like underneath the covers, they're using data streaming.

If I'm using notion for productivity reasons, I don't need to know about how they're using data streaming. So that's one thing. You're just going to see these services do a much better job at connecting to your data sources and doing this all for you. So you don't have to do this crazy rag vector database stuff. Like even that is too much work. It's a lot easier than it used to be like you said, like tremendously, but that's a lot more

work. Second thing I think is we're going to see a lot more industry-specific integrations, training services take place. So if I'm a healthcare company, there's going to be companies that specialize in how do I deal with patient information. That's a very hard problem right now. You know what I mean? So I think we're going to see a lot of industrialization, energy, healthcare, financial services. It's happening already, but it's going to become mainstream. All right. So, Will, we've covered so much in a very short. amount of time, right? From from data privacy and governance to how data streaming is changing both internal operations, external expectations. But as we wrap up here, maybe what's your one

most important piece of advice for business leaders out there, you know, who are really wanting to maximize the power of AI with data streaming? I mean, this is so cliche now, but it's so true. Just remember this. Garbage in, garbage out. Like large language models are great, but if you want to use it for your business, you have to have high quality, well-governed data that's always up to date. Because as you said to start the show, an hour late is far too late, right? You're just going to piss off your customers, right? I don't even want to see data a second late, to be honest. Like, I get, I get cranky, right?

So making sure you get high-quality data that's fresh and up-to-date is absolutely critical. So like the first thing in any major, if you want to make use of generated by it's a data problem. It's always a data problem first. All right. Well, this was, I think whether you're a data geek or just trying to better understand data streaming and how it, how it impacts us all. I think today's conversation was an especially important one. So thank you so much for taking time out of your day to join the everyday AI show. My pleasure.

All right. And hey, everyone. Yeah, we just all got like some kind of degree and data streaming. That was great. If you weren't taking notes as quickly as I was in the background, if you hear, you know, pound, pound on the keyboard, that's me getting our newsletter ready for you. So yeah, we covered a lot. And it's going to be a lot more, some of the most important takeaways. So if you haven't already, please make sure to go sign up at your everyday AI.com.

If this show was helpful, tell someone about it. Yeah, I know everyday AI might be your little secret to make you the smartest person in an AI at your company. but share the wealth. Please, if this is helpful, if you're listening on the podcast, leave us a rating, subscribe to the podcast, and go to your everyday AI.com. Sign up for that free daily newsletter, and we'll see you back tomorrow and every day for more everyday AI. Thanks, y'all.

Meet Firefly AI Assistant. Now live in Adobe Firefly, the Allman One Creative AI Studio. Just describe what you want to create in your own words and the assistant handles the rest, orchestrating multi-step workflows across Adobe Creative Cloud apps, including Photoshop, Premier Express, and more in one conversational interface. You direct the outcome while the assistant accelerates execution. Stand control with the ability to step in and refine at any time. See it today at firefly.adobie.com.

And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going. For a little more AI magic, visit Your EverydayAI.com and sign up to our daily newsletter, you don't get left behind. Go break some barriers and we'll see you next time.