Everyday AI Podcast – An AI and ChatGPT Podcast - EP 232: Creating and Capturing Business Value with GenAI - Insights From HPE

Episode Date: March 20, 2024

Awesome Stuff From Our Partner, NVIDIA -Register for the FREE virtual NVIDIA GTC Conference or buy tickets to the in-person event and fill out this form here: https://www.youreverydayai.com/nvidia-giveaway/Every business leader is wondering: How can you create value company-wide with Generative AI? And how can you actually capture that value and show the impact? Evan has some answers. Evan is the General Manager and VP for AI Solutions & Cloud at Hewlett Packard Enterprise. Join us as we get answers from an HPE leader in the GenAI Enterprise space as part of our partnership with NVIDIA at the NVIDIA GTC conference. Newsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageJoin the discussion: Ask Jordan and Evan questions on GenAI and business valueUpcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: [e-mail] with Jordan on LinkedInTimestamps:00:00 About Evan and HPE05:11 Blindly trusting tools leads to data leaks.08:40 Enterprise search evolving to synthesize and respond.10:46 Cloud providers and software vendors handle basics13:47 Genai tools make analysts more efficient, valuable.16:18 Data is essential for significantly better models.Topics Covered in This Episode:1.  Creating value in business with generative AI2. Adoption and Effectiveness of Generative AI3. Customer and Industry Involvement4. Pitfalls and Successes with Generative AI5.  HPE Collaboration with NVIDIAKeywords:generative AI, business value, ChatGPT, language models, NVIDIA GTC conference, AI Solutions, HPE, hardware, infrastructure, enterprise customers, computer vision, deep learning, NLP, generative AI, customer experience, retailer, search, mobile app, data security, privacy, cost, small model, large language model, AI workloads, cloud, data centers, RAG, enterprise search, document store, software, microservices, venture, data, moatSend Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info)

This is the Everyday AI Show, the Everyday Podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life. Meet Firefly AI Assistant, now live in Adobe Firefly, the All In One Creative AI Studio. Just describe what you want to create and the assistant handles the rest, orchestrating multi-step workflows across Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome. The assistant accelerates execution. How can you create value in your business with generative AI?

Here we are. We've all been using chat GPT and so many large language models. And we're still trying to see, is this working? And how can we keep it working across our entire business? That's what we're going to be talking about today and more on everyday AI. Welcome. Thank you for tuning in. You might notice this is a little different setup.

That's because we are live at Invidio. GTC conference, and I'm excited today to talk about creating value in your business. So make sure, before we get started, if you haven't already, go to your EverydayAI.com, sign up for the free daily newsletter. We'll be recapping it right after this episode. So please help me welcome. We have Evan Sparks, the GM and VP for AI Solutions at HPE. Evan, thank you for joining the show.

Thanks so much for having me. Really excited to be chatting with you. All right. Can you tell us a little bit about you and your role at HV. Yeah, absolutely. So at HPEI oversee our AI Solutions business, which is really about kind of combining the best of what we do on the hardware infrastructure side together with NVIDIA with some software assets that we've got that we've been building over the last few years to help enterprise customers in particular achieve their goals with respect to AI. Historically, of course, we've been doing a lot of computer vision and kind of classical deep learning in NLP, and lately a lot of that has become generative AI. And so thinking through what are the places where we can create a lot of value in the enterprise with generative AI and how do we help customers accelerate the realization of that value?

I really like to know from your perspective, why do you think it is still a struggle for so many businesses to realize if generative AI is working? Because it seems like you see all these studies, you know, you can get back X percentage of your time and, you know, 5x this, 10x this. Do you think that a lot of companies are still struggling to see if they're actually getting value out? Or do you think now that we're a year and a half into this large language model world, our business is finally able to say, we're getting value out of this? I think back to November of a year and a half ago when we had our chat GPT moment hit. And I really think that that was the moment where all of our imagination got captured by what this technology could do. But the practical reality of getting that implemented in the context of business processes,

really identifying where are the places we can save time and money or create new customer experiences. That was really worked out over the course of 2023. So we saw a lot of big companies doing pilots in 2023. It was sort of the year of the prototype. And I think 2024 is the year where we're going to see a lot of these experiences move into much more kind of production environments and start to see a lot more scale come. I think that, you know, we talk to customers about how they're seeing this value get created. And a lot of them will want to have a customer-facing experience among their customer set. So talk to a large retailer yesterday who embedded generative AI right in search on the front page of the mobile app.

And it was a totally different customer experience, but it was something where their customers could enjoy a much more valuable shopping experience. They're also doing things to help the workers in their warehouses or in their stores be more efficient with their time by helping them find products faster or these sorts of things. And then they're also thinking about how do I make business processes much more efficient? Generally, I can help in all of this, but prioritizing those things and then getting it right from a safety, security kind of perspective, that is hard stuff. And we're trying our best to accelerate them in those efforts.

Yeah, Evan, I definitely want to jump in and talk about those a little bit more. But our audience is from all over the spectrum, from small business owners to people who work in big tech. So maybe first, before we dive in a little bit, can you tell us just the type of companies? I'm sure it runs the whole gamut. But what type of companies are you generally working with that? Yeah, so I mean, a big multinational company,

and we sell from small to large. We definitely have a big footprint in the Global 2000 and in on-prem data centers for sure. But we're also working with a number of, I look out over the show floor, and I see a number of customers in cloud service provider land, in large global 2000, but also in the mid-tier. So we're working across the board.

And I think that our view really is that AI is fundamentally a hybrid workload. And we're going to see people choose to run some AI workloads in the traditional cloud, some in their data center, some right at the edge. We want to be our corporate strategy is to be an IT provider that helps people run technology wherever it makes most sense for them. So, you know, you kind of talked about this transition, you know, from 2023 into 2024. By, you know, you having worked closely with so many companies, what are some of the biggest maybe mistakes that you saw companies make when they are trying to create value with generative AI? What are some of those common pitfalls?

I think the first thing was just blindly trusting the tools and thinking, oh, not thinking about the data security and privacy implications. I think especially last spring, we saw some high-profile cases of accidental data leakage by going out to some of these web service providers that we're offering the models. I think companies are getting more attuned to that being something that they have to watch out for, particularly if it's, you know, if it is some marketer trying to write a writer marketing copy or something like that, that's not corporate secrets that you're worried about getting out there. But when it comes to your HR data or your really sensitive trade secrets, that's where you really want to have some protection around that. So that was one mistake we saw people made. The other was sometimes the cost of these things.

It's just really, really high. And so if you don't do the math ahead of time around, okay, how much usage do I think this particular feature is going to get? And how many queries do I have to fulfill and so on? It might break the bank really quickly. So we saw that happen and then people start thinking about, okay, how do I maybe use a smaller model, tighter model? They're going to give me the same level of accuracy,

these sorts of things. But they tend to be not first design principles. They tend to be afterthoughts, as kicked off a successful prototype of the application. And sometimes that's okay. People have to see what's possible before they decide they really want to invest in making a track. And I think one of the things of making something practical

is finding the right provider or partner. And speaking of that, we are here at NVIDIA, I think they're probably one of the best in the world. Can you talk a little bit about how you and your clients work with Nvidia and what that enables you all to do that maybe many years ago wouldn't have been possible? Yeah, so I think we've been a deep partner with Nvidia at HPE for many years, I mean, decades at this point. And probably the first high-density GPU server that got created was an HPE box.

But over the last few years, we've really seen Nvidia. become the leader specifically in silicon, but also in the lower layers of software for all kinds of AI, specifically generative AI. And so we've really leaned into partnering at the ecosystem level with Invidia and saying, okay, you want the best way to run this open source language model? There's probably an NGC container for that.

Let's embed that directly in our user software experience. We've been, we announced a technology preview of a product today in the inferencing software that will leverage under the hood, a bunch of the NIM microservices that Antonio talked about on his keynote at the show yesterday or the day before. And so there's a number of ways that we can partner with the video down to the level of that hardware integration all the way up through the software stack. So we're really pleased to be participating as part of this ecosystem. Yeah, and I want to go back to some of these actual use cases because I think that's where, you know, people that

haven't found or companies that haven't found value can learn from others and try to implement it themselves. So you mentioned as an example, you know, a client that, you know, just kind of launched a new initiative on their website, but they also had things on the back end. Maybe could you talk us through, where are, you know, maybe for those smaller enterprises out there who still haven't yet, you know, fully adopted degenerative AI, where should they be looking both on the consumer side and on, you know, the back end for their own teams? I think probably the clearest example I can come up with right now that is that is pervasive across industries, it's enterprise search. So, you know, it used to be a model where you'd have your enterprise document database behind

something that looked kind of like a Google search. You know, that wasn't exactly it. You search for the keyword and you get the document back. Okay. It was never as good an experience as Google, but you at least had an index on those things. Now, because of the tools that people are using, they're actually starting to expect. I want something that is going to synthesize a response from me out of my enterprise document store and answer questions that span HR to sales marketing and so on and know about my products and my terms and so on. We've piloted a number of these uses cases internally ourselves as well, and we've seen lots of customers adopt these.

And so we're starting to see these rag-based applications become a conventional architecture that customers are asking us to deploy. Unfortunately, those architectures have like 15 components just in software land to stand up. But one of the things that Vita is doing really well with its retriever microservices is giving customers a blueprint for that. And we do our best to fill in the blanks on some of the technology choices there and offer something that's going to be robust and scalable. But that's an example of an application where AI-powered search shouldn't just be on documents that are on the open web. It should also be on documents that are relevant to your business, but you want to be able to protect those in all the right ways. So that's been a really good use case.

Yeah, so you talk there about RAG and how important that is to bring company data into the picture. Even as we talk about going from 2023 to 2024, I mean, is RAG going to be what a lot of these enterprises are focusing more attention on in 2024 as we look forward in the future? Or where should enterprises be focusing on? right now because it seems like if you're not looking ahead, the pace of the technology so fast, it feels like you can get left behind. Yeah, I think so there's table stick stuff. And I think that actually for the table stick stuff, you're going to end up with a lot of cases where the large cloud providers and maybe the big software vendors are doing commodity.

So if I auto-complete my email, probably that's going to be handled by my email provider, similar support tickets and so on. But every enterprise is going to have applications that are really unique to its business that is built on their data, their unique data assets. They're 30 years or however long of doing business and insurance or in oil and gas exploration or in defense. And those applications, I think, are the areas where enterprises are going to be able to build a defensible mode. So I don't think there's a one in size fits all answer. Unfortunately, I think the real answer is treasure your data, really,

introspect, where can I create an intelligent application that only I can create and use this as a way to provide additional value to my customers? It's hard work. It requires a lot of thought, but I think that's where we're going to see the most value get generated over the next five years. Yeah. And speaking of value, it seems like that's where everyone's focusing on, right? Everyone knows it's a big investment to make both financially and time-wise and to generative AI across the enterprise. But how can companies or how should companies be looking at measuring the return? Is there a good formula to say, hey, this big investment is paying off and what should companies be looking at? Well, top line and bottom line. Those are my two big numbers

that I think most companies care about. A lot of these efforts should directly translate into sales, more sales of your product, faster, better customer experiences, higher on PS scores, etc. That can be one important measure. The other is cost savings. If there are things that that you are using an army of people to do that can now be done by a smaller set of analysts or associates. Invest there, that doesn't mean those jobs go away forever. It means those people maybe can be retrained and start adding value to your business in other places.

And that's really what we hope to see this to sell go when it comes to creating efficiencies. It's about leveling up ourselves as a society. I think about farming, for example, do we want to stop giving people plows because we're worried about their happy people in the field, or do we want to start harvesting a lot more food for everybody so that's important? I think we're hoping for a future that's a market. So that's the value that I'm hoping. What could that look like? Because I think that,

you know, some companies, even ones that we've talked to on the show, you know, are sometimes hesitant, especially smaller businesses because they don't know what it looks like if AI works. So what is the equivalent in business of, you know, kind of your analogy of the farmer in the plow? Is it just, you know, maybe people who are doing a lot more knowledge to build AI systems or to collect better data or what does it look like in the enterprise if AI works? I think it is, it can be hard to put your finger on it, but it's one of those things you know when you see it. When you, there was a recent study out of BCG and APS that talked about analysts in an entry-level

consulting class. And half of them were given Gen. AI tools and half of them weren't. And what they found pretty quickly, the half that were given those tools to do their job were about 30% more efficient, started performing at the level of the second year associate very, very rapidly.

And that's the natural ABE test to say, okay, I have more of these people. If I'm a client of one of those businesses, do I want a bunch of second year associates or do I want a bunch of first years? I would much rather take the second year, and I'll probably even pay a little more money for that. And so I think that's kind of value that can be generated here and that's really measurable for these kinds of businesses.

You know, earlier we talked about common mistakes that you've seen, you know, having worked with so many businesses implementing generative AI, what are some common wins that you've seen that maybe companies out there might not be thinking of? Yeah, I think the common, we talked about the common mistakes. I think the common wins come from one, standardizing, on tools across the enterprise that will allow you to iterate rapidly. We all know this is a really fast-moving kind of part of the industry right now, and the tooling is changing on a daily basis.

And the nice thing about AIM is that if you give them more data over time, they get better. And so leveraging what's going on in the open source and the new models that are coming out in the leaderboards and being willing to drop in the latest and greatest throughout your development cycle, but then also being able to inform those with with the company's own and error can really drive optimized return. And so that nimbleness and that building for iteration ends up being a really positive pattern among companies that we see that are successful with this. Common theme I keep hearing is data, right?

Not just in this conversation, but in so many conversations. Have you seen in your experience over the last couple of years, especially, you know, after this chat, GPT boom, Is there a bigger focus on data than there was before? Because we've been told for a decade data is the new goal, but now that people are seeing what data can create with generative AI, is there even more attention being paid or should more attention be paid to data? I think for sure.

I think data is the fuel for these models in a lot of ways. We have seen pretty scientifically documented, at least the evidence, suggest that order of magnitude and more data leads to significantly better models. people are going to be looking for more and better data that they can feed these models as they over time. It's not going to eliminate the need for better modeling and better thinking and so on, or even choosing the right application to go after in the first place, but to a first approximation data rules.

Yeah. And Evan, just as we wrap up, you know, today's show, because we've talked about a lot from from RAG and guard rails to, you know, how you can actually measure business value. What is your one most important takeaway for those listening out there in order to create and capture value that generative AI can bring? Think about where you have a moat in your data and if I'm allowed to iterate rapidly. Those are my two key tables. I love it. I love it. Well, hey, there's going to be a lot more in today's newsletter.

So make sure you go to your Everyday AI.com. Evan Sparks, thank you so much for joining the Everyday AI show. Absolutely. Thanks, I'm all right. Thank you. And hey, we'll be here back all week for more at GTC. So thanks for tuning in and we'll see you back tomorrow and every day for more Everyday AI. Thanks. Meet Firefly AI Assistant. Now live in Adobe Firefly, the Allman One Creative AI Studio.

Just describe what you want to create in your own words and the assistant handles the rest, orchestrating multi-step workflows across Adobe Creative Cloud apps, including Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome while the assistant accelerates execution. Stand control with the ability to step in and refine at any time. See it today at firefly.adobie.com. And that's a wrap for today's edition of Everyday AI. Thanks for joining us.

If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going. For a little more AI magic, visit Your EverydayAI.com. and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.