The AI Daily Brief: Artificial Intelligence News and Analysis - Study Finds 370% ROI for Enterprise Generative AI

Episode Date: November 21, 2024

A new study by IDC, commissioned by Microsoft, reveals generative AI delivers an average ROI of 370% in enterprise settings, with AI leaders achieving over 10x returns. Key findings show productivity use cases dominate current AI applications, but a shift toward revenue generation and industry-specific use cases is emerging. Despite these promising returns, lacking employee skills remains the top barrier to wider adoption. Brought to you by: Vanta - Simplify compliance - ⁠⁠⁠⁠⁠⁠⁠https://vanta.com/nlw The AI Daily Brief helps you understand the most important news and discussions in AI. Subscribe to the podcast version of The AI Daily Brief wherever you listen: https://pod.link/1680633614 Subscribe to the newsletter: https://aidailybrief.beehiiv.com/ Join our Discord: https://bit.ly/aibreakdown

Today on the AI Daily Brief, a new study finds the ROI of AI at 370 percent. Before that in the headlines, what Microsoft announced at yesterday's Ignite and what it means for the industry. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show notes. Welcome back to the AI Daily Brief Headlines edition, all the daily AI news you need in around five minutes. Yesterday was Microsoft Ignite. And as you would expect, a huge amount of the emphasis was on artificial intelligence. The company announced something like 80 new products, so we're going to try to quickly go through

and understand what the biggest part of the announcements were. Overall, Microsoft announced a rebrand and redesigned of their Azure AI studio, renaming the platform AI Foundry. The platform serves as a central hub for building Gen. AI apps as well as managing deployments in an enterprise environment. Jessica Hawk, Microsoft's VP for Data, AI, and digital application. said, business leaders are looking to reduce the time and cost of bringing their AI solutions to market while continuing to monitor, measure, and evaluate their performance in ROI,

which is why we're excited to unveil Azure AI Foundry today as a unified application platform for your entire organization in the age of AI. Azure AI Foundry helps bridge the gap between cutting-edge AI technologies and practical business applications, empowering organizations to harness the full potential of AI efficiently and effectively. New on the platform is tooling to help customers deploy and manage AI apps and agents, with Microsoft saying the new management center will help teams manage and optimize AI apps at scale, including resource utilization across multiple hubs and subscriptions, access privilege, and connected resources.

Now, you might have noticed recently that Salesforce's Mark Benioff has been absolutely ragging on Microsoft as basically misleading the world with their assistant version of AI. Salesforce has been pushing agents quite heavily and effectively says that agents are the U.X that AI has always wanted and that the co-pilot era was just a big distraction. Well, in this new AI Foundry, agents were of course front and center with an integrated AI agent service platform within the suite, along with the ability to customize agentic workflows, the platform also supports bring your own storage and private networking features geared towards protecting corporate data.

Hawk said, in a market flooded with disparate technologies and choices, we created Azure AI Foundry to thoughtfully address diverse needs across an organization in the pursuit of AI transformation. It's not just about providing advanced tools that we have those two. It's about fostering collaboration and alignment between technical teams and business strategy. The feature that's catching a lot of attention is the ability to easily test and switch between different models. AI currently supports 1,800 different AI models, all with different capabilities and cost profiles. That is nothing, if not a nightmare to manage, so Microsoft

have made it much easier to find the best model for the job. At least that's their promise. Cloud Computing Chief Scott Guthrie said, what developers are often finding is that each new model, even if it's in the same family of models, has benefits in terms of better answers or better performance on many things, but you might have regressions on other things. If you're a business that has got a mission critical application. You don't just want to flip a switch and hope it works. Guthrie noted that this emphasis on customer choice could harm the company's partnership with Open AI, but commented, for a huge number of use cases, the open AI models are absolutely the best today in the industry. At the same time, there are different use cases and sometimes people do

have different reasons for wanting to use different things. Choice is also going to be important. So taking a step back, what was interesting about this Ignite announcement? Sure, part of it is that agents are all over the place, as you would expect. But really, I think that this is a good signal of the place that we are in when it comes to AI adoption. Rather than just every big event being about the latest model or pushing the boundaries of the state of the art, we're increasingly seeing usability, U.X, practical integrations, basically all the stuff that it takes for enterprises and customers to actually use these tools become the major focus. It's no longer just about who has the most technical magic.

It's about who makes it the easiest to use that for whatever it is that I'm trying to do. That, I think, is where the big competition is going to be in the immediate future. Now, one of Microsoft's big competitors is, of course, Meta. And that company announced a new division specifically for building AI tools for business. The announcement of the group also comes with the announcement of a heavy-hitting leader, Clara Sheep. Up until about last week, she was the CEO of Salesforce AI. She was elevated to lead AI efforts at Salesforce in May 20203, but wrote on X,

I'm thrilled to share today that I've joined Meta to lead a new business AI group. Our vision for this new product group is to make cutting-edge AI accessible to every business, empowering all to find success and own their future in the AI era. Of course, this announcement posts also shows some of the advantages that Meta has. 200 million businesses, she writes, each month turn to Facebook, Instagram, and WhatsApp to connect with billions of customers around the world. Meta's Lama models have over 600 million downloads to date, and Meta AI now has more than 500 million monthly active.

Not to mention, the incredible ways will bring these AI advancements into the physical world through AI glasses and VR headsets. Meta's global reach and leadership in AI represent a generational opportunity for businesses and I couldn't be more excited and grateful to help take this from zero to one to scale. And this notion that Meta has seen massive business adoption of their AI tools

is pretty true and it's of course being driven by the fact that they're being baked into the company's advertising platform. During last month's earning call, CEO Mark Zuckerberg, boasted businesses using Meta's image generation tools were already seeing a 7% increase in conversions. The company also recently added customizable AI chatbots to aid with customer service.

With a new division dedicated to the market segment, Meta is making a big bet that making user-friendly AI tools for businesses is the way forward. Meta's VP and head of monetization, John Hegeman said, We believe these latest advancements in AI represent a significant opportunity for businesses to drive more efficiencies and significantly improve the experiences they offer to their customers. Last news, once again, from another one of these competitors, and this one is just a little one, but Google's Gemini can remember you now. The company's flagship chatbot has a new memory feature.

Similar to ChatGPT's memory, Gemini will now recall. details about the user, including food preferences and interests. The chatbot will then use this context to tailor responses, like only recommending restaurants serving your favorite foods, or responding to coding questions in the correct language. The feature is being rolled out to Gemini advanced subscribers at the moment, and interestingly, Google is making this feature transparent and customizable. Users can view, edit, or delete any of the retained memory. Gemini will also note when it uses its memory as context for an answer. Google claims the store in memories won't be shared or used to train future models. And that ability to customize memory

might be really appealing to some people. For example, Professor Ethan Malik tweeted yesterday, odd, why does my chat GPT advance voice mode keep slipping into an English accent? Ah, because I told it to try accents once and it recorded that in memory. I bet 59% of weird GPT experiences are people not realizing how memory works. The other 41% is chat GPT is just weird. Anyways, friends, that is going to do it for today's AI Daily Brief Headlines edition. Next up, the main episode. Today's episode is brought to you by Vanta. Whether you're starting or scaling your company's security program, demonstrating top-notch security practices, and establishing trust is more important than ever. Vanta automates compliance for ISO-2701, SOC2, GDPR, and leading AI frameworks like ISO-42,1,

and NIST AI Risk Management Framework, saving you time and money while helping you build customer trust. Plus, you can streamline security reviews by automating questionnaires and demonstrating your security posture with a customer-facing trust center, all powered by Vanta AI. Over 8,000, global companies like Langchain, Lila AI, and factory AI use Vanta to demonstrate AI trust and prove security in real time. Learn more at vanta.com slash NLW. That's vanta.com slash NLW. Today's episode is brought to you, as always, by Superintelligent. Have you ever wanted an AI daily brief but totally focused on how AI relates to your company? Is your company struggling with AI adoption, either because you're getting stalled figuring out what

use cases will drive value or because the AI transformation that is happening is siloated individual teams, departments, and employees, and not able to change the company as a whole? Super Intelligent has developed a new custom internal podcast product that inspires your teams by sharing the best AI use cases from inside and outside your company. Think of it as an AI daily brief, but just for your company's AI use cases. If you'd like to learn more, go to Bsuper.a.i slash partner and fill out the information request form. I am really excited about this product, so I will personally get right back to you. Again, that's besuper.a.ai slash partner.

Welcome back to the AI Daily Brief. One of the biggest questions when it comes to enterprise AI, and specifically generative AI, of course, is what the ROI is truly going to be. Now, we're in a period where there is a broad assumption that this technology is so powerful that ROI is basically inevitable, and that frankly, if you're not capturing it, it's probably your fault, not the AI's fault. Whether that period lasts forever remains to be seen, but it's created the context for a lot of pilots. However, inevitably, as organizations get deeper and deeper into their gen AI journey, they are starting to try to figure out how to quantify the benefit that is happening. For example, many organizations are painfully aware that pretty much all of the benefit of AI is

accruing to the individual employee who's getting those productivity gains because the organization doesn't really have any way of tracking the benefit that employees are getting. So it's entirely contingent upon that employee about whether they deploy their saved time to more work pursuits or whether it's just entirely for them. Point being that, I think that enterprises are getting a lot more keen to try to figure out how AI is actually benefiting them in specific numerical terms, and that's why a new study from IDC really jumped out at me. Now, one thing that's important to note, just for the sake of caveating and grains of salt,

is that this study was commissioned by Microsoft, so obviously that is not an unbiased party. Still, in terms of how the study was conducted, it's not like this was pulled out of thin air. IDC surveyed over 4,000 people that they call business leaders and decision makers who are responsible for, quote, bringing AI transformation to life within their organization. Supplementally, they interviewed eight large enterprises about their AI strategies and use of AI within their businesses. Let's talk about some of the key findings and then dig in on a deeper level. First of all, and perhaps most expectedly, generative AI saw a huge jump in usage between

2023 and 2024. It was 55% in 2023, jumping all the way to 75% this year. Also, in the realm of the unsurprising, so far a lot of the emphasis for businesses has been on productivity. The way the IDC frames it is they say that the primary way in which organizations are monetizing AI today is through productivity use cases, and on a worldwide level, the top two business outcomes organizations are trying to achieve using AI are employee productivity and top line growth. This makes sense. Thinking about employees saving time doing the things that they're already doing is a very natural place to start the AI journey. What's interesting, though, is that there does seem to be a shift.

the fact that organizations are starting to think about top-line growth, and also the IDC says in the next 24 months, a greater focus will be placed on functional and industry use cases. Maybe the most eye-popping finding. For every $1 a company invests in Gen.A.I. The ROI is 3.7X across industries. What's more, organizations consider leaders in AI are seeing their investments pay off at a significantly higher rate than the average.

Top leaders using generative AI are realizing a 10.3x return on their investment. Finally, these organizations say the top challenge is a lack of employees with necessary skills and capabilities to utilize AI. I have a lot of thoughts on that that I will come back to in a minute, but let's try to dig in on this ROI question, because that seems to be the one that is really notable. Now, what's challenging about this is that this comes from a survey question, what would you estimate your organization's ROI is for every $1 spent on generative AI projects or initiatives? In other words, this is self-reported, it's estimated, and there's no guarantee that how one

AI officer thinks about how they determine ROI is anything at all like how another AI officer thinks about ROI. This doesn't mean that it's not an interesting data point. Even if they're off, in other words, the fact that these folks are estimating their ROI at 370 percent is in and of itself telling. And this maybe moves us back to how people are using this today. It seems likely that the most common measure is going to be around time saved. IDC writes that productivity use cases are delivering the greatest ROI today. When asked which AI use case has provided the greatest ROI for organization, 43% said productivity use cases, in other words, individual employee productivity and efficiency, such as reducing time analyzing or completing tasks, versus 31% saying functional

use cases, use cases specific to individual lines of business or business functions, and 26% saying that it's industry use cases, such as improved retail ordering or streamlined manufacturing. In terms of where in the organization enterprises are using AI, it ranges from the top line of 90% using it for marketing and PR, makes sense, there's a lot of words and images there, and those are the two most used categories of Gen AI right now, down all the way to product development, which is still at 56%. Again, caveat, given that these were people who were managing AI transformation inside their organizations, this is already a subset of businesses that are probably more AI savvy than the average. All over this survey is the sense that we're transitioning from

productivity alone to revenue generation, from just saving time to improving outcomes. 38% of organizations said that they had a plan to monetize functional use cases within the next 24 months, and 37% said they had a plan to monetize industry use cases. Let's shift over then to what is holding adoption back. In the realm of the expected, once again, security, privacy, and compliance remain major considerations, but by far, organizations identified their top challenge as a lack of employees with the necessary skills. And the numbers didn't come down hugely between 2023 and 2024. In 2023, 52% of organizations said that their top challenge was a lack of skilled workers

versus 45% of respondents saying that now. The next highest challenge category was cost and concerns about data or IP loss, which were down all the way at 27%. I wanted to soapbox for just a minute here, because obviously skills and capabilities is where superintelligence started its journey. It has been very clear for some time that there is a big what we call enablement gap, the space between what an enterprise or organization believes they can get out of, of AI and what they are actually getting out of AI right now. It made sense to us to start the journey

of trying to solve for the enablement gap by trying to improve skills or capabilities. That's where the tutorial version of the superintelligent platform first came from. What we found was not that that wasn't a concern, but that when you really pushed on it, the challenge for employees wasn't just that they didn't know how to use AI tools. It was more that they didn't know what to use them for. Framed differently, you can know all the prompting techniques in the world, but if you don't have any sense of which workflows and business processes could be updated and improved with those prompting techniques, it's not going to move the needle. Most organizations lack a system for tracking broadly and publicly how employees are experimenting with and getting value out of AI right now. What that means is that every experiment that an employee does, every pilot that a team undergoes, really only stands to benefit the experiment, or the pilot participant. There's no way to translate the experience, the learnings, the new

best practices, the new techniques that come out of those experiments in pilots. This means that adoption happens in fits and starts. Every person across the organization is forced to be a use case creator, rather than just copying off the homework of the early adopters and power users who figure it out the fastest. That's why superintelligence shifted so much emphasis to use case sharing and helping organizations broadcast what their teams and individuals are learning about how to use AI to get value to everyone else in the org. The point being ultimately is that organizations aren't wrong when they're identifying that their employee's ability to utilize AI is a big blocker. It's just that we don't believe that the solution is going to be a bunch of courses in traditional

learning and development. It's going to be much more about systems for amplifying and speeding up the process by which business process improvements and new AI-enabled workflows to fuse across the organization. Now, summing up this study, it's clear that it paints a snapshot of an enterprise AI adoption period that is in transition. Gen AI has fully infiltrated at this point the enterprise. If your company is not using Gen AI based on these numbers, you are now significantly in the minority. More than that, for many organizations, they're already past their first or even second phases of experiments. They're starting to be able to measure the ROI of productivity gains and they're thinking about how they can use AI to create new

opportunities for themselves, to generate more revenue. This is exciting because, as I've always said, one of my fears with AI is that companies will view it exclusively as an efficiency technology, a way to do the same with less. I think the companies that win ultimately will be those who view it as an opportunity technology, a way to do more with the same, or way way, way more with just a little more. I think when organizations reframe their goals as capturing totally new and previously unavailable opportunity, that's where we avoid big negative externalities of entire categories of jobs wiped off the face of the planet, and instead we think about how we supercharge all of our people to use AI to create and capture opportunities that simply were not possible before.

I think there are some telling and promising statistics in here that suggests that that's the way that companies are starting to think about this, and that is a trend I can certainly get behind. For now, though, that is going to do it for today's AI Daily Brief. Appreciate you listening or watching, as always. Until next time, peace.