The AI Daily Brief: Artificial Intelligence News and Analysis - 67% of Enterprises Scaling Generative AI Pilots

Episode Date: September 4, 2024

Deloitte has released its latest “State of AI in the Enterprise” report, highlighting that 67% of companies are increasing investments in generative AI due to strong early results. However, scaling AI pilots into full production remains a significant challenge. Tune in for a detailed analysis of the report’s key findings, the obstacles enterprises face, and what this means for the future of AI in business. Concerned about being spied on? Tired of censored responses? AI Daily Brief listeners receive a 20% discount on Venice Pro. Visit ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://venice.ai/nlw ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠and enter the discount code NLWDAILYBRIEF. Learn how to use AI with the world's biggest library of fun and useful tutorials: https://besuper.ai/ Use code 'podcast' for 50% off your first month. The AI Daily Brief helps you understand the most important news and discussions in AI. Subscribe to the podcast version of The AI Daily Brief wherever you listen: https://pod.link/1680633614 Subscribe to the newsletter: https://aidailybrief.beehiiv.com/ Join our Discord: https://bit.ly/aibreakdown

Today on the AI Daily Brief, where enterprises are with generative AI as we kick off the fall. Before then, in the headlines, what Wall Street thinks about AI currently. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show notes. Welcome back to the AI Daily Brief Headlines edition, all the daily AI news you need in around five minutes. Last week, we did a show about the five questions that would shape the near-term future of generative AI, and one of the big ones was how Wall Street would think about Gen A.I. heading into this autumn season. Reuters did a little bit of a summary of the market as a whole, summing it up by saying

tech market values fall on AI costs and recession fears. And basically the idea here was that in August, there were two big reasons, or seemingly two big reasons at least, that companies like Alphabet and the other magnificent seven companies had a bit of a struggle. One was macro, a rising risk that the market was placing on the possibility of a recession, but the second was the narrative concerns around the ongoing and even escalating AI infrastructure buildout cost. One of the best examples of this was Nvidia, whose share price fell by 7.7% in the last week of August. That came after it projected third quarter gross margins below estimates and reported revenues that only slightly outperformed investor expectations. Interestingly, Reuters framed it as only met expectations, even though it was

actually a couple billion dollars over analysts' average estimates. Still, only being a couple billion over, not being some huge blowout, did feel like it met expectations. One counterweight to this came from meta that showed some ROI from AI investment based on strong digital ad revenue growth. Still, by and large, the narrative heading into the fall is summed up by this Bloomberg piece, shorts are circling some of the AI boom's biggest question marks. Bloomberg writes, It's the story of so many stock market manias. A transformative technology juices a few companies, a bunch of more questionable outfits follow in their wake, and Wall Street buys it all. Then time sorts out what's real from fake. Now, this specific report looks at a group of companies

that have been recently targeted by bearish research reports. Hindenberg research, for example, went in on Server Maker Super Micro, ripping around $10 billion of the company's $36 billion market capitalization off the top last week, and other companies like Lumen and Symbiotic have also faced bearish reports, although perhaps nothing on the same level. Now, interestingly, this report makes it clear that there is a big difference between the invidias of the world, even though Nvidia did fall last week, and the hangers-on that benefited from the tailwind set by companies like Nvidia. Mahoney Asset Management CEO Ken Mahoney characterized what the market is doing as separating winners and losers, pointing out, quote,

"...invidia and the other big AI plays in The Magnificent Seven have been largely consistent with results in growth. That's what sets them apart from the likes of Super Micro, which have been sporadic. Hindenberg came back again on Thursday with another report with an even more dramatic accusation against iLearning Engines' holdings, accusing it of faking its financial figures. While the company denied it in a release, shares were still down 53%. One of the things that I've tried to make clear on this show is that Wall Street repricing some of these companies is not the same as AI being the next great bubble.

John Belton of Gabelli funds put it this way. The stock market always has a way of moving too far too fast and then going through a period of digestion. For a lot of these companies, we're in a healthy period of digestion. Now, one interesting little bit of speculation is at what point some of these big private market AI companies will have to go public. Corey Weinberg from the information wrote a piece this week called Why OpenAI Needs an IPO that outlines a set of reasons that the company might want to consider something like this in the future. That includes the incredible capital intensiveness of OpenAI's business model, the potential that private capital might be running dry or might be

problematic, with the U.S. not necessarily loving a big dose of funding from a sovereign wealth fund in the Middle East, for example. And while Corey is clear that he's, quote, thinking about the following set of facts rather than acting on any particular news tip, I wouldn't be surprised if we see a little bit more of this chatter in the months to come. Staying on news from these public market companies touching AI, people are digging in a little bit more deeply to METAs recently announced AI numbers. We discussed last week that Zuckerberg had said that META's AI tools have more than 400 million monthly users and more than 185 million weekly users, but some wondered if that was driven by accidental usage, given that AI rolled out as just an extension of the search bar.

Microsoft is dealing with its own challenges. From the Wall Street Journal, Microsoft rolled out AI PCs that can't play top games and there's no quick fix. The short of this story is that in the process of getting a new generation of chips that are good enough to actually run these AI applications, Microsoft has made some of its most cutting-edge laptops incompatible with games like League of Legends and Fortnite, which are made instead to work with Intel's X86 chip. An independent research analyst tested 1,300 PC games and found only about half ran smoothly on the new AI-focused PCs. Now, obviously, it seems like this will likely not be all that much of a problem in the long run, but in the short run, it's another sign of the potential bumpiness

of adoption that we're likely to experience. Lastly, today in the headlines, a follow-up from our long read about the homework apocalypse. Axios pointed out that even beyond AI, schools are having a rethink when it comes to homework. In 2012, for example, 21% of 13-year-old students said that they had no homework assigned, in 2020 that figure had gone up to 29%, and by 2023, it was up to 37%. Another study they referenced found that 67% of high school students cited homework load as a major source of stress, which went up to 80% among those doing three or more hours of homework a day. Then there's recently a bill that was passed by California's legislature that would recommend that school districts evaluate the mental and physical health impacts of homework assignments

as well. One assembly member, Pilar Shiavo, said, quote, as a single mom, I only have a couple of hours with my kid at night before they have to go to bed. Spending most of that struggling to get homework done creates a lot of stress on a family. The point here is that it's not just AI forcing this conversation, although certainly AI is forcing it a lot faster. For now, though, that is going to do it for today's AI Daily Brief Headlines edition. Next up, the main episode. Today's episode is brought to you by Fractional. When we wanted to build an AI powered feature of superintelligent, our AI ToolFinder, I went straight to Fractional. The Fractional team is a group of senior engineers in San Francisco working on some of the most exciting projects and applied AI. Working with them is basically like hiring an absolute top flight AI engineering team, but in a way that you can customize exactly for your particular needs. Like I said, that AI Toolfinder feature that we built with them

is already a key part of the superintelligent platform and we are working on something new as well. Fractional works with everyone from startups to the Fortune 500. And to request a free consultation, you can go to fractional.AI. If you want help identifying and building AI projects for your business, then I highly recommend you hit pause, open a web browser, and go to fractional.com. To request a free consultation. Today's episode is brought to you by Venice. The leading AI companies store your entire conversation history and attach it to your identity forever. That's every question you ask, every answer you receive, every image you generate, every thought you share with the machine it's all being spied on. If you trust all the company's hackers and

NSA board members that will ever have access to your AI conversations, then rejoice, for you are well served. For the rest of us, Venice is an alternative. Venice is a powerful AI app for text, image, and code generation that respects you as a sovereign individual and believes privacy and free speech are not only human rights, but necessary for civilizational advancement. Private, permissionless, and uncensored, you can try it for free without an account. AIA Daily Brief listeners receive a 20% discount on Venice Pro. Visit Benis.a.ai slash NLW and enter the discount code, NLW Daily Brief. That's NLW Daily Brief, all one word. Today's episode is brought to you by Super Intelligent, which is of course our platform that helps you learn how to use AI tools and perhaps even more importantly, gives you ideas

on the best use cases that are actually going to help you achieve whatever it is you want to achieve. To recognize the end of summer and back to school slash back to work, we are running our best promotion ever when you sign up for Super Intelligent between now and the end of August using code so back, your first month will be 100% free. The platform features over 600 fun, highly practical AI tutorials that get you using AI fast and with an eye to actually transforming how you get things done. We've just launched Super for Teams. So if you have a group of people at your company that want to figure out how to use AI together, I highly suggest you check it out. But for those of you who are using superintelligent as an individual, once again, if you sign up for superintelligent between now and the end of the month using code so back, you will get your first month 100% free.

Go to be super.a.I. And check it out today. Welcome back to the AI Daily Brief. In the headlines today, we discussed how Wall Street is kicking off this fall when it comes to attitudes around generative AI. and so it only makes sense that for our first main episode of September, we're looking into AI in the context of the enterprise, and specifically where generative AI sits within companies now after nearly two years since the launch of ChatGBT.T. Every quarter, Deloitte releases an update to its state of generative AI in the Enterprise Report. This latest report that just came out covers the period between May and June of this year. They call it moving from potential to performance and sum up, investment is increasing, but the clock is ticking to scale and create value.

So let's discuss what their big findings were. Broadly speaking, one of the things that you're starting to see everywhere is that we're moving out of the technology novelty phase of AI adoption. Individuals and organizations are both getting more nuanced and sophisticated in terms of how they're thinking about using generative AI, and along with that is coming more expectation and less do-eyed interest in anything anyone has to sell them. Unsurprisingly, a big theme for this report is organizations wanting to to move beyond pilots and into a phase where generative AI is actually driving real business value

inside their companies. Deloitte sets up a bit of a path diverging in the wood analysis. On the one hand, as you'll see, there is a ton of movement shifting from pilots to deeper integrations, but as they also point out, quote, the clock is ticking for organizations to create significant and sustained value through their generative AI initiatives, promising pilots have led to more investments, escalating expectations and new challenges. During this pivotal phase, C-suits and boards are beginning to look for returns on investment. There is a chance that their interest in generative AI could wane if initiatives don't pay off as much or as soon as expected. It would take a whole episode to discuss why I think that enterprises are largely in a situation where it would be much, much riskier

to turn away from experimentation with AI than to accidentally overspend on it. But for now, let's just stick closely with what Deloitte is presenting here. So what is actually going on inside these companies that are testing AI? A couple big blinking banner headlines for me. First, two-thirds of companies are increasing their investments because they've seen strong early value to date. That is an extremely telling statistic that suggests there is much more than just hype going on inside these organizations. What's more, the benefits that people are finding are getting more diverse. On the one hand, improved productivity and efficiency as well as cost reduction are still both the top benefits sought by organizations, as well as those most cited as the most

important benefits achieved to date, but importantly, 58% reported they realized a more diverse range of most important benefits, such as increased innovation, improved products and services, or enhanced customer relationships, meaning there's a certain growing sophistication that's not just about ROI measurement. Something that we discuss a lot at Superintelligent is the way in which the AI transformation is proceeding through a sequence of steps. The first step, the low-hanging fruit, if you will, is personal productivity among employees. This is where a ton of the experimentation is happening now. When some individual is just figuring out if their work happens faster, if they integrate chat

GPT with their email writing process, that's sort of this first step. The second phase is all about unlocking new opportunities that weren't possible before. Totally reconsidered ambition, for example, around a marketing campaign because things that were simply outside the capability set of the participants are now firmly within their grasp. This to me is what this 58% reflects when they see increased innovation, improved products, and services. This is more than just personal productivity. its wider benefit. Then, of course, there's an even higher level stage, which is organization-level

transformation. Some organizations have gotten here, but by and large, this remains something for the future. This is all about how we might see organizations redesigned and reimagined from the ground up based on the new capabilities that AI enables. I think where you're likely to see this happen first is actually with entrepreneurs who create totally different types of organizations and model how you can do more with less or much more with the same, and slowly those types of transformations will find their way into the enterprise as well. In any case, again, two big banner headlines here. Two-thirds of organizations are increasing their investments in Gen AI because they're seeing strong value, and nearly 60% are seeing benefits that are not just about productivity and

efficiency or cost reduction, but are about innovation and improved offerings. There is lots of challenge here, though, as well. 70% of the organizations surveyed said that their organization has moved 30% or fewer of their Gen AI experiments into production. In other words, there does seem to be a big scale barrier. Those barriers, come in a few varieties. One is around data. Fifty-five percent of organizations told Deloitte that they're avoiding certain Gen-AI use cases because of data-related issues. Some are worried about regulatory uncertainty. Three of the four things holding organizations back from developing and deploying Gen-AI tools are risk regulation and governance issues. And then, of course,

there's the old chestnut of ROI measurement. More than 40 percent of respondents said that their companies are struggling to define and measure the exact impact of their GenAI initiatives, and less than half said they're using specific KPIs to measure performance, with many standard measures of success not currently being applied. There is also definitely a broad sense here among these organizations that they do not feel prepared. Only 45% think that they have the appropriate technology infrastructure. Only 37% said that they feel they have the appropriate strategy.

Only 23% said they have the right risk and governance systems, and only 20% said they have the right talent. For the sake of this report, Deloitte decided to hone in on two particular areas around data and governance and risk and compliance. The TLDR on the data side is that data has come even more to the four thanks to generative AI. Seventy-five percent of organizations surveyed said that they've increased their tech investments around data lifecycle management. Data-related concerns are frequently cited as holding back Gen. AI implementations. And when it comes to risks and regulation, as we said, these are meaningful hindrances as well.

And this is me editorializing a little bit, but likely to get more so, especially as the U.S. starts to deploy state-by-state type legislation like SB 1047 if it gets signed by Governor Gavin Newsom in California, as opposed to federal regulations, which might be a little bit clearer. Now, of course, even beyond these specific risks, there was never going to be a period of attempting to move from pilots to scale, where the challenges of measurement weren't going to start to become a big issue as well. The story that this report tells is a lot of experimentation around AI-R-OI tracking. Forty-eight percent of those surveyed have used specific KPIs for evaluating Gen-AI performance, 38% have attempted to track changes in employee productivity.

34% have tried to track non-financial benefits. Only 6% are doing none of these types of things. So clearly there are experiments here, even if there aren't really best practices yet. My best guess is that we are going to see a continued period of experimentation around ROI measurement. I think that Deloitte is right to call out that there is a world in which the difficulty in measurement

leads to a hindrance in organization's abilities to actually scale these AI pilots. Clearly, we're already seeing some of that. However, I believe that the continued pressure on basically everyone from a director-level position up to have some sort of AI adoption strategy will actually produce an incredibly fertile set of experiments around AI-R-O-I measurement that might get us to some of those best practices a little bit faster than we might anticipate. If you are interested in going deeper and checking out past reports, I will include a link

to the overall webpage for this on Deloitte.com. For now, thanks to the team over there for preparing another interesting report, and thanks, as always to you guys for listening or watching. Until next time, peace.