The AI Daily Brief: Artificial Intelligence News and Analysis - Is the AI Revolution Losing Steam?

Episode Date: June 3, 2024

That's the argument in a new WSJ piece: https://www.wsj.com/tech/ai/the-ai-revolution-is-already-losing-steam-a93478b1 In this episode, NLW goes section by section through the argument -- that AI is reaching a peak of performance, that it's too expensive, and that uses are limited, discussing where he agrees and disagrees with each one. ** Join Superintelligent at https://besuper.ai/ -- Practical, useful, hands on AI education through tutorials and step-by-step how-tos. Use code podcast for 50% off your first month! ** ABOUT THE AI BREAKDOWN The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://aidailybrief.beehiiv.com/ Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@AIDailyBrief Join the community: bit.ly/aibreakdown

Today on the AI Daily Brief, we're asking if the AI Revolution is already losing steam. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show notes. Hello, friends, quick note before we get into the episode today, the main part of the episode, which is a meta-analysis, sort of a narrative watch of sorts, got much longer than anticipated. Stack that with the fact that the headline news was kind of sparse today, and I decided to just make the main part of the episode the entire episode.

Presumably we will be back with our normal format tomorrow, but I wanted to just give you a heads up. Also, in lieu of smashing a super intelligent ad in the middle of that, I will just note that if when you listen to this, you come away convinced as I am, that the uses of AI are not, in fact, limited, and in fact it represents one of the most quickly adopted workplace technologies we've ever seen, I highly encourage you to check out Superintelligent, our platform for helping people learn how to actually use and take advantage of AI. If you are interested after you check it out at Busper.A.I. Use code podcast for 50% off your first month. All right, with that, let's do this episode. Welcome back to the AI Daily Brief. Today we are doing a narrative watch. And for those of

you who aren't long-time listeners to my show, these narrative watch episodes are basically where I look at an emerging discourse happening in the space to try to understand how the meta-conversation around AI, in this case, is evolving. I think it's a good way to understand how people are interacting with the industry, not just what's happening in the industry itself. And Today we're looking at a Wall Street Journal piece from this weekend called The AI Revolution is already losing steam. Now, this is hardly the only piece that is making this point currently. I think especially last week's earning cycle has really kicked this narrative up.

And so what I'm going to do today is go section by section through this and almost give an assessment of how accurate I think it is. I will also point out this piece is by Christopher Mims. He's someone I've talked to at the WSJ before. You've occasionally heard me get frustrated with people for being demagogues who are just looking for evidence to support their own arguments against AI. Christopher certainly doesn't count as that. He's a great reporter who's just trying to understand what's going on, and this is 100% my take on where I think this particular narrative is getting a little bit ahead of itself.

So to kick off, Christopher writes, Nvidia reported eye-popping revenue last week. Elon Musk just said human-level artificial intelligence is coming next year. Big Tech can't seem to buy enough AI powering chips. It sure seems like the AI hype train is just leaving the station, and we should all hop aboard. But significant disappointment may be on the horizon, both in terms of what AI can do and the returns it will generate for investors. Now, right away, we have an important thing that we're going to have to keep in mind throughout this conversation that what AI can do and the returns it will generate for investors are potentially very different things. And to some extent, a big part of the conversation

and why we're having this right now is Wall Street. You have to remember that for the last two years, really ever since the rate tightening cycle began, most of the tech world, and really most of the investing world, was dealing with the painful unwinding of Zerner. Zerp era policies. The one counterbalance to that was the incredible excitement around AI, which bolstered the entire market even through things like geopolitical conflict. In other words, AI has had a disproportionate impact in the market over the last couple of years relative to what you would expect, and that's certainly part of the context for all these conversations. But let's start with the first section that Christopher writes the pace of improvement in AI's is slowing. Effectively,

Christopher repeats the argument here that most of the improvements coming in today's models are about just getting more data into them. If that's the case, however, and simultaneously we are running out new data to suck up, as Christopher puts it, there aren't 10 more internet's worth of human-generated content for today's AIs to inhale. Does that mean we're going to come up against some natural limits? Mims writes, to train next generation AIs, engineers are turning to synthetic data, which is data generated by other AIs. That approach didn't work to create better self-driving technology for vehicles, and there's plenty of evidence it will be no better for LLM, says Gary Marcus, a cognitive scientist who sold an AI startup to Uber in 2016.

Marcus continues that while AIs like ChatGPT rapidly got better in their early days, for the past 14 and a half months, we've only seen incremental gains. Says Marcus, quote, The truth is the core capabilities of these systems have either reached a plateau or at least have slowed down in their improvement. One can be forgiven for taking this point if you use GPT4 as the benchmark. GPT4 came out early in 2023, and most of the time since then has been all the other non-open AI companies rushing to catch up with it.

This has created a lot of discourse around whether we're reaching some natural asymptote in the capacity of these LLMs. Now, I think one thing that is worth contextualizing here is that while I said that Christopher Mims is a person who hasn't made his career, trying to be a critic and or de-hyper of AI, Gary Marcus absolutely is. Marcus appears to be annoyed at the AI industry itself for hyping things. Andy appears to be annoyed at the AI safety movement who's focused on X-risk, for not focused on the issues that he thinks are more important. All of this is not to say that Marcus isn't someone worth listening to. It's just important to have the context of where he's coming from.

In other words, I would argue that he is looking for evidence that support his priors, rather than just trying to understand what the evidence is saying in general, and his priors are that AI is overhyped. I think, however, that one of the big X factors here is what is actually going on behind the scenes at OpenAI itself. These guys were testing GPT4 18 months ago. They only didn't release it until a year ago because of red teaming, And so either one of two things is happening.

Either one, there really are some limits being reached internally, and there just hasn't been that much progress during those 18 months. Or two, Open AI is deliberately slow playing increased capacities. We obviously don't know the answer until OpenAI shows their hand a little bit more, but I will say that it's notable that Sam Altman has increasingly been saying that they're seeing no evidence that they're reaching some limits. Let's listen to this recent interview. We don't expect that we're near an asymptote, but

you know, this is like a debate in the world and I think the best thing for us to do is just show, not tell. You know, there's a lot of people making a lot of predictions and I think what we'll try to do is just do the best research we can and then figure out how to responsibly release whatever we're able to create. I expect that it'll be hugely better in some areas and surprisingly not as much better in others, which has been the case with every previous model. But this feels like the conversation we've been.

had every other model release. You know, when we were going from 3 to 3.5 and 3.5 to 4, there's a lot of debate about, well, is it really going to be that much better? If so, in what ways? And the answer is there still seems to be a lot of headroom. And I expect that we will make progress on some things that people didn't expect to be possible on the whole. The other point that's worth noting is that this is very LLM general model-centric.

When you look at other areas, more special-economic, models, it's pretty hard to deny how incredible the rate of change is. On the screen is currently a clip of Will Smith eating spaghetti from March of 2023, compared to what we got from OpenAI SORA less than a year later in February, which has now been close to matched by Google's VO. Video generation is emblematic then of a specialized category that is truly and fundamentally different than it was just a year ago in ways that open up entirely new opportunities. A final dimension of this is that the productization of these tools is also developing rapidly. Marbleism is an example of a text-to-UI tool that can build basic software in just minutes. This isn't necessarily representative of just generalist

upgrades in the LLMs powering these systems, but about an application of them that makes them radically more useful in the world. Ultimately, when it comes to this question of the pace of improvement in AI slowing, I would argue that one, when you zoom out from simply LLMs, it's just not true. You're seeing incredible advances in all the applied uses of AI in basically every other area. But when it comes to LLMs and these generalist frontier models themselves, there are arguments and evidence on both sides, and ultimately we're just going to have to see. Next up is an interesting one. The section is called AI could become a commodity. This one is a little bit less about technology and a little bit more about market forces. Christopher Mims writes, a mature technology is one where everyone knows how to build it.

Absent profound breakthroughs, which become exceedingly rare, no one has an edge in performance. At the same time, companies look for efficiencies and whoever is winning shifts from who is in the lead to who can cut cost to the bone. The last major technology this happened with was electric vehicles and now it appears to be happening to AI. I think this is a really interesting conversation. Right now, going back to Open AI, we have that company who are clearly making their bet on being the state of the art. It's why, like I said, while it could be that they really have reached limits internally, it feels a little bit more like they are managing their place in the pole position when it comes to the state of the art. They spent about a year letting the Googles and Anthropics of the world come

close to their performance, or even, by some arguments, succeed it, only then to just slightly outdo it once again with GPT40, but it all has the flair of someone who, at any given moment, can one up you just slightly, and continue to be that leader. At the same time, we're seeing a very different approach from companies like Google. While Google hasn't surrendered state of the art, it's very clear that they understand that their comparative advantage is putting AI everywhere, taking advantage of their installs to have good enough versions of AI across a suite of products. Apple is an even further extreme of that, having not even really built much of their own technology, at least not that we've seen yet, and instead just focusing on integration across their massive

install base. The way that this plays out will have significant impacts on the shape of AI. If we really do reach an upper threshold that everyone can achieve, it's certainly you would think going to make it better to be a Google or an Apple than it will be to be an Open AI or an Anthropic. But again, at that point, this comes back to the question of whether we really are reaching a natural upper bound, at least with this type of technology. Next, we have a section which is pretty undeniably true. At least in its general conceit, today's AIs remain ruinously expensive to run. There is absolutely no doubt that AI is incredibly expensive. Indeed, interestingly, the Microsoft Inflection deal was put above in the section about AI being a commodity, but I think it's much more reflective of this question of expense. inflection, which raised $1.2 billion just last summer, shocked everyone in March when the entire team went to Microsoft, who then paid the remaining company a $650 million licensing deal effectively to buy out their early investors. Now, the inflection deal isn't quite as uncomplicated as just them making an assessment that it was too expensive to compete in the frontier model space.

They had also obviously taken a very specific approach to trying to have a more human, interactive, non-professional type of AI, which may just not have worked out at this time. but it certainly does reflect how expensive it is to compete, especially at this frontier model space. Interestingly, I think this is an area that, while undeniably true, quote unquote, is maybe being thought about a little bit wrong and too much in the context of Wall Street. For example, Mims writes, an off-sighted figure in arguments that were in an AI bubble is a calculation by Silicon Valley venture capital firm Sequoia that the industry spent 50 billion on chips from Nvidia to train AI in 23, but brought in only $3 billion in revenue. First of all, that is an unbelievably quarter-by-quarter short-sighted Wall Street type of analysis, not a venture capital type of analysis. And I think of anything

reflects just how uncomfortable these two bedfellows are at the beginning of a technology movement. Usually at a year and a half or two years into a new technology like generative AI, we wouldn't give a crap what Wall Street thinks. The problem is just that AI is so expensive that big tech was implicated right from the beginning. There simply wasn't enough venture capital in the world to actually do what these companies needed to do, which is why you saw OpenAI take 10 billion from Microsoft, and Anthropic take billions from both Google and Amazon. They're the only funding games in town that were actually big enough. What's more, because those public companies were making these big bets, those things are factoring into how Wall Street is valuing them. Every quarter,

Wall Street looks at the bottom line impact of generative AI on Google's cloud business, on Microsoft's Azure, and is making assessments around whether we've quote-unquote gotten ahead of ourselves. Despite the fact that probably a better way to look at that type of big capital expenditure is much more long-term sort of R&D than it is short-term profitability, but that's a very hard pill for Wall Street to swallow. I don't really see a good resolution to this. I just think that when it comes to those of us who are sitting here listening to this podcast, we need to be able to break apart a Wall Street analysis, which is allowed to only care about quarterly numbers from a broader, quote-unquote, bubble analysis that probably would miss the significance of AI and where

it is right now. Now, I think probably a better question when it comes to how expensive it is to run these companies is to look at whether smaller startups can actually keep running. And for those companies, it's probably not so much capital out versus revenue in. The more interesting thing to look at is the rate of the decline of the cost of tokens. There is a massive decrease in cost that we've seen over the last year. Open source is driving this even farther. And especially as some of these cheaper models catch up in performance, it's creating a much more robust ecosystem of options for those smaller companies. So in this category, while it is absolutely true that it's very expensive to run AI and that is having meaningful impact on the way the space is evolving, I don't think it's

as clear cut as to use that argument once again. The industry spent 50 billion to train, but only brought in $3 billion in revenue. There is another interesting piece here which sort of comes into this section but isn't really and I'm not exactly sure even how to categorize it, but it's absolutely a big part of why we're seeing this narrative shift now. Another piece from the Wall Street Journal from last Thursday, Salesforce darkens the skies for cloud software as AI threat looms. Wolf Street writes, software stocks got massacred after Salesforce and UiPath bloodbath. Bad breath of AI. So what's going on here? Well, first of all, Salesforce had a really bad quarter. The WSJ writes, revenue for the quarter ending in April rose by a record low 10.7% year over year to 9.1 billion,

and the company projected just 7% growth for the current period. were below Wall Street's forecasts. More important billings, a measure of business transacted during the quarter, increased an anemic 3% year-over-year, another record low, and well, under the 9% growth analysts had expected. So what's going on? Well, President Brian Milham described, we saw compression on many deals that we ultimately ended up getting done, but they got smaller when we ultimately closed them. A natural question is whether this is a general SaaS issue, In other words, is this a macro-environmental sort of situation coming home to roost in this particular area? Or is it something specific about Salesforce?

Is it possibly both? The WSJ again writes, smaller deals aren't great news for a software company now generating nearly 36 billion in annual revenue, especially when much larger Microsoft is now expanding its business at a faster rate, thanks in part to burgeoning demand for its Gen. So here we have two different parts. First of all, there does seem to be a trend in software businesses in general. workday saw a 15% share price decrease after it had a disappointing quarter, with their CEO citing, quote, increased deal scrutiny and lower headcount levels on deal renewals.

Overall, the WSJ writes, of the 10 largest cloud software providers by annual revenue, eight have seen their stock sell off by an average of 9% the day after their latest results. The piece continues, it's likely no coincidence that the tighter deal environment comes as more companies are pouring investment dollars into generative AI. That is a point that cloud software executives are all hesitant to address, given that they are also building their own AI services with haste, but some on Wall Street are starting to make the connection. In a note to clients Thursday, Brian Schwartz of Oppenheimer said that the quote,

Slowdown in Enterprise Software Spending likely reflects AI crowding out investments and slower hiring. Brad Zellnick of Deutsche Bank went further. While Bulls might be willing to look through the disappointment given it's just a Q1, we believe these results raise more meaningful questions around the adoption curve and ultimate monetization of GenAI for seat-based SaaS companies. I think they're nibbling at a real point here, but it's actually multiple points at once. The first is just how much it costs to invest in AI. For example, quote, Snowflake saw its stock fall 5% following its own report, which included a sharp cut in its operating margin projection for the year because of its AI investments. In other words, it costs a lot to invest in AI. Short-term investors don't necessarily like those long-term investments. And so the share price falls, putting even more pressure on the company's AI to deliver fast results, which run up against some amount of natural inertia in the enterprise buying sphere. This is what Brad Zellnick is talking about when he says, there are more meaningful questions. around the adoption curve and ultimate monetization of GenAI for seat-based SaaS companies.

But a second piece of this has to do with AI competing with existing SaaS services. Social Capital and All-In podcast Chamath Palahapitia writes, Salesforce share price dropped more than 20% after releasing its Q2-2020 earnings, despite earnings falling just 0.3% below Wall Street analysts' expectations. What's going on? Two factors appear to be responsible for this decline. First, a slowing economy poses a risk to revenues as customers more carefully evaluate the return of investment of Salesforce products before,

committing to a purchase. Second, analysts are concerned that generative AI could help competitors deliver similar functionalities to Salesforce at much lower costs, which could erode the company's margins over time. So this second part is a totally different reason why Wall Street might be getting nervous about AI, that AI is one of these technologies that is actually disruptive to the existing business models. And when you start looking around, it's hard not to see this emerge. Just think about the trouble that Google is having figuring out how to deal with AI overviews. On the one hand, they're clearly a valuable thing. Perplexity is. becoming this in-demand service that's totally reimagined the process of search,

and yet at the same time, if Google goes all in on AI overviews to completely hold aside their challenges with the actual quality of the responses, are they completely undermining their business of sending people to sponsored links? This is a real concern and attention that a company like perplexity doesn't have. There is a piece going around Twitter slash X by Chris Pike called The End of Software. He basically argues that AI is going to fundamentally transform the software space. The piece concludes, Software is expensive because developers are expensive. They are skilled translators. They translate human language into computer language and vice versa. LLMs have proven themselves to be remarkably efficient

at this and will drive the cost of creating software to zero. What happens when software no longer has to make money? We will experience a Cambrian explosion of software the same way we did with content. Vogue wasn't replaced by another fashion media company. It was replaced by 10,000 influencers. Salesforce will not be replaced by another monolithic CRM. It will be replaced by a constellation of things that dynamically serve the same intent and pain points. Software companies will be replaced same way media companies were giving rise to a new set of platforms that control distribution. So like I said, this hasn't exactly found its way into this narrative that Christopher in his piece that we're basing today off of, but it's something that I do think is coming up

more. The last section is called narrow use cases slow adoption. There's three or four arguments here. One which I want to spend the least time on is the comparison between OpenAI's revenue and its valuation. They basically say that OpenAI's $2 billion of annual revenue doesn't justify the $90 billion valuation, but that is a very Wall Street analysis. OpenAI's valuation right now is based on the idea that they are the leader in perhaps the most significant workplace technology to come along in a generation. In other words, the value of a startup like that is not really tied to its annual revenue, even though we still look at those numbers to try to give them some comparison to public markets, it's tied to investors' perception of its upside potential.

The second argument, though, is one that I think is worth getting into a little bit more. Christopher writes, a recent survey conducted by Microsoft and LinkedIn found that three and four white-collar workers now use AI at work. Another survey from corporate expense management and tracking company Ramp shows about a third of companies pay for at least one AI tool up from 21% a year ago. This suggests there is a massive gulf between the number of workers who are just playing with AI and the subset who rely on it and pay for it. Microsoft's AI co-pilot, for example, cost $30 a month. Okay, so effectively here we have an argument that while 75% of white-collar workers now use AI at work, only 33% of companies pay for AI tools. So there's a big gap between who's

paying for it, and who's just dabbling. However, I think this actually quite misses the point of the LinkedIn and Microsoft survey. This was the 2024 Work Trend Index, and effectively it came to a very different conclusion. Their main conclusion was that, quote, employees want AI at work and won't wait for companies to catch up. So yes, there's 75%, three and four knowledge workers who use AI at work. However, this is not mandated or even approved by the managerial class. It is in fact done outside of their knowledge. 78% of those workers are bringing their own AI tools to work, effectively smuggling them into work and not telling people because they don't want to be told they're not allowed to use them. These are people who are signing up with their Gmail accounts, often paying for them themselves with their own credit cards,

because they're just that valuable for their actual day-to-day. This survey tells the story of a complete disparity, yes, but a disparity not between those dabbling and those paying, but a disparity between workers who are actually at the front lines of using AI, and managers who are so concerned with understanding ROI that they're not moving fast enough. The survey found that while 79% of leaders agree AI adoption is critical to remain competitive, 59% worry about quantifying the productivity gains of AI, and 60% worry their company lacks a vision and plan to implement it. That concern is leading to paralysis. It's leading to small proof of concepts rather than full-on strategies. And it's in that vacuum that these employees are smuggling in their own

AI. In other words, I think that the conclusion that employees aren't really finding a lot of value in AI predicated on the fact that employers aren't moving quite as fast as we might have expected to implement solutions is completely inaccurate. What employees are telling us is that AI is so valuable that they're not going to tell their bosses that they're using it for fear of being told they're not allowed to anymore. The evidence could not point more directly in the opposite of this conclusion. As another aside, there is a whole thing here, which we spent a lot of time talking about it super intelligent, which is the difference between horizontal and vertical AI. Vertical AI is what gets most of the headlines. It's bosses who are figuring out which

LLM solution they're going to customize and how, that they're comfortable pouring all of their company data into, that they've addressed all those security concerns, that they're going to try to find those big productivity changes by having everyone on the same system. Inherently, those types of decisions are going to take longer and have a higher level of scrutiny. Those are the things that are being caught up in some of this analysis paralysis that's going on. Horizontal AI, on the other hand, is the long tail of individual employees just finding solutions that work for them and improve their lives on a day-to-day basis, that either save them time or that allow them to do things they couldn't easily do before. These are employees who are finding ways to save half an hour a day, every day,

which translates at the end of the year to about three full workweek saved. Point just being, I don't believe that the slow adoption is based on narrow use cases. I believe it's based on managerial paralysis as they try to wrap their heads around this stuff. One more nuanced and interesting argument from this is the idea that if AI systems boost productivity by helping people do their jobs but can't actually replace them, that means they're unlikely to help companies save on payroll, which means that companies are going to be less likely to adopt them. On the one hand, I think it's wildly reductive to look at productivity gains as only based on how many jobs you can eliminate, but I also live in the real world, and there are going to be some number of companies that view it that way.

So I think understanding that conversation and whether companies really are just looking to do the same amount with less versus do much, much more with the same amount will have an impact on how adoption happens. So we come back to the question, is the AI revolution already losing Steam? It is certainly the case that our understanding of the AI revolution is getting more advanced and consequently more nuanced. It is also the case, as I've said numerous times on this show, that the nature of the headlines at the beginning of this movement, that promised entire category,

of jobs gone the next day when you woke up, gave people, I think, a misperception of just how fast this was going to happen, making it so that this type of narrative is plausible, even though in many ways this is some of the fastest adopted work technology we've ever seen. It will probably not surprise you then, that I do not believe that the AI revolution is already losing steam. I'm skeptical of the argument that the pace of innovation is slowing. I'm in agreement that the cost of running it remains exorbitant, but I also think even that has more nuance when it comes to the productization of it with smaller startups. And I am dead set against, and I am dead set against, the argument that its usefulness is limited.

But go read the piece for yourself, consider it alongside all the arguments that I've had, and then come let me know what you think. Anyways, friends, that is going to do it for today's AI Daily Brief. Until next time, peace.