The AI Daily Brief: Artificial Intelligence News and Analysis - AI Is The Fastest Adopted Work Tech Ever, But Still Not Fast Enough for Some

Episode Date: July 13, 2024

Explore why AI is the fastest adopted work tech ever, yet still not fast enough for some. A nuanced discussion on AI’s current place in the hype cycle, insights from former a16z partner Benedict Evans, and the gap between managerial and user perceptions of AI. Concerned about being spied on? Tired of censored responses? AI Daily Brief listeners receive a 20% discount on Venice Pro. Visit ⁠https://venice.ai/nlw⁠ and enter the discount code NLWDAILYBRIEF. Learn how to use AI with the world's biggest library of fun and useful tutorials: https://besuper.ai/ Use code 'podcast' for 50% off your first month. The AI Daily Brief helps you understand the most important news and discussions in AI. Subscribe to the podcast version of The AI Daily Brief wherever you listen: https://pod.link/1680633614 Subscribe to the newsletter: https://aidailybrief.beehiiv.com/ Join our Discord: https://bit.ly/aibreakdown

Today on the AI Daily Brief, a more nuanced discussion of where AI is in its hype cycle. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show notes. Welcome back to the AI Daily Brief. It is the weekend now, and that means, of course, that it is time for a long reads episode. And given that the theme of this week has really been all about AI and the hype cycle and where we are in that hype cycle, I thought it would make sense to read something that has a little bit more nuanced than perhaps some of the other pieces that we've covered earlier. Let's read the piece by former A16Z partner

Benedict Evans, and then we'll come back and discuss it. The piece is called the AI Summer. The summary reads, hundreds of millions of people have tried Chatcheebt, but most of them haven't been back. Every big company has done a pilot, but far fewer are in deployment. Some of this is just a matter of time, but LLMs might also be a trap. They don't look like projects and they look like magic, but they aren't. Maybe we have to go through the slow, boring hunt for product market fit after all. And yes, this is real NLW reading this time, not AI. Benedict writes, my old boss, Mark Andreessen, liked to say that every failed idea from the dot-com bubble would work now. It just took time. It took years to build out broadband. Consumers had to

buy PCs. Retailers and big companies needed to build e-commerce infrastructure. A whole online ad business had to evolve and grow. And more fundamentally, consumers and businesses had to change their behavior. The future can take a while. It took more than 20 years for 20% of U.S. retail to move online. People forget this now, but the iPhone took time as well. Apple sold just 5.4 million units in the first 12 months, and it took until 2010 for sales to really work. The iPod took even longer. The same, of course, applies to the Enterprise. If you work in tech, cloud is old and boring and done, but it's still only a third or so of enterprise workflows 25 years after Mark Beniof tried to persuade people to do software in the browser. ChatGPT happened a lot faster. It exploded into our

consciousness in late 2022, and it's taken all the oxygen in tech almost immediately. If you're building a today that isn't focused on generative AI, all your friends will point at you and laugh. But much more importantly, chat GPT got to 100 million users in just two months. Editor's note, I think it was actually five weeks. By the spring, unprecedented numbers of people had both heard of it and used it. As with every observation about the acceleration of tech adoption, a lot of this is standing on the shoulders of giants. OpenAI didn't have to wait for people to buy devices or for telcos to build DSL or 3G. For consumers, ChatGPT is just a website or an app, and to begin with, it could ride on all of the infrastructure we've built out over the last 25 years. So a huge number of people went off to try it last year.

The problem is that most of them haven't been back. If you ask what used actually means, it turns out that most people played with it once or twice and go back only every couple of weeks. The next part refers to a chart. So if you're listening to this, I suggest you go check out the actual piece. This is a very glass-half-empty, glass-half-full kind of chart, as the caption points out. On the one hand, getting a quarter to a third of the developed world's population to try a new product in 18 months is very hard. But on the other, most people who tried it didn't see how it was useful. I will once again make a note that the argument that they didn't find it useful is that they don't use it daily. Instead, for example, in the U.S., it's a little under 20%

of people who are using it weekly. Back to Benedict, though, of course, there's a selection bias here. If you bought a $650 smartphone, you've already decided that it's useful, and you're a lot less likely to abandon it than a website you spend five minutes playing with. And you could also point out that the best versions of the models are often behind paywalls. But if this is the amazing, magical thing that will change everything. Why do most people say, in effect, very clever, but not for me, and wander off with a shrug? And why hasn't there been much growth in the active users, as opposed to the vaguely curious at the last 9 to 12 months, as shown in a bunch of similar surveys? And most revealing, possibly is Google Trends, which must always be used with caution, but which

seems to show a correlation with school holidays. There are a couple of ways that you could answer this. It does take time to change your habits and ways of thinking around an entirely new kind of tool. Remember when we printed out emails? We can be certain that the models will get better, at least to some extent, agents' voice and multimodal will expand the problems they can solve. But I've also argued that an LLM by itself is not a product. It's a technology that can enable a tool or a feature, and it needs to be unbundled or re-bundled into new framings, U.X and tools to become useful. That takes even more time. I think you can see all of the same issues in this data from Bain, surveying enterprise use of LLMs. Again, this is glass half empty, glass half full. There's a lot

of interest in quite a lot of deployment, but it depends where you look. Unlike some surveys, which just ask in effect, does anyone at all anywhere in your organization? using this, Bain tried to split the pilots, experiments, and trials from the deployment. Everyone has a bunch of tests, but far fewer people are trusting something in their business to this yet, and all of that varies a huge amount depending on your use cases. LLMs are already very useful for coding and marketing, but much less useful for lawyers or HR, though, of course, lawyers are notably slow adopters of any new tech. Accenture, meanwhile, gave us a great illustration of the scale of the enterprise experimentation,

but also how much it's only experimentation for now. Again, a glass-half-full, half-empty illustration. Last summer, it proudly announced that it had already done 300 million of generative AI work for clients and that it had done 300 projects. Even an LLM can divide 300 by 300, that's a lot of pilots, not deployment. The number has gone up a lot since then, but what's the mix? Indeed, with BCG saying that it expects 20% of its revenue this year will be helping big companies work out what to do about generative AI, the single biggest business from this in 2024 might be for consultants explaining what it is. Today's episode is brought to you by Super Intelligent, the platform for fun, fast AI learning. Super has a ton of new things going on.

We recently announced our partnership with Spotify, through which users of that app can now access Superintelligent content directly from their mobile apps. We've also just launched the AI learning feed. In addition to seeing the tutorials that we're dropping, there are polls, news items with related lessons, and a chance for people to show off the projects and use cases that are making AI come alive for them. We've also just kicked off the Super Summer Challenge, where each week we'll share a new challenge that you can use to discover new AI tools and use cases. Go to be super.aI and use code superfund for 50% off your first two months. That's besuper.aep. Today's episode is brought to you by Venice. The leading AI companies store your entire

conversation history and attach it to your identity forever. Every question you ask, every answer you receive, every image you generate, every thought you share with the machine, it's all being spied on. If you trust all the companies, hackers and NSA board members that will ever have access to your AI conversations, then rejoice, for you are well served. For the rest of us, Venice is an alternative. Venice is a powerful AI app for text, image, and cogeneration that respects you as a sovereign individual and believes privacy and free speech are not only human rights, but are necessary for civilizational advancement. Private, permissionless, and uncensored. You can try it for free without an account at venice.a.i. A lot of these charts are really about what happens when the U.T.

Topian dreams of AI maximalism meet the messy reality of consumer behavior and enterprise IT budgets. It takes longer than you think, it is complicated. This is also one reason why I think Dumers are naive. The typical enterprise IT sales cycle is longer than the time since Chat GBT 3.5 was launched, and Morgan Stanley's latest CIO survey says that 30% of big company CIOs don't expect to deploy anything before 2026. They might be being too cautious, but the cloud adoption chart suggests the opposite. Remember also that the Bain production data only means this is being used for something, somewhere, not that it's taken over your workflows. Stepping back, though, the very speed with which ChatGPT went from a science project to 100 million

users might have been a trap. LLMs look like they work and they look generalized and they look like a product. The science of them delivers a chatbot and a chat bot looks like a product. You type something in and you get magic back. But the magic might not be useful in that form and it might be wrong. It looks like a product, but it isn't. Microsoft failed and forgotten attempt to bolt this onto Bing and take on Google at the beginning of last year is a good microcosm of the problem. LLMs look like better databases and they look like search, but as we've seen since, they're wrong enough and the wrong is hard enough to manage, that you can't just give the user a raw prompt and a raw output. You need to build a lot of dedicated product around that,

and even then it's not clear how useful this is. Firing LLM web search out of the gate was falling into that trap. Satya Nadella said he wanted to make Google Dance, but ironically, the best way to compete with Bing co-pilot might have been to sit it out, to wait, watch, learn, and work this thing through before launching anything. If Wall Street had allowed that, of course. The rush to bolt this onto search came from competitive pressure and stock market pressure, but more fundamentally from the sense that this is the next platform shift and you have to grab it with both hands. That's much broader than Google. The urgency is accelerated by that standing on the shoulders of giants moment. You don't have time to wait for people to buy devices,

and from the way these things look like finished products. And meanwhile, the firehose of cash that these companies produced in the last decade has collided with the enormous capital intensity of cutting-edge LLMs, like matter meeting antimatter. In other words, these things are all the future and we'll change everything right now, and they need all this money, and we have all this money. As a lot of people have now pointed out, all of that adds up to a stupefingly large amount of capax and a lot of other investment, too, being pulled forward for a technology that's mostly still only in the experimental budgets. The rush means we've skipped the slow, painful period at the bottom of the S-curve, where you try to work out what product market fit looks like,

while you build the actual product. The web and e-commerce and the iPhone had to go through a painful process of growing and learning to become useful. The App Store wasn't part of the plan for the iPhone, and Tim Bernersley's original web browser included an editor, because this looked like a better network drive, not a publishing platform. LLM skipped that part where you work out what this is and what it's for and went straight to it's for everything before meeting an actual user. That makes the chart below, which is a Y Combinator Startups by Field chart, particularly interesting. The straightforwardly skeptical interpretation, which shows a radically increasing percentage of YC startups being from the AI field, is that this is a classic surge in investment that will inevitably turn into a bubble if it

isn't already. But you could also suggest that these startups are a collective Silicon Valley bet that LLMs are a technology, not a product, and that we need to go through the conventional process of custom discovery towards product market fit. The thing that really drives a bubble in generative AI, at least arguably, is the idea that history is over, and that LLMs will be able to do everything. And in that case, we wouldn't need any of these companies. On that view, these companies are the anti-bubble. Of course, the crazy dreams of the dot-com bubble really did happen, and the AI maximalists may be right. It may be that LLMs can do the whole thing. LLLLLM, may be able to swallow most or all of existing software,

and they may be able to automate vast new classes of tasks that were never in software before, just by themselves and with whole new layers of product, company and enterprise sales built around them. This might be the first S-curve in tech history that turns out to be a J-curve. But not this year. All right, so I said this is more nuanced,

so let's talk a little bit about that. First of all, once again, you're getting this similar theme that we've seen from so many of these pieces, where the question isn't long-term efficacy, it's about timescale. The core of Benedict's argument is that technologies always go through a messy process of productization where opportunity meets actual consumer need and turns ultimately into a collection of products

that go on to change how we do things. To the extent that his argument is simply that AI, no matter how obviously transformative it is, will still have to go through some version of that process, I absolutely agree. There is, however, a presumption in this piece that it's not already doing so, that the only thing happening is consultant-led pilots. There's a lot of lot of evidence that that isn't the case. First of all, as we've been over over and over again, there is a absolute chasm between the perception of management on AI and the perception of workers on AI. For those of you who haven't heard me endlessly rant about this, the TLDR is that while managers are awash in questions of ROI and focus on pilots and proof of concepts, their

employees are simply taking matters into their own hands and figuring out workflows that actually work for them. Does this under-maximise the productivity gains available from AI? Absolutely. But does it also radically understate how much AI is being used right now? Absolutely. So one counterargument to some of this is simply that the studies that Benedict is quoting seem to be fairly reflective of the managerial class perception of AI as opposed to the individual user class perception of AI. And those things, I think, are radically different. Second, when it comes to any new technology, the vast majority of people will not be iterators and experimenters who figure out the right use cases. It is a tiny fraction of people for any technology

who spend the time to hack at it, explore, experiment, fail, start again, to figure things out which they then transmit to all the other people who take on those strategies. Think about something like Photoshop. Everyone didn't just get this software from a Basel State and figure out all the tips and tricks. A very, very tiny percentage of people figured out the key lessons and then transmitted them through courseware, through online tutorials, through all sorts of different mechanisms. that transmission is happening radically less than it might otherwise have because of that phenomenon that we were just talking about, which is that people feel they have to smuggle AI into their offices because if they let people know they were using it, they would either be looked down upon,

being seen somehow as cheating, or simply told they weren't allowed to do it. And contra to all of these points that people aren't finding it useful, in my experience, hearing from tens if not hundreds of thousands of people about AI, the average experience actually is that when you find a use case that works, you are completely and fundamentally unwilling to go back to the way that you used to do things. So that fear of being told you can't is a very profound one. Okay, so we now have a difference in the perception of managers versus employees on AI, an artificially reduced transmission process from the early adopters and experimenters in terms of sharing what works. And as an extension of that, you also have, I think,

broad misperceptions of exactly where the value is going to come from. One thing that I think Benedict is right about is that the excitement around the possibilities, the sheer feeling of being a wizard, combined with the media narrative of how disruptive AI was likely to be, created a situation where everyone acts, like where if you don't wake up tomorrow with half of the world's jobs gone from AI, it didn't live up to its promise. Meanwhile, the way that it's actually rolling out is all of those long tail of individual people, as I mentioned before, finding these processes that save them 20, 30 minutes at a time, which day and day out add up to weeks or even months every year. What's more, the natural tendency has been for people to try out AI in the context of the things that they know well.

So the writer experiments with AI for writing and naturally comes back a little unimpressed because, hey, they could do it better. But across the thousands of individuals and organizations using a platform like super intelligent, we're seeing is that where AI is really thriving is reducing the time and pain of secondary tasks rather than increasing the performance around primary tasks. In other words, every job has things that aren't the core part of the job that you still have to do. it's the stuff that makes jobs frustrating in a day-end, day-out kind of way, and that's exactly the stuff that AI is great at. If you're an artist who creates beautiful web flows for clients, but hates writing copy, guess what? LLMs are your new best friend. A last point that I'll make

about Benedict's arguments has to do with this idea of LLMs as a product or not. I think it's perhaps a little bit more nuance than he's suggesting, but it is certainly the case that the trajectory we are seeing from a product perspective is for people to take these models and integrate them into products that are focused on very specific use cases and very specific user experiences and user interfaces that make sense to the ultimate end user, whoever they happen to be. At first, we derisively call these wrapper products. And if you weren't spinning up your own model underneath, why were you even messing around? Now I think we're starting to see things a little bit different, the success of, for example, a company like perplexity, which has been

ruthlessly focused on a specific type of research-focused search experience, taking advantage of other people's models, has given us a glimpse of what it looks like to combine product thinking with technology thinking. And it's not just perplexity. This is happening absolutely everywhere. It's happening with fine-tuning and it's happening with UX. And I think if I have a critique of anything around that point for Benedict, it's that he seems not to realize just how much of that is going on. But given that we're often creating tutorials around 15 or 20 new products per week, at Super we see this every single day. The last thing I will say is just that all of this rests on a perception and an interpretation issue. Benedict is smart enough and nuanced enough to point out that

all of these things are, to some extent, in the eye of the beholder. It's why he keeps using that phrase, glass half full, glass half empty. But let's zoom back three years. What would you think, if I told you that there was a technology that was about to be launched, that within around 18 months or so of it launching, would be used every single week by nearly 20% of U.S. citizens. My strong guess is that you would either think that I was referring to some hot new social network that was replacing TikTok, or you would think I was full of it. When you take a step back, regardless of all of these concerns and all of these questions, we have still never seen anything even close to as explosive as the adoption curve of generative AI. Could it have been faster based on how strong

our conviction is that it will change literally everything in the long run? Sure. But ultimately, that's a counterfactual and does not mitigate just how fast it's been. Still, like I said, I like Benedict Evans' take on this a hell of a lot better than the folks who are out there with a particular axe to grind or media coverage to get. And so a big thanks to him for writing this. And of course, a big thanks to you guys for listening or watching. Until next time, peace.