Motley Fool Hidden Gems Investing - AI’s ROI

Episode Date: June 22, 2024

In 2023, the AI industry spent an estimated $50 billion on Nvidia chips, with the purpose of training AI models. The payoff for all that spend, according to Sequoia Capital, is $3 billion in revenue. Is that a return worth bragging about? RIcky Mulvey talks with Fool analyst Asit Sharma about how investors might think about companies’ AI spend. They also discuss: - The rate of improvement for AI models - How non-Mag 7 companies are using AI - And one company that’s spending smartly on the new technology.  Take a look at the Gartner Hype Cycle.  Host: Ricky Mulvey Guest: Asit Sharma Producer: Mary Long Engineer: Tim Sparks Companies discussed: GOOG, MSFT, NVDA, ARM, AMD, ORCL Learn more about your ad choices. Visit megaphone.fm/adchoices

You've got to try breakfast at A&W. You've got to try breakfast at A&W. And what better way than with the delicious Pret Organic Coffee? Starting with just one dollar, all day, every day, now until December 31st. You've got to try breakfast at A&W. At participating A&W locations in Ontario. The basis of all this is that the processes are much more intensive on servers. They cost a lot more because they're delivering some kind of magic to us that we didn't have before.

Now, yeah, there's hallucinations in there. There is perceived inaccuracy of some responses. It's early stages. But so far, companies are willing to invest and some consumers are willing to pay for this. Many of us are getting it free as an offer just now. And it is being integrated with products that we use every day. But the general gist of this is that people are willing to pay for that extra bit of magic. I'm Mary Long, and that's Asit Sharma.

Artificial intelligence is everywhere, and all that data is expensive. A KPMG survey suggests that 43% of U.S. companies with over a billion dollars in revenue plan to invest at least $100 million in generative AI over the next 12 months. But how to tell whether all that investment is actually paying off? My colleague, Ricky Mulvey, caught up with Asit to talk about how investors might consider a company's AI spend. They also discuss how non-Mag7 companies are using AI, the difference between large and small language models, and the future of AI friendships. So Asit, let's start with the positive. I mean, artificial intelligence represents

world-changing technology breakthroughs. It's already providing a co-pilot. At my job, It's changed the way I search on the internet, probably for you as well, if you're on Google, which I assume you're not a Bing guy, right? You're not out there on the Ask Jeeves, Yahoo, Bing platforms. You read me right, brother. I am a Google person. I don't have anything against Bing. I do use it from time to time, but I'm a Google person first.

I'm trying to get us to this place where we're in the middle, where we recognize the extraordinarily positives and also some of the risks that come with AI in this investment hype, what I would call boom cycle. Yeah. Let's start with the positive. What's AI allowing you to do that you weren't doing just a couple of years ago? Ricky, one of the things it's allowing me to do is to catch up on a business story that might have changed in a really, really fast way. I cover many, many companies for different

services that I work on and you can't keep track from day to day with what's going on every last business. And sometimes I'm called upon to find information quickly and try to make a quick assessment. In the old days, yeah, I would have to go to Google, SEC filings, interviews, et cetera. And now I can just ask an LLM, well, just brief me on what happened during this timeframe when X event occurred. That's one thing. So, I think that's really cool. Another use is to use large language models as a surfacing tool to try to find great investments. Let's be real, their reasoning capabilities aren't better than mine or other analysts yet. They get a little better every day, but they're extremely useful

if you know how to sort of point them in the direction to show you companies that might be worthier for the research. And I think lastly, there's a foreign language I've been trying to learn and just being able to ask ChatGPT, okay, look, here's the sentence. All right. I just don't understand what's going on here with this freaking past participle. Can you explain this to me? And it gives a really cogent answer, helps me fill in the blanks. And so, I think that's really fun i think it's to your point though it's it's good for idea generation but it requires further skimming like for my job i'll use it you know i'll ask it how does this company make money if i need to like sort of get up to speed on a company that i may be talking about in the future

it's almost it's almost like a you know it's like a handyman at times where i've asked it for hey i have this thing going wrong with a piece of software that i use for work you know what what can i do to like accomplish the thing that i'm trying to accomplish sometimes it's with with audio editing and then the other thing i'm excited or that i've used it for is cooking so i have these ingredients in my refrigerator what can i do with it and it can give you some different options so it's that kind of idea generation stuff that is is great but you know what asset is every time i ask it a question every time you ask it about a piece of investing news It costs someone money because these artificial intelligence programs are really expensive to run.

The infrastructure is really expensive. That's why NVIDIA is making so much money right now is because you have so much energy on the spending side. According to a research firm, KPMG, about 43% of companies who are making a billion dollars a year in annual revenue. So big companies making billions of dollars are expected, on average, to spend, just spend, $100 million on generative AI. Companies aren't stupid, generally. We'll just make a general statement. What's the reasoning or fear for spending this much money? Why not take a more wait-and-see approach?

Let's tackle the reasoning part of it first, Ricky. One is that companies of this size are pretty good at understanding where they can get a return on investment. So they'll do a lot of experimentation, conduct experiments, have smart people like software engineers just try to run things in a different way. And what they're finding is the small experiments are paying off. So could we save 4,000 person hours by having the large language model solve this coding problem for us rather than hiring people to do it next year? The answer to the experiment may be yes, we definitely can do that. Second is that the same types of software engineer people, IT people have already identified where proprietary data can be leveraged. So whatever your industry is, you've got something in your data stream, historical data stream, ongoing data stream, that's yours, that's proprietary, that you can use to further sales if you can only find the right product fit or service fit or extend your current lineup.

And many companies are starting to see how they can use large language models to make sense of that proprietary data and sell that upstream. So that makes the investment promising. And lastly, there's just a lot of opportunities for cost savings. I mean, if you, Ricky, or me can now hold up a camera to our fridge and get a recipe out of that, that turns out to be a remarkably tasty dish. We've saved ourselves a heck of a lot of time. Those types of optimizations are happening all over the business world. And then to get to the first part of your question, yeah, I mean, there's a fear case

there. There's a FOMO case here. some companies are making decisions because they don't want to get left behind even though they haven't really pinned down the return on the investment yeah there's definitely some some pivoting where what was machine learning just a few years ago makes makes you an artificial intelligence thought leader now asset because people want to see those mentions on the earnings calls even for non-computing companies let's talk about the spend so sequoia capital estimates that the industry spent about $50 billion on chips from NVIDIA to train AI, to train these models

in 2023. That spending only brought in $3 billion in revenue. I say only with some quotations, it's a lot of money. It's also 6% on the total spend. So we got a 6% return. You're the CFO of all of AI now. Are you satisfied with that return? Is this inefficiency? Is this overspending? Or is this just the cost of growth and emerging technology, darn it, and you got to spend money to make money? Yeah, I'm taking the pencil from between my ear and my shock of hair and waving it back at the CEO and saying, yeah, this is about what we expected, boss. Now, I don't mean to make fun of CFOs there. I'm an accountant myself, but this sounds to me where we should be in this stream. Let me give you an example in the real world. If you and I are real estate

developers, right? And we're looking at a big parcel that we can develop on the edge of a city where the downtown is petering out a bit. We're going to have to buy that land, put up a few buildings, maybe a commercial building, some housing, but we've got this long-term master project going. So, in the first year, the income stream that we can capitalize ain't all that much. You know, we might be losing money and we probably are overall. We consider the cost of our debt to pull this project off. But over time, Ricky, businesses move in, subdivisions get built, restaurants come in. Then that part of the city gets big enough that the government wants to put a public library there. So, everyone can benefit. There are a lot of players that come in and we

start making money in year X. We see we've recouped our investment and then we're a little bit off to the races. So, this is how, when we call it the AI industry, this is how the hyperscalers, the companies that are buying the GPUs, the Amazons, Metas, Microsoft, Oracles, and other companies, private businesses as well, enterprise business are treating it like a long-term development and they're not expecting to be making a bunch of money upfront in year one, year two, year three. Yeah. The internet can make every year feel like a dog year in a way. And it's, it's, I think it's easy to forget that this, this boom has only been around for what, about 18 months now, really since like, I think it was November of 2022 ish.

So maybe you're not getting that immediate return. These applications are very expensive. We mentioned that earlier. And, you know, for just one use case, I think it's asking, you know, like a chat GPT question versus Google a question. It's much more expensive to ask chat GPT a question than it is to ask the Google search engine. I want to use that as an example to get into the cost of it. Why are these AI applications so much more expensive to run than the computer applications that we're used to? When we ask Google a question, we're asking basically a computer program to run, a series of algorithms to run that look at an index that's been built up over a long, long period of time. It's a predictable workload. There's this really neat framework of knowledge that Google dips into. It has something called relevance.

So the latest algorithms tell it which search results to put up top. But that workload was figured out a long time ago. And Google knows how to make money off of that. When we ask a large language model, say chat GPT to do the same thing, the process is totally different. First of all, it's inferential. So the large language model is taking our question and using a probability-based decision making metrics to give us an answer.

It also consults Google. You can see that now when you do certain web searches with different LLMs, different chatbots, they're consulting Google. But then not only this is happening, it's also serving it up to you in natural language, which is another inferential process, which is a harder workload on the server. So that costs more money too. And then, you know, there's the constant training behind the scenes of the models. There's so many engineers who are working on this and making sure that the AI version of Google doesn't tell you to do something weird with your pizza to cool it down. Yeah, add glue to it.

Add glue to it. The basis of all this is that the processes are much more intensive on servers. They cost a lot more because they're delivering some kind of magic to us that we didn't have before. Now, yeah, there's hallucinations in there. there is perceived inaccuracy of some responses it's early stages but so far companies are willing to invest and some consumers are willing to pay for this many of us are getting it free as an offer just now and it is being integrated with products that we use every day but the general

gist of this is that people are willing to pay for that extra bit of magic well in a lot of tech adoption cycles right you deliver something of value to your customers below cost hope they enjoy it enough and then eventually you figure out how to make a profit on it we've seen a lot of you know the rate of growth on these models these video generative ai models have been just incredible asset you've seen the differences in video that have come from these tools just a year ago to now right every have you seen the video of like will smith eating noodles back when these back when these models first started you mentioned this but i haven't seen it oh it's it's like it's it's cartoonish it doesn't quite look it doesn't look human it looks very much like it came came

from a you know like a weird computer algorithm and now there's another image of a of a man eating noodles and it looks high definition it looks like it came from uh that someone shot in a cafe rates of growth like this always have to slow down and uh christopher mims of the wall street journal he's a tech columnist there he's been a guest on motley fool money i i really admire his column in his writing basically points out that all of these AI models have been trained on quote more or less the entire internet and are running out of additional data to hoover up there aren't 10 more internets worth of human generated content for today's AI models to to inhale end quote basically saying that these AIs are going to start learning off each other and that may be

a little recursive and you're not going to get the same rate of change and improvement Do you think this sort of data limit could put a damper on the rate of improvement for AI models? Or, you know, maybe are we underestimating how much improvement we have left to go? So on the consumer facing frameworks that are iterating on answers to everything, I think Christopher Mims has a great point here. I mean, we've already trained most of the prodigious, the biggest LLMs on the bulk of human knowledge that's on the internet, right? There's so much human knowledge that isn't on the internet. But having said that, yeah, it's hard to see how the quality will improve over time without having a phase of degrading first. And we see this in large language models too.

As some of their own output starts to creep in, they get a little dumber. And then the engineers go in and tweak the models. They get a little bit smarter again. So I think that's a great point. But for things that aren't trying to be all things to all people in terms of question answer frameworks, the underlying technology behind this is strong. It's getting stronger. And there are so many specialist companies that are working on various ways that information

is analyzed and regurgitated back to us. I mean, synthetic data is one solution partial to this problem. If you're a business that has a specific use case, synthetic data is data that's created to train a model like human data or human generated data. And so this could be important, say, in a manufacturing industry where you don't necessarily need private data, but you want to have outputs of a certain process to make something better. So synthetic data is going to be more important to such models. Then there's also quality over quantity. If you're working on specific questions in, let's say, the pharmaceutical industry, you don't really need to keep feeding more and more information into a model.

What you're trying to do is to make sure you have a better tool for analyzing the information. Just a couple of other things I wanted to mention. There are small language models, which don't need as much information or even processing power to run. Like Microsoft has a couple that are pretty interesting. And these are solving the problems of large language models through reasoning. So some models are trying to get better at reasoning the way our minds reason. And that could be a way that all of this ecosystem gets a little better without having to digest

so much information and shove a bunch of information down a pipe and get smarter. Instead of information, let's work on the thought process. Let's work on how these models think. And then finally, of course, you've heard me say this and other analysts say this before, proprietary data sets for companies are very powerful. I was just talking about this a moment ago. And that's another case where, at least in business, you don't necessarily need to worry about the problem that Chris Moons is talking about.

you're focused on your data stream and how you can derive insights and maybe better products from that. I want to talk about small language models for a moment. A lot more familiarity with large language models. How might someone encounter a small language model? What are these looking like? What kind of problems are they addressing? Yeah. So small language models, I think we'll see them more in the real world now, but I think that Microsoft, who I name, has a couple of these. We're going to see these with devices in the future. So you can spin a small language model from a device that merges with chip design. So companies from ARM to AMD to Intel are designing their chips to have some neural

processing units right on the chip, right on the phone. So when you combine these two concepts, you get devices that are smarter. They don't necessarily have to go to a server out in the cloud to answer a big question, but they could tell you, Ricky, if you take that photograph of your fridge, how to make a better recipe based on information that's already stored on your phone themselves without you having to go to a chat GPT or whatever model you're using. So a bit of reasoning capability, plus information that already exists on device, plus information that's taken in, spins out something that's reasoned and solves your quick problem. And then to your earlier point, yeah, there are times where synthetic data would work for something like a digital twin. You have a manufacturing floor and it doesn't matter if you're using recursive data, you're just trying to make the model better so you can make maybe an assembly line, you know, take two, you know, two tenths of a second quicker than it normally runs. Totally. I should say we're already at that point that Jensen Huang predicted a couple of years ago. And he said in the future, there's going to be more virtual robots and there will be physical robots. Maybe we haven't reached it quite yet, but we're headed there pretty quickly. We'll see if Elon gets his way with the humanoids. That's a separate show, though.

And we talk often about how the, you know, the MAG-7 companies seem to be using AI. You know, there's clear winners, which if you're a smaller company and you need to build an LLM, you probably need NVIDIA's chips for that. Maybe you'll use AMD's chips if the NVIDIA chips are not available for sale. But I'm curious about how the other companies seem to be using it. And it breaks down into a few categories. It comes down to essentially, I would say personalized chatbots. So if you go on Airbnb and you need help with something, an Airbnb chatbot might be able to help you a little bit quicker or a little bit more efficiently than a human agent will. I should say more

efficiently for the business. Simplifying contracts, that's how Cisco is using it. Co-piloting, Microsoft is famous for this. I've already used a Mag7 in the example, but Microsoft co-pilot on Office 365 to help you write emails a little bit more quickly. And then forecasting is another example where if you're a retailer and you want to forecast how many jeans you're going to sell in, let's say, Indianapolis, Indiana in May, and you can pull all of the data that you have on similar regions, Indianapolis, and then maybe make a better prediction about how many jeans you need to sell to your retail store. Any reflections on these use cases or anything else you'd want to add for these smaller companies spending money on AI? They're all fun. Let's look at that personalized chatbot one. It's intriguing to me. Intuit, which you and I had talked about before the show, is really interesting because they have those proprietary data sets that are in consumer-facing applications. So if you use QuickBooks online, they have information not only about you, but about every other business in your zip code that's similar to you. And this is interesting.

I had a conversation with the CEO of Intuit here on Motley Fool Money just about a year and a half ago, Sasan Ghadarzi. And he was telling us, Ricky, that, look, we're going to combine all this interesting data and help you learn how to manage your cash flow better. Are you running out of funds? Well, the chatbot will be able to tell you, look, every other lawn mowing business in this zip code doesn't spend quite as much on seed. Maybe you're buying a premium seed and your profit and loss would be better if you drop down one level. Now, he didn't use that very specific example, but he was spinning up a few similar types of scenarios. And I think that's a great illustration of a use case of a company that, yeah, it's big and to it has a big balance sheet. They could go out and buy a bunch of GPUs, but really what their focus is, is let's make

something that will help our end user, the consumer. I think that's going to make them some money over time. And I do like also, you mentioned co-piloting of code. There's so many companies that are offering co-pilots now in various walks of business life. And this is something not just that we'll see on the consumer side, Ricky, but people and business organizations will see. I think it almost becomes commoditized though. At some point, these models become so powerful. We don't even think of it as co-piloting anymore. It's just

part of our daily routine. You just type in or talk to or show an image to your personal co-pilot for whatever your workflow is in any industry. And it's going to automatically do things that advance your day. Now, hopefully that results in a little more free time in our day. It doesn't mean that we don't have work, but we'll have to see what that future looks like. I mean, the Jetsons' future has always been promised with every advancement. We'll see. It had such snazzy music. I could not help but think that that's an optimistic case because of the Jetson music.

But now, you know, we grow up, Ricky. We're no longer watching the cartoons and now we wonder, is the music going to be really peppy? Is it going to be a little ominous? I don't know. It's time for real work. You hit on two, basically two areas that Sequoia Capital is bullish on, which is customer support and AI enterprise knowledge in the near term. The other area that Sequoia is bullish on is AI friendship, pointing out that when people use these AI chatbots as friends, they have maybe stickier adoption rates, which I understand the business perspective, but also a little concerning and terrifying.

because, you know, hey, let's find more friendships through these LLMs that really know me for me. And, you know, you don't necessarily have the consequences of human friendships and relationships. That's one thing to flag. Any companies impressing you with how they're spending on AI? We've talked about the amount of spending. We've talked about how companies are creating these new applications for customers and businesses. But are there any companies impressing you with the way that they're spending on AI in this boom cycle? Wait a minute, Ricky. I'm going to answer your question. Yeah.

But you can't bring up such an intriguing point for me and our listeners without, let's just talk about two seconds. We got time. Let's talk about AI friendships. I find it also fascinating. The transformer model, the technology that underlies all LMs is just sneakily good at figuring out what's important in a group of words, in a text, in any kind of visual data you give it that's why they're so powerful and i'll use this word again so magical but it also means that they're sneakily good at understanding the buttons to push to get certain responses so if you build an ai with a lot of data to be able to manipulate emotion

you you can develop ai that many people will respond to and feel up to a point like this is It's very similar to relationships I have in real life. And actually, I'm lonely and I want to go home and talk to my bot. I find that interesting. I don't know quite what to make of it. What do you think about this? What do you make of it? I mean, I think it's hugely concerning at a societal level because it's going to offer people, you know, sort of facsimile of human relationships that may be serviceable to you. What happens when, you know, you don't need to go out on a date because you have a chatbot who's interested in you and nice to you and always texts you back and is interested in the same things that you are. And that's so hard to find in the real world. The chatbot is never going to betray your trust, right? I think that it's one of those things that seems silly on the surface level. And there's a movie, Her, that kind of covers it a little bit. But it's going to be one of those things that sort of continues the trend of, I would say, social media and a lot of sort of life becoming more digitalized, which is that, you know, people aren't going out as much as they used to. because you can still get those dopamine buttons pushed on your phone. I think these AI relationships are going to continue to be an amplification of that. The police said, or maybe it was Sting,

a scientific means of bliss will supersede the human kiss. And this is sort of that coming true a little bit to me. I also find it unsettling. I mean, I think too, the pendulum swings both ways. I just see younger people getting fatigued of social media and going out more. I think there comes a point where the emotional support you may get from a bot just starts to feel synthetic and you crave that human touch, that human conversation. But what it means is to get to that point, many people may dive into this stuff just

as they did with social media and that's a scary thought. We should also say on a societal level, it may be very helpful for some people who are say, elderly or otherwise unable to do what they used to do in terms of social connection. So, it may provide a use there. Again, not a perfect use, but of some comfort or to the sick. There are some really interesting philosophical questions that have been posed over time by people who thought this out before the technology reached the point where it is today. So, Austin, how about that for a setup for a company discussion about which companies

are spending on AI in an intelligent way. I'm looking at the outline. You have one we've talked about on the show previously, but what company is impressing you with how they are spending on artificial intelligence? Yeah, I'm going to be boring here, but I'm going to say it because it's true. I think Oracle is really impressing me and some other investors. They had earnings recently come out and I think the stock was up 12 or 13%. But