Motley Fool Money - AI’s ROI

Episode Date: June 22, 2024

In 2023, the AI industry spent an estimated $50 billion on Nvidia chips, with the purpose of training AI models. The payoff for all that spend, according to Sequoia Capital, is $3 billion in revenue. Is that a return worth bragging about? RIcky Mulvey talks with Fool analyst Asit Sharma about how investors might think about companies’ AI spend. They also discuss: - The rate of improvement for AI models - How non-Mag 7 companies are using AI - And one company that’s spending smartly on the new technology.  Take a look at the Gartner Hype Cycle.  Host: Ricky Mulvey Guest: Asit Sharma Producer: Mary Long Engineer: Tim Sparks Companies discussed: GOOG, MSFT, NVDA, ARM, AMD, ORCL Learn more about your ad choices. Visit megaphone.fm/adchoices

This episode is brought to you by Indeed. Stop waiting around for the perfect candidate. Instead, use Indeed sponsored jobs to find the right people with the right skills fast. It's a simple way to make sure your listing is the first candidate C. According to Indeed data, sponsor jobs have four times more applicants than non-sponsored jobs. So go build your dream team today with Indeed. Get a $75 sponsor job credit at Indeed.com slash podcast. Terms and conditions apply.

The basis of all this is that the processes are much more intensive on servers. They cost a lot more because they're delivering some kind of magic to us that we didn't have before. Now, yeah, there's hallucinations in there. There is perceived inaccuracy of some responses. It's early stages. But so far, companies are willing to invest. And some consumers are willing to pay for this. Many of us are getting it free as an offer just now.

And it is being integrated with the products that we use every day. but the general gist of this is that people are willing to pay for that extra bit of magic. I'm Mary Long and that's Asset Sharma. Artificial intelligence is everywhere and all that data is expensive. A KPMG survey suggests that 43% of U.S. companies with over a billion dollars in revenue plan to invest at least $100 million in generative AI over the next 12 months. But how to tell whether all that investment is actually paying off. My colleague, Ricky Mulvey, caught up with Asset to talk about how to talk about how.

how investors might consider a company's AI spend. They also discuss how non-Mag7 companies are using AI, the difference between large and small language models, and the future of AI friendships. So, awesome. Let's start with the positive. I mean, artificial intelligence represents world-changing technology breakthroughs. It's already, you know, providing a co-pilot. At my job, it's changed the way I search on the internet probably for you as well if you're on Google, which I assume you're not a Bing guy, right? not you're not out there on the Ask Jeeves Yahoo Bing, Bing platforms. You read me right, brother. I am a Google person. I don't have anything against Bing. I do use it from time to time, but I'm a Google person first. I'm trying to get us to this place where we're in the middle

where we recognize the extraordinarily positives and also some of the risks that come with AI in this investment hype, what I would call boom cycle. Let's start with the positive. What's AI allowing you to do that you weren't doing just a couple of years ago? Ricky, one of the things it's allowing me to do is to catch up on a business story that might have changed in a really, really fast way. I cover many, many companies for different services that I work on. You can't keep track from day to day with what's going on every last business. And sometimes I'm called upon to find information quickly and try to make a quick assessment.

In the old days, yeah, I would have to go to Google, SEC filings, interviews, etc. And now I can just ask an LLM, well, brief me on what happened during this time frame when X event occurred. That's one thing. So I think that's really cool. And other uses to use large language models as a surfacing tool to try to find great investments. Let's be real, their reasoning capabilities aren't better than mine or other analysts yet. They get a little bit every day. But they're extremely useful if you know how to sort of point them in the direction to show you companies that might be worthy of further research.

And I think lastly, there's a foreign language I've been trying to learn and just being able to ask chat, TPP, okay, look, here's this sentence. All right. I just don't understand what's going on here with this freaking past participle. Can you explain this to me? And it gives a really cogent answer helps me fill in the blanks. And so I think that's really fun. I think to your point, though, it's good for idea generation, but it requires further skimming. Like, for my job, I'll use it.

you know, I'll ask it, how does this company make money? If I need to like sort of get up to speed on a company that I may be talking about in the future, it's almost it's almost like a, you know, it's like a handyman at times where I've asked it for, hey, I have this thing going wrong with a piece of software that I use for work. You know, what can I do to like accomplish the thing that I'm trying to accomplish? Sometimes it's with audio editing. And then the other thing I'm excited or that I've used it for is cooking. So I have these ingredients in my refrigerator.

What can I do with it? And it can give you some different options. So it's that kind of idea generation stuff that is great. But you know what, Asset is every time I ask it a question, every time you ask it about a piece of investing news, it costs someone money because these artificial intelligence programs are really expensive to run. The infrastructure is really expensive. That's why Nvidia is making so much money right now is because you have so much energy on

the spending side. to a research firm KPMG, about 43% of companies who are making a billion dollars a year in annual revenue. So big companies making billions of dollars are expected on average to spend, just spend, $100 million on generative AI. Companies aren't stupid, generally. We'll just make a general statement, what's the reasoning or fear for spending this much money? Why not take a more wait and see approach? Let's tackle the reasoning part of it first, Ricky. One is that companies of this size are pretty good at understanding where they can get a return on investment. So they'll do a lot of experimentation, conduct experiments, have smart people like software engineers, just try

to run things in a different way. And what they're finding is the small experiments are paying off. So could we save 4,000 person hours by having the large language model solve this coding problem for us rather than hiring people to do it next year? The answer to the experiment may be, yes, we definitely can do that. Second is that the same types of software engineer people, IT people have already identified where proprietary data can be leveraged. So whatever your industry is, you've got something in your data stream. historical data stream, ongoing data stream, that's yours, it's proprietary that you can use to further sales if you can only find the right product fit or service fit or extend your current

lineup. And many companies are starting to see how they can use large language models to make sense of that proprietary data and sell that upstream. So that makes the investment promising. And lastly, there's just a lot of opportunities for cost savings. I mean, if you, Ricky, or me, can now hold up a camera to our fridge and get a recipe out of that. That turns out to be a remarkably tasty dish. We've saved ourselves a heck a lot of time.

Those types of optimizations are happening all over the business world. And then to get to the first part of your question, yeah, I mean, there's a fear case there. There's a FOMO case here. Some companies are making decisions because they don't want to get left behind, even though they haven't really pinned down. the return on the investment. Yeah, there's definitely some pivoting where what was machine learning just a few years ago

makes you an artificial intelligence thought leader now, because people want to see those mentions on the earnings calls, even for non-computing companies. Let's talk about the spend. So, Sequoia Capital estimates that the industry spent about $50 billion on chips from Nvidia to train AI to train these models in 2023. That spending only brought in $3 billion in revenue.

I say only with some quotations, it's a lot of money. It's also 6% on the total spend. So we got a 6% return. You're the CFO of all of AI now. Are you satisfied with that return? Is this inefficiency? Is this overspending? Or is this just the cost of growth in emerging? technology, darn it, and you got to spend money to make money. Yeah, I'm taking the pencil from between my ear and my shock of hair and waving it back at the CEO and saying, yeah, it's about what we expected, boss. Now, I don't mean to make fun of CFOs there. I am an accountant myself, but this sounds to me where we should be in this stream.

Let me give you an example in the real world. If you and I are real estate developers, right, and we're looking at a big parcel that we can develop on the edge of a city where the downtown is peering out a bit, we're going to have to buy that land, put up a few buildings, maybe a commercial building, some housing, but we've got this long-term master project going. So in the first year, the income stream that we can capitalize ain't all that much. You know, we might be losing money, and we probably are overall. We consider the cost of our debt to pull this project off.

But over time, Ricky, businesses move in, subdivisions get built. restaurants come in, then that part of the city gets big enough that the government wants to put a public library there. So everyone can benefit. There are a lot of players that come in. We start making money in year X. We see we've recouped our investment and then we're a little bit off to the races. So this is how when we call it the AI industry, this is how the hypers, the companies that are buying the GPUs, the Amazon's, Metas, Microsoft oracles, and other companies, private businesses as well, enterprise business are treating it like a long-term development and they're not expecting to be making a bunch of money upfront in year one, year two,

year three. Yeah, the internet can make every year feel like a dog year in a way. And it's, I think it's easy to forget that this, this boom has only been around for what, about 18 months now, really since like, I think it was November of 2022-ish. So maybe you're not getting that immediate return. These applications are very expensive. We mentioned that earlier. And for just one use case, I think it's asking, you know, like a chat GPT question versus Google a question.

It's much more expensive to ask chat GPT a question than it is to ask the Google search engine. I want to use that as an example to get into the cost of it. Why are these AI applications so much more expensive to run than the computer applications? applications that we're used to. When we ask Google a question, we're asking basically a computer program to run, a series of algorithms to run that look at an index that's been built up over a long, long period of time. It's a predictable workload. There's this really neat framework of knowledge that Google dips into. It has something called relevance. So the latest algorithms tell it which search results to put up top. But that workload was figured out along.

time ago. And Google knows how to make bunny off of that. When we ask a large language model, say chat cheap ET to do the same thing, the process is totally different. First of all, it's inferential. So the large language model is taking our question and using a probability-based decision-making metrics to give us an answer. It also consults Google. You can see that now when you do certain web searches with different LLMs, different chatbots, they're consulting Google. But then not only this is happening, it's also serving it up to you in natural language, which is another inferential process, which is a harder workload on the server. So that costs more money too.

And then, you know, there's the constant training behind the scenes of the models. There are so many engineers who are working on this and making sure that the AI version of Google doesn't tell you to do something weird with your pizza to cool it down. Yeah, add glue to it. Add glue to it. The basis of all this is that the processes are much more intensive on servers. They cost a lot more because they're delivering some kind of magic to us that we didn't have before.

Yeah, there's hallucinations in there. There is perceived inaccuracy of some responses. It's early stages. But so far, companies are willing to invest. and some consumers are willing to pay for this. Many of us are getting it free as an offer just now, and it is being integrated with the products that we use every day. But the general gist of this is that people are willing to pay for that extra bit of magic.

Well, in a lot of tech adoption cycles, right, you deliver something of value to your customers below cost, hope they enjoy it enough, and then eventually you figure out how to make a profit on it. We've seen a lot of, you know, the rate of growth on these models, these video, generative AI models have been just incredible, Osset, you've seen the differences in video that have come from these tools just a year ago to now. Have you seen the video of like Will Smith eating noodles back when these models first started?

You mentioned this, but I haven't seen it. It's like it's cartoonish. It doesn't quite look, it doesn't look human. It looks very much like it came from a, you know, like a weird computer algorithm. And now there's another image of a man eating noodles. And it looks high definition. It looks like it came from that someone shot in a cafe.

Rates of growth like this always have to slow down. And Christopher Mims of the Wall Street Journal. He's a tech columnist there. He's been a guest on Motley Full Money. I really admire his column and his writing. Basically points out that all of these AI models have been trained on, quote, more or less the entire internet and are running out of additional data to hoover up. There aren't 10 more internet's worth of human generated content for today's AI.

models to inhale, end quote, basically saying that these AIs are going to start learning off each other and that may be a little recursive and you're not going to get the same rate of change and improvement. Do you think this sort of data limit could put a damper on the rate of improvement for AI models or maybe are we underestimating how much improvement we have left to go? So on the consumer-facing frameworks that are iterated. on answers to everything. I think Christopher Mims has a great point here.

I mean, we've already trained most of the prodigious, the biggest LLMs on the bulk of human knowledge that's on the internet, right? There's so much human knowledge that isn't on the internet. But having said that, yeah, it's hard to see how the quality will improve over time without having a phase of degrading first. And we see this in large language models too, as some of their own... output starts to creep in, they get a little dumber, and then the engineers go in and tweak the models, they get a little bit smarter again. So I think it's a great point. But, you know,

for things that aren't trying to be all things to all people in terms of question to answer frameworks, the underlying technology behind this is strong, it's getting stronger. There are so many specialist companies that are working on various ways that information is analyzed and regurgitated back to us. I mean, synthetic data is one solution partial to this problem. If you're a business that has a specific use case, synthetic data is data that's created to train a model like human data or human generated data. And so this could be important, say, in a manufacturing industry where you don't necessarily need private data, but you want to have outputs of a certain process to make something better. So synthetic data is,

going to be more important to such models, then there's also quality over quantity. If you're working on specific questions in, let's say, the pharmaceutical industry, you don't really need to keep feeding more and more information into a model. What you're trying to do is to make sure you have a better tool for analyzing the information. Just a couple of other things I wanted to mention. There are small language models which don't need as much information or even processing power to run. Like Microsoft has a couple that are pretty interesting.

And these are solving the problems of large language models through reasoning. So some models are trying to get better at reasoning the way our minds reason. And that could be a way that all of this ecosystem gets a little better without having to digest so much information and shove a bunch of information down a pipe and get smarter. Instead of information, let's work on the thought. process. Let's work on how these models think. And then finally, of course, you've heard me say this and other analysts say this before. Proprietary data sets for companies are very powerful. I was just talking about this a moment ago. And that's another case where, at least in business, you don't

necessarily need to worry about the problem that Chris Williams is talking about. You're focused on your data stream and how you can derive insights and make better products from that. I want to talk about small language models for a moment. A lot more familiar to. with large language models, how might someone encounter a small language model? What are these looking like? What kind of problems are they addressing? Yeah, so small language models, I think we'll see them more in the real world now. But I think that Microsoft, who I name, has a couple of these.

We're going to see these with devices in the future. So you can spin a small language model from a device that merges with chip design. So companies from ARM to A&M.N. to Intel are designing their chips to have some neural processing units right on the chip, right on the phone. So when you combine these two concepts, you get devices that are smarter. They don't necessarily have to go to a server out in the cloud to answer a big question. But they could tell you, Ricky, if you take that photograph of your fridge, how to make a

better recipe based on information that's already stored on your phone themselves without you having to go to a chat. GPT or whatever model you're using. So a bit of reasoning capability plus information that already exists on device plus information that's taken in spins out something that's reasoned and solves your quick problem. And then to your earlier point, yeah, there are times where synthetic data would work for something like a digital twin. You have a manufacturing floor and it doesn't matter if you're using recursive data. You're just trying to make the model better so you can make maybe an assembly line, you know, take two, you know, two tenths.

of a second quicker than it normally runs. Totally. I should say we're already at that point that Jensen Wong predicted a couple of years ago. And he said in the future, there's going to be more virtual robots than there will be physical robots. Maybe we haven't reached it quite yet, but we're headed there pretty quickly. We'll see if Elon gets his way with the humanoids. That's a separate show, though.

And we talk often about how the, you know, the MAG-7 companies seem to be using AI. You know, there's clear winners, which if you're a smaller company and you need to build an LLM, you probably need NVIDIA's chips for that. Maybe you'll use AMD's chips if the Nvidia chips are not available for sale. But I'm curious about how the other companies seem to be using it. And it breaks down into a few categories. It comes down to essentially, I would say, personalized chatbots. So if you go on Airbnb and you need help with something, an Airbnb chatbot might be able to help you a little bit quicker or a little bit more efficiently. than a human agent will, I should say more efficiently for the business.

Simplifying contracts, that's how Cisco is using it. Co-piloting, Microsoft is famous for this. I've already used a Mag 7 in the example, but Microsoft co-pilot on Office 365 to help you write emails a little bit more quickly. And then forecasting is another example where if you're a retailer and you want to forecast how many jeans you're going to sell in, let's say, Indianapolis, Indiana, in May, and you can pool all of the data that you have on similar regions, Indianapolis, and then maybe make a better prediction about how many genes you need to sell to your retail store.

Any reflections on these use cases or anything else you'd want to add for these smaller companies spending money on AI? They're all fun. Let's look at that personalized chatbot one. It's intriguing to me into it, which you and I had talked about before the show, is really interesting because they have those proprietary data sets that are in consumer-facing applications. So if you use QuickBooks online, they have information not only about you, but about every other business in your zip code that's similar to you.

And this is interesting. I had a conversation with the CEO of Inuit here on Motley Fool Money. It's about a year and a half ago, Sasan Gaddaarzi. And he was telling us, Rookie, that, look, we're going to combine all this interesting data. and help you learn how to manage your cash flow better. Are you running out of funds? Well, the chatbot will be able to tell you, look, every other long-wing business in this zip code doesn't spend quite as much on seed.

Maybe you're buying a premium seed and your profit and loss would be better if you drop down one level. Now, he didn't use that very specific example, but he was spinning up a few similar types of scenarios. And I think that's a great illustration of a use case of a company that, Yeah, it's big, and two, it has a big balance sheet. They could go out and buy a bunch of GPUs, but really what their focus is, is let's make something that will help our end user, the consumer. I think that's going to make them some money over time. And I do like also, you mentioned copiloting of code.

There's so many companies that are offering co-pilots now in various walks of business life. And this is something not just that we'll see on the consumer side, Ricky, but people in business organizations will see. I think it almost becomes commoditized though. At some point, these models become so powerful. We don't even think of it as co-piloting anymore. It's just part of our daily routine. You just type in or talk to or show an image to your personal co-pilot for whatever your workflow is in any industry and it's going to automatically do things that advance your day. Now, hopefully that results in a little more free time. in our day doesn't mean that we don't have work, but we'll have to see what that future looks

like. I mean, the Jetson's future has always been promised with every advancement. We'll see if we get there this time off. It had such snazzy music. I could not help but think that that's an optimistic case because of the Jetson music. But now we grow up, Ricky. We're no longer watching the cartoons and now we wonder, is the music going to be really peppy? Is it going to be a little ominous?

I don't know. time for real work. You hit on two, basically two areas that Sequoia Capital is bullish on, which is customer support and AI enterprise knowledge in the near term. The other area that Sequoia is bullish on is AI friendship, pointing out that when people use these AI chatbots as friends, they have maybe stickier adoption rates, which I understand the business perspective, but also a little concerning and terrifying because, you know, hey, let's find, let's find more friendships through these LLMs that really know me for me and, you know, you don't necessarily have the consequences of human friendships and relationships. That's one thing to flag. Any companies

impressing you with how they're spending on AI, we've talked about the amount of spending, we've talked about how companies are creating these new applications for customers and businesses, but are there any companies impressing you with the way that they're spending on AI in this boom cycle. Wait a minute, Ricky. I'm going to answer your question. Yeah. But you can't bring up such an intriguing point for me and our listeners without, let's just

talk about two seconds. We got time. Let's talk about AI friendships. I find it also fascinating. The transformer model, the technology that underlies all LMs is just sneakly good at figuring out what's important in a group of words, in a text, in any kind of visual data you give it. that's why they're so powerful and I'll use this word again, so magical.

But it also means that they're sneakily good at understanding the buttons to push to get certain responses. So if you build an AI with a lot of data to be able to manipulate emotion, you can develop AI that many people will respond to and feel up to a point. Like, this is very similar to relationships I have in real life. and actually I'm lonely and I want to go home and talk to my bot. I find that interesting. I don't know quite what to make of it. What do you think about this?

What do you make of it? I mean, I think it's hugely concerning at a societal level because it's going to offer people, you know, sort of facsimile of human relationships that may be serviceable to you. What happens when, you know, you don't need to go out on a date because you have a chatbot. who's interested in you and nice to you and always texts you back and is interested in the same things that you are. And that's so hard to find in the real world. The chatbot is never going to betray your trust, right? I think that it's one of those things that seems silly on the

surface level. And there's a movie, Her, that kind of covers it a little bit. But it's going to be one of those things that sort of continues the trend of, I would say, social media in a lot of sort of life becoming more digitalized, which is that, you know, people aren't going out as much as they used to because you can get a, you can still get those dopamine buttons pushed on your phone. I think these AI relationships are going to continue to be an amplification of that. The police said, or maybe it was sting, a scientific means of bliss will supersede the human kiss. And this is sort of that coming true a little bit to me. I also find it unsettling.

I mean, I think, too, the pendulum swings. both ways. I just see younger people getting fatigued of social media and going out more. I think there comes a point where the emotional support you may get from a bot just starts to feel synthetic and you crave that human touch, that human conversation. But what it means is to get to that point, many people, you know, may dive into this stuff just as they did with social media. And that's a scary thought. And then we should also say on a societal level, it may be very helpful for some people who are, say, elderly or otherwise, you know, unable to do what they used to do in terms of social connection. So it may provide a use there, again, not a perfect use,

but of some comfort or to the sick. There are some really interesting philosophical questions have been posed over time by people who thought this out before the technology reached the point where it is today. Yeah. So, Austin, how about that for a setup for a, uh, for a company discussion about which companies are spending on AI in an intelligent way. I'm looking at the outline. You have one. We've talked about on the show previously. But what company is impressing you with how they are spending on artificial intelligence?

Yeah, I'm going to be boring here, but I'm going to say it because it's true. I think Oracle is really impressing me and some other investors. They had earnings recently come out, and I think the stock was up 12 or 13%. But all along, Larry Ellison, the chairman of Oracle and Safra Katz, the CEO, have been saying that their technology that Oracle has, RDMA architecture, is just faster. It's less expensive for companies to train on AI. And look, they just signed a bunch of contracts this last quarter and added in the tens of billions to their future revenue that they're going to recognize out of these. So look, there's some of that return coming in, Ricky. It wasn't in the first year, but companies like Oracle, they're going to see that return.

So I think that's a really smart case. They had a little bit better technology than some other web service providers. I would say that in some ways, their server configurations are technically superior to some of Amazon setups, some of Microsoft setups. And that's why those companies are forming relationships with Oracle. So there's one. And I think that you and I were chatting about the hype cycle. And I think, you know, me personally, and I love to hear your opinion, I think we're sort of where that troughed is.

Look up those of you who aren't familiar with this, the Gartner hype cycle in particular, because these are sort of industry experts who really nail what a hype cycle looks like. So according to the way they label these, I think, Ricky, we're somewhere between the trough of disillusionment and the slope of enlightenment. So where people are like sort of past the hype and many investors are starting to recognize that, boy, there's only a few concentrated winners in this game. So they're getting disillusioned. But that's a great point of the cycle because it means that there are companies that are working on stuff now that we're going to admire and invest in as time goes on. Where do you think we are in this cycle?

So I like how you separated that out for the biggest companies and then the smaller ones. I think for the smaller ones, we might be moving more toward the trough of disillusionment. But for those big companies like Nvidia, where at the time of this recording, their market cap is around $100 million per employee, that sounds like some inflated expectations to me, Osset. Yeah, probably. And this is not news to anyone, but Nvidia is a cyclical company. So at some point in time, demand supply will hit equilibrium.

And we've seen this story with Nvidia. I mean, they're going to sell off some. Over time, I think it's still a smart investment, but sure, things are so concentrated now. But that bodes well for the future if this suburban development works out, Ricky, because I've seen them where they were just sort of middling or they hit the wrong part of real estate outside that city center, which was piddling around a bit. Awesome. Thank you so much for your time and your insight.

I love these longer form conversations. Thanks for being here. Same here, Ricky. I appreciate it. As always, people on the program may have interests in the stocks they talk about, and The Motley Fool may have formal recommendations for or against, so don't buy ourselves stocks based solely on what you hear. I'm Mary Long. Thanks for listening. We'll see you tomorrow.