No Priors: Artificial Intelligence | Technology | Startups - Cloud Strategy in the AI Era with Matt Garman, CEO of AWS

Episode Date: August 29, 2024

In this episode of No Priors, hosts Sarah and Elad are joined by Matt Garman, the CEO of Amazon Web Services. They talk about the evolution of Amazon Web Services (AWS) from its inception to its current position as a major player in cloud computing and AI infrastructure. In this episode they touch on AI commuting hardware,  partnerships with AI startups, and the challenges of scaling for AI workloads. Sign up for new podcasts every week. Email feedback to [e-mail] Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil  Show Notes:  (00:00) Introduction  (00:23) Matt’s early days at Amazon (02:53) Early conception of AWS (06:36) Understanding the full opportunity of cloud compute (12:21) Blockers to cloud migration (14:19) AWS reaction to Gen AI (18:04) First-party models at hyperscalers (20:18) AWS point of view on open source (22:46) Grounding and knowledge bases (26:07) Semiconductors and data center capacity for AI workloads (31:15) Infrastructure investment for AI startups (33:18) Value creation in the AI ecosystem (36:22) Enterprise adoption  (38:48) Near-future predictions for AWS usage (41:25) AWS’s role for startups

I don't know how many times in the first couple of years I had to explain why a bookseller was offering compute services and storage services. There's not a lot of business opportunities that are as big as cloud computing and as potentially transformational. Original AWS thesis was we'd take care of the muck so you don't have to. I want my customers to want to run on us. Hi, listeners, and welcome back to No Pryors. Today we're talking to Matt Garman, who took over as CEO of AWS in May. Matt has been with AWS since it was a $0 billion business to today's $100 billion run rate business. Welcome, Matt.

Thanks so much for joining us today. It's a real pleasure to have you. One thing that I think is really fascinating is you actually started on AWS in the very early days it was just getting started. And you did that as an intern while you were getting your MBA. Could you tell us a little about both the origins of AWS as well as your own involvement with it? Sure. So it was in 2005. I did my business school internship at Amazon.

and as we were looking around for projects, I talked actually at the time to Andy Jassy and he was telling me that he was starting a new business inside of Amazon that was technology focused and he couldn't tell me about it before I started and I thought it sounded interesting so I joined them and worked on that

and it was AWS pre-launch I got to work on that as intern. It was a super cool opportunity. I came back full-time effectively as the first product manager for AWS and I'm now on year 18 or just, you know, so I've been working on the business the entire time. And, you know, it's a fascinating space.

Even back then, as we saw the potential of what AWS could be. Obviously, that was, you know, it was a startup, right, inside of Amazon. And so it was, you know, we had visions of what we thought it could be. But, you know, there's a lot of hard work and some good luck and some things that have gone right for us and a great team over the last couple of decades to go build AWS. us. And the fascinating thing today is it's still in the very early stages of what the business can be. There's not a lot of business opportunities that are as big as cloud computing and as potentially transformational to every industry out there. And so there's an exciting place

to work just as much as it was in 2005 when I was an intern, kind of writing the original business plan of who, when we first launched our services, which companies might possibly be interested in using these things. What were the other projects that you were offered at the time. I'd worked at startups before going to business school. And part of what I was looking for is I wanted to see how larger companies did new projects and kind of entrepreneurship, if you will, inside. So there's a couple of few technology companies that I looked at and was excited about Amazon. And there was a couple of other kind of retail businesses that, I mean, the internships were mostly in retail. And, you know, there's some new categories that they were

starting up and things like that that could have been interesting. But I always also knew I wanted to go back to technology. So this was far and away. It's what convinced me to come to Amazon because it seems so exciting. How well fleshed out were the original plans? Because I ended up using AWS for my first startup in 2007 or 2008. And it was pretty new and there's a small subset of services at the time. And to be honest at the time, Amazon wasn't thought of as deep of a technology company as it both was and is.

And AWS was really kind of like, wow, Amazon is doing this thing that, you know, at the time you maybe it would be natural for Google or somebody else to have started with. And so I'm a little bit curious about that sort of early conception and roadmap and what was in place and what you added over time. I don't know how many times in the first couple of years I had to explain why a bookseller was offering compute services and storage services. I think the approach that we took worked out to be one of the key pieces to our early success. I think if you look at some of those others like Google and later Microsoft,

they kind of went at this space for first we were the first ones out there that had anything like this but even soon after that I think they went after this space like that was going to force the developers to change how they build applications and kind of build them in a new way

and we went after it as in we're going to build building blocks and let developers and builders go build interesting things and so our view and this started really from internal Amazon if you scroll back all the way to 2003 or so Jeff Bezos basically

mandated across the company that in order to move from a big monolithic stack that wasn't going to scale anymore for Amazon, we had to move everything to services. And that was really a lot of the impetus for us looking at after that, and we saw the success of that, we said, well, maybe this would work for other people who are likely going to have the same struggle that we're going to have as Amazon. And so we thought, starting from first principles, what are the things that we would need to go build to help people build a company? And we knew people needed compute. We knew people needed storage, we needed databases. And so we built those things, right?

And we didn't force people to change how they architected, right? When we gave you a virtual Linux server, when you logged in, it was just a Linux server. Like, it wasn't magic, you know, it auto-scaled and you got it in 30 seconds, which was pretty awesome at the time where normally it was going to be six months or something for you to go get a server. And this is when everybody was like, when you had a startup, you had to go to Exa Data and buy racks of servers in order to get your startup off the ground. And so that was obviously transformational,

but once you actually got the infrastructure, it kind of operated the same way that you were used to. S3 was a little bit different, right? The put-get delete was a little bit different, but the storage concept wasn't too different. And so I think that was one of the things that really helped us move quickly and helped people not have to have a total paradigm shift

of how they develop. It was more how they get the infrastructure to develop on. And then over time, we could add things like Lambda or we could add things like Bedrock and AI services and other things like that that are more different that people weren't used to, but kind of starting out with some of those things people were familiar with

and they could instantly grab on and build quickly without having to learn a new concept, I think was a really big accelerator for us the beginning. Yeah, it was huge. I remember when it first came out, I thought there was a few things that were done really well. One to your point is just, you know, providing really basic building blocks.

Second was just the iterative nature of it where you launched with a small number of services and then you kept adding stuff that were kind of the obvious next steps and early on people would wonder, or at least I in the startup community wondered, will they end up with everything I need? And you very quickly did. But to your point,

you also built it in a way where before that, I think most people building today, I have no idea. To your point, you literally would have to set up your own server rocks or find somebody who would do it for you. It was hugely painful. You had a whole team that was doing that for every company.

And in some cases, it caused real problems. My company eventually got bought by Twitter, my first startup. And then at Twitter, we ran into real issues in terms of data center capacity and planning and all these things that you can now to scale on Amazon for.

So it's pretty radical in terms of what's been enabled for startups and the decrease in headcount per company that's needed to associate with that. I remember as late as 2010, 2015, maybe you guys are still having this conversation with the largest customers that, you know, if you're, let's say, a large financial, you had like this whole platform team, several different iterations of it. And the line I would get from people would be like, we're never going to do that public cloud thing from a security perspective. you can't compete with us on cost. Our platform is better. Like, you know, it's not reliable. And just, like, there's a very strong orientation toward skepticism of this, you know, even in 2010, like, upstart company versus like, oh, we are X large financial customer.

I think almost everybody has seen the light at this point. But, you know, you described Amazon, AWS in particular, is still being quite early in that journey. And I think one thing that people, even those in the tech industry who experience more exponential growth than most, like thinking about how large these markets become is really challenging. And we might be, you know, I think we're at one of the, at the beginning of one of those cycles again with AI will come to that. Like, at what point said AWS internally did you guys like know that this was going to be that large? And how did you talk about size and opportunity early on. And just to give scale real quick on this, I think you guys went from

something like 500 million or so in 2010 to about 90 billion last year in terms of revenue for AWS. That's right. And so it's just, it's this insane ramp of 89 and a half billion dollars in incremental revenue over 14 years or whatever it is. So that's amazing. Yeah, it's, it's easy to get caught up in those big numbers and that the fact that it's so early. And so, you know, back to your question on when we kind of knew we were, we were kind of on that track. I remember it's probably 2008 or 2009. I can't remember exactly, but I definitely remember the trip.

We went on a trip to New York, and actually there's a bunch of financial services companies that Goldman Sachs and J.B. Morgans, and they wanted to learn about, what is this cloud computing thing? And they mostly were just fact-finding. They were trying to get information from us on how they could more efficiently run their internal IT systems,

I'm pretty sure. But we went in there, we're like, well, it's worth a shot. And, you know, they were like, okay, look, our workloads are never going to run you. Like maybe someday our website will run. or something like that, but, but never any of our internal workloads. And, you know, and we listened to them and we said, why? And then we said, awesome, tell us why. And they are, okay, we have to have this compliance. You have to meet this rule. You have to meet this thing. We have people to do

these audits. And we spent the next decade just checking those things off the list. And we basically never said, you know, our part of what we did is we said, we want to know what are the most difficult workloads to run. What are the hardest things to do? And let's go solve those. Because if I can solve the, you know, JPMC running in AWS, if I can run, solve the U.S. intelligence agency running in AWS, like the reasons for regular other companies are diminishingly small. And so that was kind of our mentality as excited. And I love getting the startups running up to us, but the big enterprises can kind of easily dismiss. You're like, well, it's a startup. You know, they don't have all these things. And so what we did is we just

did both. We said, look, we're going to go in as much business with startups as possible as we can. And we're going to check off the list all the things that are going to help JPMC or the U.S. government or Pfizer or whoever it is run on us in a secure, safe way. And that's what we did. And today, those are all huge customers of ours. After I was in AWS for about a year, I remember sitting down with a friend of mine, who was a business school classmate of mine, is also working a different part of Amazon. And he was like, oh, how's that AWS thing going? And I was like, you know what? I think this thing could be a billion dollar business. And he looked at me and he's like, he's like, dude, do you know how big a billion dollars is?

Like that seems unlikely. I was like, no, no, seriously, I think we could get to be a billion dollar business. So, you know, we knew it was going to be successful and we didn't know, you know, quite how is successful or when, I would say. And so, you know, now that we're at a hundred billion run rate, you look at, you still go out there. And I think 85% of workloads are still running on prem today by most estimation, somewhere in that range, you know, pick your number, whether it's 80 to 90, whatever it is. Like,

that's enormous. Like, if I have, there's still 10x growth of just existing workloads. Forget all the new Gen AI workloads that are being created every day. These are just existing workloads to move. There's a, there's a 10x number in there. And so that, that business is massive. And I think there's a couple of, you know, one of the big inflection points we saw is we went after the intelligence agencies for the U.S. government. And we won that contract, and it was secret. And it, you know, we, we push really hard to go in that. It was against all the incumbents, HPs and IBMs and on oracles and whatever. And we won the contract for to do this cloud workload. And, uh, but it was, um, confidential. And we couldn't share with anybody that we were

doing it. And, uh, IBM suit. Yeah, I remember this to, uh, because they, they said it's unfair. It wasn't. And so then that became, so then it became public that we won this deal. And then the intelligence agencies went out to public and said, no, AWS is the most technically sophisticated. They have the most capabilities. They're the operationally strongest, and that's who we're going to go with. And so we had the government out there now saying that we're the best and most technically capable to run these highly important workloads.

And that was a huge kind of stamp of approval for us, if you will. And I do think that that's one of those moments that in some ways we kind of got lucky that they sued. Otherwise, it might still be secret that we were doing that. And I do think that that helped gain a lot of credibility in the enterprises. What do you think is, you mentioned that

80% of workloads still haven't migrated over. What do you think are the main blockers to that today? Is it just momentum? Are there specific features? Are there big things still to build? There's some technologies that, you know, I think look, if I had an easy button and by the way, we're trying to build an easy button,

but if I had an easy button that would just migrate mainframes to a modern cloud architecture today, almost everyone will push that button, but it doesn't quite exist today. And it's not as simple as like, great, I'll go run your mainframe in the cloud. Like, that's not what customers want. They want to actually modernize those workloads and have them into, you know, microservices

and canaerize workloads and other things like that. So that's one, is there's just a bunch of workloads like that that are old and their customers running a big SAP thing and they want to move it to the cloud, but it just takes time because it's tied into a bunch of other things like that. There's also a bunch of workloads that as you get out of core IT workloads that are in line of business, that are the next set of things. And whether that's, you know, say, telco workloads, right, that are running kind of the 5G infrastructure around the world. We've slowly been moving those to the cloud and helping those customers get that flexibility and that agility of running those in the cloud as well, but they're slower to move.

If you think about all the compute that runs factories out there today on factory floors, most of those have not been modernized. Most of those are thinking, and there's a huge opportunity, by the way, for AI to totally revolutionize how you think about factory workflows and efficiency. there. But a lot of that hasn't moved. And so some of this is, you know, there's an on-prem infrastructure that people are still amortizing. There's people who's, there's still people whose job it is to run on-prem data centers. And so they're kind of resistant to moving things. So, you know, there's a bunch of factors in there. And so some of it is just takes time. Some of it

is technology pieces. Some of that is we still have stuff to go build and innovate and help make it easier for customers to do that. I'd love to hear about just the initial investigation of like generative AI as a technology change and like how AWS began to react to it and invest in it. Because to some degree, it puts us all back in the like on-prem Kolo era of the world where to get one of these, you know, if you're doing any sort of real pre-training to get your startup off the ground, you're back to, I guess I'll buy a bunch of DGX boxes somewhere. And like, I'm I need to think about the cost and management of that. Well, actually, most people are still buying those, but in the cloud,

but it is kind of a, it's not a serverless type of a thing. It's, you know, most people are still not buying, you know, H-100s and hosting them in a COLO or anything like that. And increasingly, I think that's going to get harder and harder as you move to liquid cooling and larger clusters. But, you know, it is a, it's a super interesting space. I think we've been working on this space for how many years now. And look, we've been investing in AI.

broadly for the last 10 years. And it's why we started five or six years ago investing at the infrastructure layer and building our own processors because we knew this was coming. We saw this path coming and we knew that that's also not a short-term investment. So that's one of those things you've got to invest way ahead.

And then we were investing in building generative AI models. And then, you know, Open AI kind of made a generational leap forward with what they were able to do and what's possible. And then many people have talked about this, but it really in some ways was a discovery as much as anything about just what was possible and kind of unleashed a new set of capabilities. And so we actually, as a business, took a half a step back and said, okay, these are going to be transformational abilities.

And assuming that this technology gets better and better and better over time, how do we make it so that every company out there can go build using those technologies? And so different than how can I go build a consumer application that people are going to be interested in, we kind of took it from the point of view of a data. WOS, right? Like just what are the building blocks that I can help all of our customers, whether they're startups, whether our enterprises, et cetera, go build interesting generative AI applications. And so we started from first principles. Customers were going to care a ton about security. That's not going to change. They're not going to all of a sudden not care about securing their infrastructure. We also had this hypothesis, two more hypothesis, one that the idea that there wasn't just going to be one model. We thought that there was going to be a lot of models for a lot of different purposes and there'd be big models and small models and people would want to come buying them in new and interesting ways. And I think the last two years have probably played that out. But I think when openly I first launched, it wasn't as obvious. But that was kind of one of the bets that we made. And then the third one is that we view that every enterprise that was

building on us, the interesting IP that they were going to bring to the table was mostly going to be their data. And they were going to care that their data didn't leak back into a model or escape from their environment. And so we built a bunch of what we did, starting from those principles, of how do we make sure that these things are secure, that their data is secure, that they can have access to every piece of technology that customers need to go build interesting applications, and they can do it in a cost-effective way.

And so that's how we approach the space. And I think we now have a platform in bedrock, in trinium chips, and inferential chips, and then a bunch of the other capabilities around, as well as the suite of models that we offer, both proprietary as well as open-source ones, or open weights ones, that I think we're starting to see that, really that momentum pick up.

And we're seeing more and more customers really like that story. They like that platform to build from. And we're seeing enterprises really lean in and want to build in that space because it gives them a lot of that control that they want as they go and build applications. How much do you think it matters that AWS has, let's say, like first-party models it offers its customers, because that's clearly a strategy for some of the other hyperscalers. Google, that's obviously their strategy. They're really the only one that has a first-party model today of the other hyperscalers.

Microsoft's done a good job of co-opting Open AI's innovation, although in their last, I saw recently they listed Open AI as one of their biggest competitors now, so it'll be interesting to see how that all plays out. But, you know, for us, I think it's important, which we do. So we are building our own first party models. We have our first party models today. In fact, the Titan Embeddings model is by far the most popular embeddings model that we have inside of Bedrock today for people that are building search indices and thinking about things like that. And we are building larger and larger models as well, first party.

I think it'll be important, but not critical. I mean, I think people love using Anthropics Clawed models. Those are fantastic. And right now, those are the best performing models in the world, which is fantastic. We just launched Lama 3.1 on the day it launched, and we have a really tight partnership with META, and their open weights model is fantastic. And I think increasingly we're seeing customers really love that open weights model

because they can go in, particularly enterprises, customize it, they can do fine tuning to it, they can add their own data to it, and really customize and distill and do some interesting things. And so I think that is super critical. You know, we're seeing kind of specialized models, if you will, where we see folks like Adobe building Firefly, is all built on top of AWS, purpose built for their own thing that they're building.

You know, whether the Amazon purpose built models are, I think they're an important part of that. Mostly it's partially it's for us for learning. Some of it's for powering our own applications and some of that may be for end customers, but it's all kind of that diversity of option, honestly. Like we want there to be the best set of options and we want them all to run in AWS, and so we want those workloads to run there. And so to the extent that we can do something novel or interesting with our own first party models, we'll do that. But we're also delighted for our partners to run as well.

So I think it's a little bit of both. Yeah, it's really interesting. Alyssa Henry, you know, long time, AWS leader as well, is a friend. And over the years, I would like ask her about some interesting new open source project. And she, you know, it was the most terrifying thing, honestly, because she would always be like,

AWS loves open source. We make more money on open source. any open source company does, which is, you know, if you think about all the advantages that AWS has, even if you're very friendly in the ecosystem, it, you know, can turn out that way. And so I think the C change that's happened in terms of availability of open weights models and multiple players here, Mistraw, Lama, et cetera, that are very competitive is like, I think, you know, a huge, huge boon to AWS's a general open ecosystem model. We've always leaned in to open source.

We're huge contributors to many open source projects, and we lead many open source projects. And I think we do a good job creating and turning those into businesses for our customers and helping run managed open source projects. And so it's a big area for us. One of the reasons, frankly, is that we've long said, we don't want customers tied to AWS because they're locked into some proprietary licensing piece. We want them to be able to. I want my customers to want to run on us as opposed to kind of locked into a Microsoft license

where you're held to some different license that you can't get off of easily or old school kind of like Oracle database that you can't get off of. You know, we want people to be able to run. And so even where we have something like Aurora, which is our kind of managed database, it is 100% Postgres compatible. And if you take that code and go run it at a Postgres database somewhere else, it won't operate as well, you know, because we do a great job at that. But it runs. Like in theory, if you run it as well, it will operate as well. And so, you know, that's how a lot of our services are built and how we think about things. We'll support proprietary things as well.

And, you know, at some level, there are some services where you've got to take advantage of the cloud where, you know, there are some proprietary things that customers can use, but things like Dynamo and other technologies like that. But we really embrace open source and I think it's been beneficial to the whole industry, And it's a way to get better security, better visibility, and kind of that license portability, I think, is a key aspect as well. You mentioned that, you know, the model side of the AI world. And there's other main AI building blocks. There's rag and, you know, there's certain aspects of fine-tuning and other things that people are increasingly doing over time.

There's other parts of it, like Eval suites. And what are the main building blocks that you can talk about in terms of things that are either coming to AWS or how you think about that, more fragmented world of all these different components and how they fit together relative to AI workloads today. That is kind of the idea of bedrock is that we want to make it easy to do. And I do think actually in many ways today the models is the front and center thing that everybody pays attention to. But I think increasingly it'll become a smaller percentage of the thing that people pay attention to because people are going to care about whether it's rag or some other sort of knowledge base. So we call it knowledge bases because it's, you know,

the technology may change over time under the covers. But that like how do you have a grounding set of truth that you use. I also think grounding data is an interesting thing for, for like real-time information that you want as part of your AI systems. We have things like guardrails, which our customers find an incredibly important because, you know, if you're building a chat butt on a financial services website, you can actually find a lot of money if that thing starts giving out financial advice. And so you really want to be able to control, let alone, you know, going down and talking about politics or something else that you definitely don't want to talk about. And so those guardrails are super important is people think about what they want their AI systems to do and interact with

and where they want to stay away. This is not controversial. I'm sure you both hear a lot about this. But again, the next generation of, and the next step forward and what we can get out of AI systems is going to depend a lot on how well we can integrate agentic workflows and actually get these AI systems to do things, not just kind of summarize and tell us information. And so building in the ability to have agents as part of that workflow is a big area, for us and want that to be easy for you to build as part of that bedrock capability. I do think that pre-training and fine-tuning is going to be something that more and more customers are going to want to do, as well as distilling over time.

Because I think I was just talking to a couple customers earlier today that are very focused on how do I get this model down to a much smaller thing so I can put it on an industrial edge or somewhere like that. And so how do you think about distilling down so I get the value of what I want? I don't need the whole kind of reasoning engine behind that. And I think there's a long roadmap of, like you said, kind of model evaluation and other things like that. And some of that is us and also some of that as partners, by the way.

And so we're, you know, AWS has been a place where I think part of the thing that has made us successful is really embracing the ecosystem to go build around there. And so thinking about labeling data as an example, we have a deep partnership with scale AI to come in and help you label your data if you're going to be doing any fine-tuning or pre-training or things like that with your data. we partner with folks like lane chain to put together some of those agent workflows other things like that. And not to mention, of course, the model providers who are super important partners of ours as well. So I think it's all of those things. And our job is to how can we make it easier and

easier for you to go build those applications in a tightly coupled way so that it's easy to go use those different components, easier to innovate rapidly, and easier to build the proprietary data that you have as part of your AWS data lake so that you can kind of pull that in because, frankly, most of these generative AI systems aren't going to be super useful if you don't have interesting data to go pull from. The other place that a lot of people are spending time right now in terms of bottlenecks to utilization or usage or future proofing is actually more on the chip side or semiconductor or system side and then in terms of DC capacity and obviously you all have been building tinium chips and other things, which I think is really

exciting to see that evolution. How do you think about future GPU shortages? Does that go away? when. I'm sort of curious about how you think about forward-looking capacity and is the industry actually ready in terms of building out data centers, building out semiconductors, all the rest of it. Packaging, you know, the whole one. Look, I think we're probably going to be in a constrained world for the next little bit of time. Just, you know, that some of these things are, they take time. Like, look how long it takes to build a semiconductor fab. Like, it's not a short lead time. And that's several years, and TSM has run fast to try to ramp up capacity, but it's not just them. It's the memory providers and the, and frankly, data centers that we're building, right?

And so as we think about, there's a lot of pieces in that, in that value chain that I think, as you look at the demand for AI, which has been, how an exponential might be undershooting it, some of those components that support that, I think, are catching up. And I think AWS is well positioned to try to do that better than others are? You know, we've spent a long time thinking about the last 18 years learning. How do we think about smart investing? How do we think about capital allocation? We've spent a bunch of time thinking about how do we acquire our own power?

How do we ensure that it's green and carbon neutral power? All super important things. And we're the largest purchaser of renewable energy over the last new contracts, right? So actually going out and adding and supporting new renewable energy projects, we're the largest provider, I think, each of the last four or five years. And so we've been leaning into that for a while to ramp up this. And there's just a step up. And so I think we're thinking about, you know, how are we acquiring enough power?

Our own chips is a way to support the growth of Nvidia chips. And so I think the more diversity there, the better off we are. We're a huge partner of invidias. We, you know, Invidia actually runs their AI training clusters in AWS because we actually have the most stable infrastructure of anyone else. And so they actually get the best performance from us. And we love that partnership and we have a great and growing relationship with them. And, you know, we think things like Traneum are a good diversification. And I think there will be some workloads that run better on Traneum and are cheaper on Traneum over time.

And as well as Inferentia, I think Inference is one of those workloads that. that today it's, you know, 50-50 maybe of training and inference, but in order for the math to work out, inference workloads have to dominate. Otherwise, all this investment in these big models isn't really going to pay off. So hopefully for the industry, that all happens. But I think we're probably going to be tight for the next little bit of time. And so, you know, because the demand is almost infinite. I mean, it seems infinite right now. How does AWS think about making investments in data sensitive? of this scale to train the next set of foundation models, right?

And because I think you could take a, you know, AWS is very educated a player. You could take a proactive approach. You could take a customer-driven approach. But the idea that there are individual players who want tens of thousands of nodes at a time and interconnected GPUs is like a sort of a new demand vector. Some of the demand for some of these really large models is very large, right? I mean, they're talking about needing gigawatts of capacity.

which is kind of a mind-boggling number that some of these models need. We're doing both proactive as well as customer-driven, right? We try to balance, because there's real capital outlays that are required as part of this, of course, and we're talking tens, if not hundreds of billions of dollars of capital investment. And so, you know, we think about it as how do you make the right investments in things like land and power and other things that are fungible and could potentially be used for other things if eventually demand changes or the slope changes, as well as then having visibility into the supply chain for more near-term things that you'll need like servers and chips and memory

and other pieces like that. And so we balance a bunch of those things, managing the financial applications of what we need to go by, as well as the long-term customer demand. We try to map out, how do we meet some of those match? And some of our customers give us long-term commitments to help with some of those things. and we give better rates for customers that give very large long-term commitments for some of that capacity that requires a lot of investment. But, you know, it said there's a lot of air bars on that, too, because at anything that's

growing at multiple hundreds of percent year over year, you're not going to nail that number appropriately. And so we try to have enough buffer in there that we can support upsides when they happen and manage if it's a little bit less than we thought. As someone who's seen just many generations of startups, decide like what investment they want to make in infrastructure. That is suddenly a much more important question to a generation of AI companies than it has been in recent history. What like advice would you have for them as the man holding the data center, I suppose? We've had to go on

this before, right? We started from a hundred million dollars of revenue to a hundred billion dollars or well, I started I started when we're at zero dollars of revenue to a hundred billion dollars of revenue. And so, you know, we've had this kind of rapid growth before. We think about how do you balance some of those pieces? And I think part of that is, is how do you make sure that for me, I think as you're as a startup thinking about this is how are you thinking about investments with a real plan of how do you have monetization and not assuming that there's always more VC funding to come and bail you out? And so kind of having a plan there to flexibility of, you know, how can I start monetizing sooner if I need to? And where can I keep

investing if that's the part that I'm in that the traction makes sense? Because I think, look, the only reason that any startup goes out of business is they run out of money. That's the only, that's as simple as that. As long as you don't run out of money, you're not going to go out of business. Obviously, easier said than done, but I honestly think some startups kind of forget that. They're always like, that's no problem. I'll just go raise more. And kind of remembering that just because there's like a hype cycle doesn't mean that someone's going to give you another billion dollars six months later. And I actually learned that early in my career, my very first startup, we raised, I think at the time was a lot of money. It was $27 million. We ran out of money

in like 18 months. And then, you know, the 2000s came around and there wasn't any more funding and owned our business. We assumed that we could just go raise more money. And so I think that was a good early lesson that you can't always do that. If I look just at, you know, my own portfolio or our friends companies, what is interesting is if, like, of course, you have a few examples that everybody is looking at, like Open AI, where value creation and dominance or at least like lead in a market is highly correlated with the amount of money they're spending. It's not true across the portfolio. And I'm like only investing in really AI companies. In that, yes, everybody absolutely needs computers and talking about companies that are doing their own training or fine-tuning.

But some of our companies that are making the most progress, and like, you know, we're still talking very early days, you know, zero to tens of millions in their first year or two. I think one of the other open questions that people have wondered about is, does all of the value creation in the ecosystem go to? your compute vendor and eventually a big piece of it over to Jensen and Nvidia or to the model vendor. And I think the answer at least to like right now is clearly not, right? I think there's different, there's capture different levels. There's probably enough for everybody. Today most of it does go to Nvidia, I think. That's a lot of it. But I just think that's because it's where it is early in the cycle. You know, I think, and they've built some incredible

technology that's enabling some really cool stuff. So I think that that's, it's, it's, it's fine. And at some point, it's going to be the companies that find out how you actually go solve real problems and deliver real value to enterprises and to customers and other things like that. And that's going to be that, you know, I see a lot of, if I take a step back and see who's implementing AI out there, it's a lot of enterprises that are doing proof of concepts and sometimes they'll find one that really works well and it'll go to production. And I think if you can have a startup that can make that part easier, that's says, look, this is a real value, right?

It's not a chat bot on your website, but it's something that helps you go faster, make sales better, innovate more rapidly, you know, do something you were never able to do before, improve manufacturing efficiency, whatever that is the startup is focused on, or the company is focused on, for that matter, it's going to be an application level, right? Most people don't build a CRM from scratch. They go use a Salesforce or something like that. And most companies don't build software from scratch. And so I think most companies are not going to build their own models over time.

They might tweak them a little bit, but I think a lot of companies are going to build, are going to use the applications that use the software and the models underneath. And so it's not surprising to me that you don't necessarily have to spend the billions of dollars to go build your own model. You can build a small model. It seems like there's a lot of precedent for this just in terms of prior waves of both software and internet, where I think code two or somebody had some very good slides where they broke down their relative value accrued by each layer. And to your point, each layer sort of ends up benefiting over time. So it seems like that question is almost overstated in terms of the importance of it.

Do you have a prediction if you just look at what happened with public cloud if what happens with AI platforms evolves differently? And the reason being, you know, I just spent a little bit of time talking to large enterprise customers again. And exactly as you said, there's a lot of activity coming from different POCs, there's investment in the area, there's top-down interest. And I do think there's a real conviction that the value will be real. There's also, like, I feel a little bit of deja vu in a bunch of large organizations saying, like, nobody meets our needs, and we're going to have to build our own platform. And it's going to be everything from, like, data management to, you know, GPU management for

training and inference to, like, eVal suite. to, you know, compliance and audit. And so I kind of want to say I've seen this story again before, but, you know, how would you predict it plays out? It's exactly that. I think every time there's a new space and some of those things don't exist, people are like, well, I've got to go build my own. Well, it's like, or, you know, likely there's a bunch of other people that are actually building that, too, like us, that once it exists, and I don't know how many times customers have told me like, oh, if I had already had this, I wouldn't have got to build my own and now I can stop investing in it because it's actually not the thing that gives me value.

And so managing GPUs, I bet very few of those enterprises, that's the actual thing that their stockholders are like, yeah, that's what this enterprise is super important, good at is managing GPUs, like outside of like, you know, hyperscalers or somebody like that, they don't really want to, right? If they could use something like Sagemaker and it has all the capabilities that they can go and build the services that they want, and it doesn't today. We know that, but we're iterating incredibly rapidly and launching new stuff every month, every week really. And so, you know, I feel pretty convicted that that, particularly for that

level of the stack, like it just doesn't make sense for people to do that part. And it's not surprising that they do it today because some of those things don't exist and they want to go deliver on something. And, you know, their priority was slightly different than when we go deliver it or something like that. But having it integrated as part of your platform, having an integrated as part of where you have your data lake, having it all of those things tied together kind of as an infrastructure piece likely make sense. And it doesn't mean that we'll do all of those things, by the way. Some of them will be partners that are built on top of us, which is great too. But I do think that it's the people that are specializing in those places are probably the ones that are going to eventually

kind of support that space, not individual enterprises. If we were to abstract out a level and, you know, ask what your vision is or how are you thinking about the next three to five years of AWS more generally as a business. What are the key things or areas of focus for you? Well, this is one. I think, I mean, I'm just as excited about generative AI and AI broadly as you all are. I do think that it's an enormous opportunity for us and for our customers to. And I think it actually, in many ways, it has a positive flywheel effect and can be a tailwind to some of that first stuff that we were talking about a little while ago about helping customers move to the cloud. You know, I think if we think about where can generative AI help, some of that can be like, how do you make that go faster, right? How do you take some of that more? You know, our original AWS thesis was we'd take care of the muck so you don't have to. There's still a lot of that that customers have to do today that I think generative AI can help with. And so over the next three to five years, there's a big investment for us in both building that tool set, building that whole platform that we're talking about so that customers don't feel like they have to go manage a bunch of these pieces.

They don't have to think about, you know, GPUs or they don't have to think about how do they think about kind of tying these clusters together or whatever. of that can be abstracted away. If you think about the start of what bedrock is, if you go use bedrock models today, you never interact with the GPU, right? You just, you send it tokens, you get tokens back. And you can, eventually you're going to be doing things like fine-tuning and pre-training where you're sending an information and training the models under the covers, but you're still just then sending it tokens and getting tokens back. And so the more we can abstract that to whether it's serverless or an application platform that people can go build. And that's just, you know, again, we're at the early stages of what that is. And so I think that

as you move forward, generative AI honestly becomes one of the compute building blocks that you think about. You're going to need storage. You need compute. You need databases. You need inference, if you will, for your application largely. And I think that's just going to be, and networking and a bunch of the other things. And inference is just going to be one of those building blocks that people come to expect. And just like with compute where you may want an Intel processor or a Graviton processor or you may want block storage or you may want object storage. You may want different models behind the inference and you may think of that and it'll have slightly different databases. Inference is going to have different flavors to it and it'll be big models and small

models and you'll trade off cost and latency and capabilities and things like that. But I do think it's part of the applications. And so we're trying to build it as part of that platform. When you're just building your application, it comes with it. And then there's a lot to get there between now and then, but I imagine that's how most applications are going to be built going forward. So we started this discussion talking about like the early days of AWS and sort of how you were discovering the sort of, you know, unthinkably large requirements from large financials. And yet, you know, $100 billion of revenue later, you are still a huge partner to startups. Like that goes against some of the conventional wisdom of like choose one audience and just,

you know, slowly move up market as something that, you know, many startups themselves choose to do. Like, why continue to work so much with startups? And I know you personally still think about this a lot. It's super important for us. And startups are the lifeblood of what helps us grow. And we get so much benefit from the learning from startups. And so they will continue to be incredibly important for us. And we're going to, if anything, lean more into startups and supporting startups as part of what we do.

Thanks so much for providing your perspective over the last, you know, 20 years or so of AWS has been really fascinating talking to you. So thanks so much for that. the time today. Thank both of you for having me. Appreciate it. It's been fun. Find us on Twitter at No PryorsPod. Subscribe to our YouTube channel. If you want to see our faces, follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no dash priors.com.