Big Technology Podcast - Microsoft's Head of Cloud & AI on the AI Buildout's Risks and ROI — With Scott Guthrie

Episode Date: October 1, 2025

Scott Guthrie is the executive vice president of Cloud and AI at Microsoft. Guthrie joins Big Technology Podcast to discuss the tech industry's massive AI infrastructure buildout and whether it is overdoing it with the hundreds of billions of investment. Guthrie discusses the way Microsoft thinks about its OpenAI investment, whether it's worth investing in scaling pre-training, and Silicon Valley's growing debt problem Tune in for the second half where we discuss the longevity of the GPU, custom silicon, and competing with NVIDIA. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. Want a discount for Big Technology on Substack + Discord? Here’s 25% off for the first year: https://www.bigtechnology.com/subscribe?coupon=0843016b Questions? Feedback? Write to: [e-mail]

Microsoft's head of cloud and AI joins us as we ask. Is this AI buildout going too far? That's coming up right after this. Welcome to Big Technology Podcast, a show for cool-headed and nuanced conversation of the tech world and beyond. We're joined today by Scott Guthrie. He's the head of cloud and AI at Microsoft, and he is the perfect guest to give us some context on the massive and some would say insane buildout of AI data centers taking place today. What does it mean? Is it going too far?

what will lead to. Scott, I'm so thrilled to have you on the show. Welcome. It's good to be here, Alex. Thanks for having me. All right, let me take you through the headlines over the past couple weeks. It's crazy that this has just been over the past few weeks, but here we go. NVIDIA agreed or announced that it would invest up to $100 billion in Open AI, starting out with $10 billion. Oracle announced it would invest $30 billion in Open AI or a $30 billion build out with the company, Anthropic raised $13 billion. You know, we're just talking about a cool 143 billion, no big deal.

Is this crazy? Is this overinvestment? Well, I think there's a great question. I'm sure that's top of mind for everyone. I think, you know, stepping back for a moment, I would say, if you look at AI and the impact, I think it's going to have in the economy, it's going to be, I think, the most profound technology shift in our lifetimes. And so I think if you look at the long.

term trend, I don't see, I don't worry about overinvesting. I think there will be a question on the horizon of different companies or making different strategies in terms of their investment and how they get the return in the one, two, three year horizon. So, you know, am I going to say that every company is perfectly timed? I don't, I'm not going to make that assertion. But at the same time, I do think the long term secular trend of AI is going to be that we're going to need more infrastructure. There's going to be more ROI. from it, and it's going to be more widely used. And so, you know, I think directionally from an industry perspective, the investments do

make sense and will ultimately yield pretty profound results. So you think that this level of buildout is healthy? I think we definitely are not nearly at the point at which there is too much AI infrastructure given, I think, the number of AI workloads that are coming for the world. And I think we're seeing over the last couple years, as people use AI, they get value, they use it more, the models get better, and people then use it even more for new use cases. And I think at this point, across the industry with AI, we're still more supply constrained than we are demand constrained. And I think, you know, I expect that to continue over the next couple of years as the technology continues to evolve and as people start to integrate AI into more and more workflows. Okay. So let me put it bluntly then. Microsoft has a partnership with OpenAI, has invested $13 billion thereabouts, has the capacity to build big data centers.

why did Microsoft make the decision not to do the $100 billion level build out with OpenAI or even the $30 billion that the company is doing with Oracle and leave it to other partners to do that? Well, we have a great partnership with Open AI and it's gone back many, many years and continues going forward. And we are building out and doing a lot of projects with Open AI. Across the Microsoft Cloud, we're building out AI data centers all over the world. And at the same time, you know, we are balancing, you know, our investment to make sure that it maximizes the AI infrastructure for both our first party Microsoft offerings that we're investing in our customers, AI offerings that we're investing in. and obviously opening eyes offerings that we're deeply enabling. So, you know, we are very invested.

I don't think it's a binary. Are we building out for opening or not? We definitely are building out for OpenEI. And at the same time, the way our partnership works is we're supportive of others participating in that as well. Okay, but I just want to put a fine point on it because, again, like if you believe that this technology is going to be massive. transformative, which you stated, and that we're not at the, at the sort of optimal point of the AI buildout yet, that there's room to continue to do more. Again, it's the partnership with what is the consensus leader in the space.

They needed more infrastructure. There must have been some calculation within your group or your company to say, is it worth it for us to be the one that goes out and builds this massive, massive footprint, you know, in partnership with them or somebody else. So I definitely understand there's multiple stakeholders, but what made Microsoft pause on that front? Well, we have a balanced view. And so it's, and we take a long-term view in terms of making sure that we're building out in all the locations that we want to build out that were being thoughtful in terms of kind of the investment spend and the infrastructure that we're building.

and also recognize that we don't have to do at all. And so I think we're always trying to kind of take a continually balanced view of that. And as you've seen from our GAPX and as you've seen from our earnings calls, we are investing a lot in infrastructure and building out like crazy. But again, at the same time, you know, we're always constantly reevaluating and watching closely, you know, which data centers in which markets to what specifications and making sure that we keep, you know, good discipline, as we're doing it, that optimizes for both the long-term, near-term, and mid-term horizons.

Okay. I'll just ask one more follow-up, and then we can move on. Good discipline. What about this would have been undisciplined to have gone to this level? Well, I don't think it's so much the volume level. I think it's one of the things that we do when we add new data center capacity or AI, infrastructure is, you know, making sure that, that we can use this infrastructure for a variety of different AI use cases. I think one of the things that's really going to differentiate

AI infrastructure companies in the future is that ability to kind of maximize yield on the infrastructure. Like, how are you driving down the cost of, you know, tokens per watt per dollar? And, you know, part of what makes the Microsoft portfolio so unique is the fact that, you know, that we have a lot of our own AI products, Microsoft 365 copilot, GitHub copilot, the work that we're doing with Nuance and Dragon and healthcare. We've got the world's largest consumer application with ChatGPT that runs on top of Azure,

and we have thousands, hundreds of thousands, and millions of businesses that are also building their own AI applications on top of us. And so as we think about, like, what market are we going to build a new data center? Is it for training? Is it for inferencing? And, you know, how do we make sure that that infrastructure is going to be

maximally used? You know, we feed in kind of each of these different customer scenarios into our calculus. And there are certain tranches of capacity that we're happy to build out because we can see very clear line of sight in terms of how we're going to maximize the usage and the revenue from it. And there's others that were maybe less. likely to see the immediate or the ROI that we'd like. And so we try to be disciplined about it.

As we've kind of shared in our blog post, you know, we do kind of look at every request first and we do have an opportunity on that. And as you've seen from our CAPEX, we are swinging in a lot of opportunities. But that doesn't, you know, we're not going to be undisciplined and say, blanket, we're going to do everything. You know, we know that, you know, some opportunities will have. more certain returns than others. And we're trying to make sure that we maximize our focus around those.

You know, as we're talking, I'm kind of laughing at myself because, you know, my question is basically boiling down to you've, like, talked about your CAPEX. Isn't Microsoft expected to spend like $80 billion on infrastructure this year in the neighborhood? I think that's what we shared in our last earn. And I'm like, well, why aren't you doing another $100 billion? in but but the fact that you're not as actually very interesting and and it goes to a point that you that you just made and I'm trying to read behind the line between the lines and you tell me if I'm if I'm getting this right you think about you're talking about where you invest and where

you're pretty sure you're going to get an ROI and to me I'm sitting in your shoes the question I would be asking is is it worth spending you know all that money on training where there's been a lot of noise about diminishing returns of training larger models with these unbelievably massive data centers. Now you have the startups like OpenAI and Anthropic, their belief in the scaling law seems unabated, and so the numbers get bigger and bigger, and they seem to believe that they'll continue to get an exponential return from training these bigger models. but is your decision in terms of being disciplined based on a belief that you're not,

you're not sure if scaling up will continue to work and therefore it's too big of a risk to make such a large bet on training an even bigger model in a bigger data center? Well, I think there's a couple different elements of that. I think one is recognizing that you want to have the best models. So, you know, training is super important because if you don't have the best models, then your actual ability to monetize AI goes down. You know, at, you know, part of what makes our partnership with OpenEI unique is the fact that we do have access to the best models,

frankly, whether they're trained on our infrastructure or anywhere else. You know, that's part of our partnership that's really important. And I also think when you think about training, training is evolving for maybe where simplistically we think of training a couple years ago of you do training in one place and then you do inferencing where you are executing the models and building applications.

You know, there's now multiple types of training. There's pre-training, there's post-training, there's reinforcement learning, there's fine-tuning. There's a lot of new techniques that both sometimes require lots of contiguous infrastructure and sometimes requires lots of infrastructure, but sometimes it's smaller sizes that can be used for very specific tasks. And so when we think about the investments of our infrastructure,

we're trying to think about all of this and compose it all end-to-end. You know, for us, that means, for example, we want to make sure that we have lots of inferencing capacity because that ultimately is how customers pay us and how ultimately you make money from any AI product that you build. And I think increasingly on the inferencing side, you know, one important element is the geopolitics of the world have gotten complicated over the last many years. And, you know, customers in Europe want to make sure that their AI is in Europe. And the customers in Asia are going to care about their AI in Asia.

Obviously, the customers in North America and the United States are going to care about their AI being delivered in North America. And so even as we build out our infrastructure, we want to think about it not just narrowly as we want to have one giant pool all in the U.S., we need to kind of be distributed around the world to kind of meet those geopolitical needs and to make sure that our AI is as close to the customers that are going to be using the AI as possible and can meet all of the data residency and data sovereignty needs. And so even if you look at our infrastructure builds around the world, we have regions in more countries, in more locations than any other.

structure provider. And again, as we balance out the investments we're making on AI info, you know, we're trying to keep that in mind versus narrowly put it all in one location. I totally understand that, but I have to go back to the diminishing returns of training question. Where do you stand on that? Well, I think if you look at training broadly, I think you're going to continue to see more value from the models by doing more training.

By kind of going back to my answer earlier, I don't know if that's always going to be pre-training. I think increasingly lots of post-training activities are going to significantly change the value of the model. And so by post-training, I mean take the base model and how do you add financial data or health care data or something that's very specific to an application or a use case. What's nice about post-training is that you don't have to do it in one large data center in one location. And so part of the technique that we've been focused on is how do we take this inferencing capacity around the world? And a lot of it is idle at night as people go to sleep. You know, how are we doing increasingly post-training in a distributed fashion across many, many different sites?

And then when employees come to work in the morning, we serve the applications. And so having that kind of flexibility and being able to dynamically schedule your AI infrastructure so that you're maximized. revenue generation and training, ideally in a very swappable dynamic way, I think is one of the things we're investing in heavily, and I think is one of the differentiators for Microsoft. Okay, but you'll forgive me for going back to this scaling pre-training question. I'm just trying to see what you believe here. And you haven't said it outright, but from your answers, it does seem to me like you believe that spending wildly on scaling pre-training is a bad bed?

I wouldn't necessarily say that. I think we've definitely seen as the scale infrastructure for pre-training has gotten bigger. We are seeing the models continually improve. And we're investing in those types of pre-training sites and infrastructure. We recently, for example, announced our Fairwater data, you know, center regions around the U.S. We have multiple Fairwaters. And, you know, we did a blog post recently of one of our new sites in Wisconsin. And these are, you know, hundreds of megawatts, hundreds of thousands of the latest G.B. 200s and G.B. 300 GPUs. And our, you know, we think

the largest contiguous block of GPUs anywhere in the world in one giant training infrastructure that can be used for pre-training. And so we're investing heavily in that, as you can see kind of from the photos from the sky in terms of massive infrastructure. And, you know, we do continue to see the scaling laws improve. Now, will the scaling laws improve linearly? Will they improve at the rate that they have? I think that is a question that everyone right now in the AI space is still trying to calculate.

But do I think they'll improve? Yes. And the question is really around what's the rate of improvement on pre-training. And I do think with post-training, we're going to continue to see dramatic improvements. And that's, again, why we're trying to make sure we have a balanced investment, both on pre-training and post-training infrastructure. And, yeah, and just to parse your words here, you can see improvement by making, by doubling the data center. But that's why I used the word bet, because are you going to get the same return if it doesn't improve exponentially and just improve? on the margins and that I think is the big question right now right it's a big question and and

you know the thing that also makes it the big question is it it's not like a law of nature that's immovable and so there could be one breakthrough that actually changes the scaling laws for better and and there could be a lack of breakthroughs that means again things will still improve but do they improve at the same rate that they historically did from a raw size and scale perspective. And that is the trillion-dollar questions. Okay, great. I do want to get to the ROI of AI spend in a moment.

You know, it's always great to have a chance to speak with someone who's in a position like you are within Microsoft because we get a chance to like take some headlines and which might paint a portion of a story and then ask you what the truth is. there were some stories over the past year talking about Microsoft had like canceled options to build data centers in certain locations and people took those headlines and they read into it that there was no demand for AI or that it wasn't going as well as Microsoft's telling us but what is the what is the reason for why those those data centers that there were the options and they were canceled what happened there well we're

constantly, I think in general, the headlines were focused on things that we canceled as opposed to all the things we signed. And so if you look at a given- It's amazing how news works that way, right? If it bleat it leads, so. If you look at kind of the overall investments, and certainly if you look at the overall cap-ex spend, it has been going up and up and up. And so as has, again, the revenue that comes from it. And so I think I would kind of focus on the overall picture. as opposed to individual tranches or individual projects that we potentially made decisions on. Now, you know, the thing that we did do and we continually do is look hard at every single investment

decision we make. We don't take this level of investment and this level of project and infrastructure lightly. It's critical that we invest wisely. It's critical if we invest that we make it successful and that we bring it to market on time with the right quality and the right security. And it's critical that we have the right go-to-market to monetize it. And so, you know, part of our calculus that we do as a leadership team is constantly looking at the variables for all of those.

And there are places and times when we slow down or pause projects, and there are times when we accelerate projects somewhere else. And kind of going back to my comment around the world, you know, also the, you know, regulation, geopolitics, of how AI is going to be used going forward has changed quite a bit. And what Europe thinks about where GPUs can be based has evolved quite a bit, I'd say, in the last 12 to 18 months. And I think it's going to continue to evolve around the world.

And so even as we think about the investments we're making, we're also being very, very thoughtful in terms of where geography-based are we investing so that we can, again, maximize the AI tokens we can serve in real production applications, and then ultimately use that maximization to ensure that we're delivering a good return on investment for every capital dollar we spend. Okay, and I have some technology questions for you, but just to keep on speaking about the financing of this stuff because it's so important. So there has been some interesting reporting about how the,

AI infrastructure build out has begun to be funded by debt, not just profits. Great story in the Wall Street Journal this week. It says debt is fueling the next wave of the AI boom. I'll read the beginning. In the initial years of the AI boom companies, comparisons to the dot-com bubble didn't make sense. Three years in growing level of debt are making them ring truer. Early on, wealthy tech companies were opening their wallets to out joust each other.

For leadership in AI, they were spending cash generated, largely from advertising. and cloud computing businesses. There was no debt-fueled splurge on computing and networking infrastructure like the one that inflated the bubble two and a half decades ago. However, that is starting to happen. Now, Open AI's deal with Oracle

has been pushed Oracle to start taking on debt. They say this is according to the story. Analysts at KeyBank Capital Markets estimated in a recent note that Oracle would have to borrow $25 billion a year over the next four years. Obviously, you guys are not Oracle, but you know, you're watching this happen as it plays out and see the parallels to the dot-com boom. I'm sure it's not fun. You've been at Microsoft for I think 27 years, 28 years, 28.

Sorry, I don't want to miss that last year there. So you've seen it, Scott. This seems to be an issue, at least from the outside. what do you think about it being on the inside? Well, I think obviously there's a tremendous amount of spend from lots of different companies. And I would say, yeah, the thing I can speak most to is what we're doing. And kind of, you know, per my comments earlier, I think we're trying to make sure that we have a smart investment play. and a long-term strategic play that allows us to ride the AI revolution

that we think is going to transform the world and do it in a way that leverages some of the strengths that we have at Microsoft, which is we have very good cash flow, we have a very diverse portfolio of businesses, in particular in the commercial enterprise space, whether it's cloud infrastructure, productivity applications, business applications, security, et cetera. All of them are going to be transformed by AI.

And, you know, if you look at, say, to your comment earlier on the Wall Street Journal post, I think if you even read further in the post, you know, it does show the ratios for different companies. And there are some companies that are 400 percent debt-to-equity ratios. And there are other companies that are much smaller, and that would be Microsoft. And I think, you know, we want to make sure that we're not, and I, I think, again, based on our CAPEX spend and the rate at which our CAPEX spend is going up, you know, we're not going to sit on the sidelines and not be bold as we invest. And at the same time, you know, I think the thing that our investors expect, and ultimately

I think every investor of every company will expect, is to see that revenue growing in terms of AI services and products that are being delivered in terms of net revenue recognized in a quarter and making sure that the proportionality of that to the spend and in particular to the obligations that maybe are being undertaken with debt are balanced. And that's the thing that we've been focused on. I think, you know, if you look at our last quarterly earnings, I think people were pretty pleased with getting the balance right there. And, you know, every quarter going forward, people are obviously going to be looking at

making sure that that balance is right so that they see us investing for the long term and going to win. And at the same time, doing it in a way that is sustainable and allows us to kind of ride through, you know, the ups and downs that inevitably will happen over the next many years as this technology, you know, transforms the world. What are the consequences if this goes wrong with the debt? You're obviously not taking on the same amount of debt. So there's a rationale behind it. What happens if it breaks? Well, I mean, we have the ability. We're not constrained.

I mean, our borrowing costs ironically right now are low. Yeah, but industry-wide, big picture industry-wide, not Microsoft specifically. Well, I think the thing that we, as an industry, I think, you know, again, you need to have that thesis of how you're going to use the infrastructure. And is it, do you have, I would focus less on the megawatts that sometimes get reported in the press. And more at where are those megawatts? And what are you gonna do with those megawatts? Is it gonna be ultimately capacity

that you can use to serve customers? Is it to build better models that help you serve customers? And you know, what is the line of sight in terms of the product services and revenue that comes from it? And I think that's a place where, again, Between Chatchibouti, which is the number one AI app in the world, between Microsoft 365, which is the number one enterprise AI app in the world,

and between GitHub, which is the number one developer AI app in the world, I feel good that we have applications using our infrastructure and maximizing it, and I feel good about the investments we're making in terms of capital spend and build out in the right locations to kind of continue to do that. And I think not every company probably has that level of, game plan and I don't think that maybe not every company is probably doing the same level of thoughtfulness of that and you know at some point you know different companies will probably be hit by it but you know we're very focused on what we do and how do we make sure that we stay

aggressive yet disciplined and make sure that we get that balance right all right i want to take a quick break and then i'm going to ask you a couple uh technology questions about the state of the buildout GPUs, custom silicon, and then maybe we can get a little bit into this ROI question. In fact, we will. We have to talk about the ROI of AI. We'll do that right after this. And we're back here on Big Technology podcast with Scott Guthrie, the head of cloud and AI at Microsoft. Scott, we have a Discord here at Big Technology, and I asked some of our members, you know,

what they would ask you. And we got a flood of excellent questions. And I think they were great because they focused on the technology. Some questions that I don't think you hear too often in the common conversation about this technology. So you're the perfect person to ask. I'm going to ask them to you. One of our members asked, what is the working life of a GPU and how long until they burn out? Are there use cases for GPUs once they are no longer top of the market?

We hear often about, well, unlike the laying of the fiber, the GPU depreciates after a couple of years. So I think this is a pretty important question. Can you handle that for us? Yeah, I think kind of going back to the comments that we had earlier on balance, I think as you think about your GPU build-out, one of the things that we think about

is the lifetime of the GPU and how we use it. I think, you know, what you use it for in your one or two might be very different than how you use it in your three, four, and five or six. And so, you know, I think that is something where, you know, so far we've always been able to use our GPUs, even ones that we deployed multiple years ago for different use cases and get positive ROI from it. And that's why our depreciation cycle for GPUs is what it is. But I do think that's, you know, as we build our infrastructure, we are definitely consciously thinking about that because you don't want to have your entire fleet in two years. suddenly have to be replaced because, you know, that that would be expensive. And so, you know,

we are very thoughtful on that. And again, I think I talked earlier about different training. I also think even as you think about training, we often, in the past, used to monolithically call training training training, there's lots of lots of different training use cases now. There's pre-training, there's synthetic data generation that goes into training. There's post-training with RL and fine-tuning. and other different techniques. And, you know, having infrastructure that's very fungible and that you can use for a variety of different training scenarios and at the same time be used for inferencing

where you ultimately enable an application to perform a query or perform an AI invocation is key. And I think that goes beyond just the GPS, Even though people often narrowly focus on that. It also is around the data center Architecture. It's around the storage and the compute that's near the GPUs and it also really comes to play with a network Because if you are for example building one large data center that only does training and it's not connected to a wide area network around the world that's close to the users It's hard to use that same infrastructure for inferencing because you can't go faster than the speed of light.

And so someone elsewhere around the world that wants to call that GPU, if you don't have the network to support it, you can't use it for those inferencing needs. And so again, going back to some of my comments earlier about how we're trying to be very thoughtful about where we place infrastructure and how we maximize the utilization,

we're definitely thinking of that, not just for this year or this quarter, but thinking about it on that four or five, or six-year horizon for how we want to basically leverage and use it. Okay. Here's another question. Are there any cool technological breakthroughs that would change the economies of data centers as we know them now?

GPU started as graphics processing units for video games. Are the resources you found that might do as well, but with fewer constraints? Well, I think one of the biggest changes that's happening right now from a data center perspective, and you're seeing this with the latest Nvidia GPUs, and I think you're going to see this in a more profound way over the next two years, is the shift from air-cooled data centers where you use, you know, effectively giant air-conditioning units or chillers, to a

liquid-liquid-cooled facility water, where you're actually pumping in water in order to cool the equipment in a closed loop circulating system. So in other words, you feed in cold water, you run it over the GPUs effectively, extract the water, cool it down again and then do it again throughout the building. That's a massive technology change. And it does mean that older data centers that are air cooled, you know, they can't just drop in liquid cooling and be effective. And so that is something that I think everyone that's in the AI space is designing for. It needs to be thoughtful of, again, with their infrastructure projects to make sure that they're ready for that technology shift.

It also is going to have a big difference and big impact in terms of the staffing. When you have an air-cooled data center, you'd have very few employees often per server. When you all start to involve water and liquid, you know, it's not massively more, but at the same time, it does change staffing because there are more things that break when you have pipes that are actually continuously flowing liquid into. a data center. So there's a lot of technology shifts that are happening right now, behind the scenes, beyond the GPUs. And then obviously, GPUs are the things that dominate the press in terms of innovations, both in terms of the silicon, but also in terms of the network. Because

at the end of the day, if you have a chip that can process a lot more information, but you don't have the ability to get that information to the chip and extract it or have it communicate with other chips, you know, then you don't get the yield out of it. And so I think it's fascinating right now in technology, the pace at which so many things are evolving so fast, both with the GPUs to the question, but then also the data centers, even the power and cooling infrastructure for the data centers and the network. And, you know, as a technologist, it's exciting times. Right. You mentioned staffing, so I want to ask a follow up on that front. I think for those who don't live, those who are not in this, deep into it, there is a perception that

data centers, they're placed near communities in some cases. They use up a lot of water. They don't provide a lot of jobs. Is that a misconception? I think it's a misconception. You give us some numbers to sort of flesh out what they actually bring to a community that they appear next to? Yeah. I mean, we've, we've talked. talked about with our Wisconsin Fairwater site that we did some press on recently and talked about, including with the governor of Wisconsin and others that were attending. You know, it's thousands of jobs that we've created on the construction of the site. I think we've shared over 3,000 jobs. And these are very skilled jobs. These are, you're talking about electricians. You're

talking about plumbers, you're talking about welders, you're talking about skilled tradescraft and, you know, high quality jobs. And I think, you know, if you look, we have a phenomenal work site, phenomenal workers there, and a phenomenal safety culture, which has allowed us to attract some of the best workers to work on that project. I think what people are missing sometimes when they say, okay, but when the project's done, how many people are going to be in the data center. And there will be, you know, hundreds of people that will be in the data center. What people are missing is the fact that right next to that data center, we're building another data center. And so those thousands of people that have been working on the first Fairwater

Data Center, we just announced are now going to be starting work on the second one. And then after the second one, we'll do a third one. And if you look at the land and you look at the power, we've accumulated in that area, it's multi-gigawatts of land, or multi-gawatts of power. And it's It's an awful lot of land. And so you're going to see us continue to employ thousands of very skilled tradescraft workers in that community. And as each one of those data centers comes online, we're going to add net new employees that will actually operate it and manage it.

So, you know, that would be an example, I think, of a community. And we, you know, we have over 400 data centers around the world. So they're not all that size. obviously, but replicating that and I think as more infrastructure gets built out, you're going to continue to see not just jobs created, but well-paying jobs that really require real tradecraft. Do you feel what, do you feel the, I don't know the right word to put it, the pressure of competing with China? Because from my understanding, China has a much looser regulation. approval process and they're just stacking, you know, data centers.

They have abundant electricity in the United States, in particular. I imagine Europe is the same way. That is not the case. So what's it like? Certainly, I think the world has a very different regulatory approval process. I mean, I think one thing that when I talk to people and they say, how can you build data centers faster, you know, there's obviously things that we can do from a technology and are doing from a technology and from a manufacturing perspective.

But, you know, candidly, you're in the U.S., the longest part of building a data center is getting permitting. It's not actually the construction. It's making sure that you, you know, get permitting approval for all the steps that you want to take. And, you know, different states and different parts of the country have different regulatory environments. And I think even if you look at a sort of a heat map,

if you will, of where data centers are being built in the U.S., You definitely see pockets, and I would say some of that approximates to where there is land and where there's power. And some of it really, you know, closely correlates with where it is easier or faster to kind of complete the permitting process. You know, in Wisconsin, we had a, you know, a phenomenal partnership with the governor and the local county. We were able to purchase some land and power that a manufacturer was previously going to use, and they pulled out of a project. And so, you know, I think that the local communities recognized if they weren't able to work with us, you know, they were going to lose jobs and have, you know, impact on the community. And they leaned in with us and, you know, can't say enough positives in terms of the speed with which, you know, we went through all the process. We got all the approvals.

It was a very thorough process. But it was streamlined so that we could move fast and that we could actually help ensure the jobs weren't lost and that instead they were created in the community. And I think there's more opportunities for public-private partnership like that that we'd certainly welcome as part of it. Okay. Another Discord question. What timeframes are you looking at to get ROI on these investments? So when would you stop investing if prior investments weren't showing returns? And how do you know when it's time to stop building? Well, I think, you know, at the end of the day, I think, you know, every quarter we share our revenue growth and we share our cap-ex spend. And, you know, to some extent, I think the, you know, markets keep companies honest in terms of that balance. And, you know, sometimes markets can be slightly irrational at times, but in the limit, the markets keep you honest.

And, you know, that's a big part of why we focus so much on making sure we get that balance right. Make sure we're, again, investing for the long run. And I don't think anyone, if you look at our CAPEX spend and our commitments and our investments would say that we are not being bold. But at the same time, you know, we have a report card every quarter where we need to kind of demonstrate and prove not just with press releases, but, you know, here's how much revenue growth we had. You know, last quarter we grew Azure 39% year over year on a very large number. And a lot of that was driven by AI and then also driven by the other systems that come with AI because there are databases and there's compute and the storage sold with that AI. And, you know, I think investors were happy with both the spend and the aggressiveness that we

were building out, but also the return. And, you know, I think that's going to be true, you know, forever. And, you know, making sure you get that balance right. And again, as part of that balance, markets want to know you're investing to win the long run. And at the same time, they want to make sure you have some level of discipline. And I think our portfolio, the balance that we have both across the products we build, but then also the fact that we have the largest AI product in the world called JetGBT, running on top of our cloud, gives us a unique opportunity to get that balance and that growth and that investment right.

Is Chat Chepti, by the way, going to stay? on Azure, even though Open AI is making these partnerships with Nvidia and Oracle? Yes. Okay, all right, it's good to get something definitive on that. You mentioned your 39% Azure growth, and I'm looking at your quarterly numbers every quarter and often talking about them on CNBC

and the numbers are massive. And the other side of it though is, so that's spend coming from clients, right? And there have been multiple studies that have come out recently that I've talked about how enterprises aren't getting the ROI that they've anticipated on their AI projects yet. When you see those studies, do they ring true to you? How do you react to them? Well, I think when you say AI in general, it's a very broad statement. This is generally, this is obviously, I mean, not obviously. It's in large

part, this is generative AI where companies everywhere have. try to adopt LLMs and try to put some version of that into play in there. And it's not recommender engines, basically. Yeah, but I think what you need to do is double click even further from Gen. I to get up co-pilot or healthcare or Microsoft 365 copilot or security products built with Geni. I do think ultimately, you know, the closer you can kind of double click on is this really delivering ROI, then you have much more precise data.

Because I do think a lot of companies have dabbled or done internal kind of, I'll call it proof of concepts. And some of them have paid off and some of them haven't. But, you know, but I think ultimately a lot of the solutions that are paying off that we continually hear from our clients and our customers is, you know, a bunch of the applications, for example, that we've built. I think similarly, you know, a bunch of the applications that our partners have built on top of it. us. And, you know, ultimately the Azure business is, you know, we get paid based on consumption. It's a consumption-based business, meaning if people aren't actually running something, we don't get paid. It's not like they're pre-buying a ton of stuff. You know, we recognize our revenue based on when it's used. And so, you know, the good news is when you look at our revenue growth,

it is, you know, it's not a bookings number. It's actually a consumption number. And you can tell that People are consuming more and, you know, the last two quarters are revenue growth accelerated on a big number. And that is a statement of the fact that I think people are getting a lot of ROI, at least with the projects that they're running on top of our cloud. Yeah, I think that's an important point to bring home. It is consumption-based. So you talked a little bit about water cooling versus air cooling. I love the term for the air cooling. It's called chillers.

And that's what my friends in high school called ourselves, you know, back in the day. And I want to end on the GPU side of things or the silicon side of things. What do you think the potential is for custom silicon in the AI world? I mean, like we talked about previously, GPUs were designed for gaming. It happened to do parallel processing. Actually ended up being really good for, you know, large language models, the training and the inference. What's your perspective on whether this industry is going to continue to run, on that type of chip and what the potential is for custom silicon.

I think a couple things. I think one is I think increasing the number of tokens you can get per watt per dollar is going to be the game over the next couple years and maximizing the ability of our cloud to deliver the best volume of tokens for every watt of power, for every dollar that's spent, where the dollar is spent on energy, it's spent on the GPUs, it's spent on the data center infrastructure, it's spent on the network, and it's spent on everything else, is the thing that we're laser focused on. And it is, you know, there's a bunch of steps as part of that, GPUs being a critical

component of it. And, you know, one of the things that our scale gives us the ability to do is, to invest for kind of non-linear improvements in that type of productivity and that type of yield. If you've got a million dollars of revenue on a couple hundred GPUs, you're not going to be investing in custom silicon. When you're at our scale, you will be. And you're not just investing in custom skillikin for GPUs for pre-training or for inferencing. You're looking at what can we be doing for synthetic data generation with silicon? What can we be doing? What can we be doing from a network compression perspective with Custom Silicon? What can we be doing from a security perspective? And we have bets across all of those, many of which are now in production and are

actually powering a lot of these AI experiences. In fact, I think every GPU server that we're running in the fleet right now is using Custom Silicon at the networking compression storage layer that we've built. Now, the GPUs themselves are also going to be a prize that people are going to try to optimize, like the actual instructions for doing the GPUs. Invidio is a fantastic partner of ours. We're probably one of, if not the biggest customer in the world of theirs. And we partner super deeply with Jensen and his team. You know, at the same time, and partly why they're so successful is they're executing incredibly well. You know, at the same time, if you look at the history of silicon, not every single company, or it's rare that have a silicon company that every single

year is doing the absolute perfect work that's differentiated. And kudos to Jensen for what he's done, and I know he's going to keep trying to do it going forward. But, you know, there will be other opportunities from other companies where people are going to look for a niche that's going to be big enough in this AI space to be truly differentiated versus what NVIDIA is delivering. and then we're doing our own silicon investment in-house, because we're going to be going after those same opportunities.

And ultimately, the way we've tried to build our infrastructure, none of our customers know when they're using Microsoft 365 or GitHub or any company on what silicon they're running on. And we're going to be constantly tuning the use cases based on the applications. And if we find ways that are breakthroughs, we're absolutely going to be taking advantage of them for those use cases. And again, at our balance of scale and our balance of use cases, I'm very confident that we're going to find use cases where custom silicon will make a difference.

And I'm also very confident we're going to continue to be a great partner to Nvidia and others in the world that are going to be selling us great solutions. All right, Scott, I want to end on this because I've always been curious about the human aspect of this. Like, you're going out and working on designing your own chips that are trying to be better than GPUs for certain parts of this. application layer and training. And then you said one of Nvidia's biggest customers, if not its biggest customer. So is this like a situation where like you go to Jensen and you're like, we're going to just both give a shot at building this stuff and made the best chip win

and it's friendly, like friendly competition or is there any awkwardness in there? Because you're like kind of building the thing that is making them the most valuable company in the world. Well, I think probably different companies handle that differently. I think the nice thing about Microsoft is A, we've been around a while, and I think also we're, you know, we compete almost in every market in some way, shape, or form. So, like, there's none of my partners that I'm not also a competitor with, I think, is probably a true thing. It's crazy.

And the important thing is, I think you have that enterprise maturity to be able to recognize, you know, I want Jensen to do the best possible work because it's going to benefit me. And we've leaned in. We were the very first cloud to deliver live GB200s, which is a massive architectural shift for NVIDIA. That's the first of their liquid. The great spackwell. Yeah.

And we were the first one running, first rack running, the first cluster running, the first data center running of any cloud or neocloud provider in the world. And so, you know, that's an example where we really leaned in and moved at the speed of light together. And we're going to continue doing those types of projects. And at the same time,

know, he recognizes and understands we're going to be doing lots of things. And I also recognize he's going to work with other providers as well. So I think the ability to kind of keep a complete thought and recognize it's not zero sum on every single decision. And that at the end of the day, you know, it's a market. We're all going to compete. And we're also going to partner. And, you know, I think we have the maturity at Microsoft to do that, again, the balance. I think I've said balance multiple times. I do think balance in life, but I'll especially, in business and especially in technology, that is the devil's in the detail. But if you can get that right and do it consistently, those are the companies that win. And those are the companies

that really have the ability to set the agenda. And that's what we're focused on. Well, Scott, I just want to say thank you for taking the time. I know you don't do this often. So I appreciate why did you say, okay, hey, I want to come out and speak about this today? well a bunch of people internally said hey you're going to talk to Alex and so that's always a good advice to follow okay so it's it's fun to get a chance to do and uh really look i really enjoyed the conversation as did I yeah thank you again for taking the time again and I know it's rare for you to come out and speak about these things uh you're running a massive massive and fast growing business and so it was great to be able to speak with you and get into peak I get a

peek into it today and look as to what the rest of the industry is doing and your perspective on that. So thanks for coming on the show, Scott. Appreciate it. Thanks for having me, Alex. All right, everybody, thank you so much for listening and watching. We'll be back on Friday to break down the week's news with Max Zhef of TechCrunch. It's going to be a great episode, and we hope to see you there. Thanks again, and we'll see you next time on Big Technology Podcast.