Odd Lots - Anjney Midha's Plan to Radically Lower the Price of Compute

Episode Date: June 13, 2026

Anjney Midha wrote the first check to Anthropic. He teaches a viral course at Stanford on how AI works. And he was, until recently, a partner at a16z. In other words, he is AI-industry royalty. Midha's new project is AMP PBC, a company that believes it can radically lower the price of compute. To accomplish that, he is working on building a compute grid that turns GPUs into a standardized utility. But right now, compute is too fragmented. It's too heterogeneous. And given the way contracts are structured, he says that labs are being forced to spend money on capacity that often goes unused. In other words, small labs are forced to pay up for big, long-term contracts, even though their own demand (particularly during model training) may be very spiky. On this episode, Midha explains how the market for compute currently works and why he believes there's a software solution that could significantly improve compute utilization. He also tells us why he does not anticipate one company will emerge as the dominate player and that instead we'll have a wide range of models, each optimally used in specific applications. Read more:Amazon Says Its Data Centers Use 2.5 Billion Gallons of WaterOracle Falls Most in Six Months on Mounting Data Center Costs Only http://Bloomberg.com subscribers can get the Odd Lots newsletter in their inbox each week, plus unlimited access to the site and app. Subscribe at  bloomberg.com/subscriptions/oddlots Subscribe to the Odd Lots NewsletterJoin the conversation: discord.gg/oddlotsSee omnystudio.com/listener for privacy information.

The Big Take podcast from Bloomberg News keeps you on top of the biggest stories of the day. My fellow Americans, this is Liberation Day. Stories that move markets. Chair Powell opened the door to this first interest rate cut. Impact politics, change businesses. This is a really stunning development for the AI world and how you think about your bottom line. Listen to the big take from Bloomberg News every weekday afternoon on the IHeart Radio app, Apple Podcasts, wherever you get your podcasts.

Inberg Audio Studios. Podcast. Hello and welcome to another episode of the Odd Thoughts podcast. I'm Tracy Alloway. And I'm Joe Watsonthal. Joe, we like to talk a lot about physical constraints on this show, right? And this is one reason why AI is a really fascinating area for us right now, because there are a lot of physical constraints on what is ultimately the sort of ephemeral technology.

And I think that the tension between those two things is really interesting, right? Like you type a prompt into chat GPT or Claude or whatever, and it's this sort of like disembodied digital platform. You don't necessarily think about the power usage, the real resources, the transformers that have to go into data centers to get compute. The thing that I've been on my mind lately and I've written about it and I plan to write more is this idea that the canonical AI thought experiment is what happens if you tell an AI to make a lot of paper clips and then it destroys the world. because in the pursuit of marshalling all of the world's resources, it just turns everything into paper clips because it doesn't know. I have to ask, is this canonical example? Is this based on your traumatic fear of Clippy from Microsoft Word?

No, but that is, you know, it all comes back full circle. But what we are seeing in real life is that everything from access to electrical grids, GPUs being the big example, energy turbines, talent, and now even including residential real estate are being reproposed to make more and more advanced AI. And in the original paperclip thought experiment, they envision, or at least in one version, the philosopher Nicholas Bostrom, envisions the AI having exhausted all of the world's resources, then sending a probe into outer space to consume star energy to build more paper clips. Just eat the universe. And to this point, we're even talking about going into outer space for data centers to build more AI. So every version of the thought experiment is being replicated, except it's just more and more resources to build the AI by humans rather than paper clips by AI.

There's this other connected theme here. So we've talked before about how one of the reasons valuations seem to be getting insane in the market is because all of this activity is being driven by like this existential need to become number one in frontier models and this new technology. And so if you say you absolutely have to be the first to invent AGI, then you can justify any amount of spending on Earth. Right. And so what we tend to see is like the biggest companies just keep getting bigger and they're the ones that can get resources for all this stuff. And I think one of the most fascinating things right now is that at least as of right now, June 4th, 2026, the frontier models are really close to each other. Right. So the 4.A, GBT, 5.5, like they're not that different. And one of the things I'm curious about is, is there something inherent in market dynamics in this space that will always keep, you know, whether it's being able to distill results from another model on quasi-steel them, whether it's information sharing among employees. Is there some inherent reason why we've seen the stability?

Or could it be that at some point one lab just like breaks out and establishes permanent egg? Still a possibility, but like I am personally on the side. of commodification and everything just becomes kind of basic. Well, is basically available. I know. I'm just kidding. All right.

Thank you, Joe. That is a joke. All right. That's a polite prompt to get to the guest. We do, in fact, have the perfect guest. We're going to be speaking with Angeny Mida. He is, of course, a former general partner at Anderson Horowitz, a Stanford University visiting scientist who teaches the viral AI lecture called Frontier Systems.

Also, one of the first guys to write a check for a student. Anthropic and is now the founder of a new company called AMPPBC. So thank you so much for coming on all thoughts, Ange. Thanks for having me. One correction, it's pronounced AMP PBC, but everything else you got perfect on the intro. Amp would make sense, wouldn't it? Yeah, as an energy. Yeah, yeah.

And just remind us, the PBC is public benefit corporation. That's right. So you're doing this for the public benefit. We're governed by a public benefit charter, which means everything we do has to follow our mission. We have a public charter mission. And we are for profit in the same way Ben & Jerry's or REI and Anthropic are public benefits. So we aim to make a healthy, modest amount of profits that can sustain our mission.

But we are, we have the flexibility to choose what that margin is. Can I just start? I want to establish your credentials, although I feel like that very long list did a pretty good job. But writing the first check for Anthropic, like tell us that kind of origin story. Because the anecdote that you hear is like 25 VCs turned them away initially. and you said yes. It was a little bit of the other way around. I said yes, then we tried to get another 25 VCs to say yes, and I failed.

It was a harrowing experience. It was a bit of a wake-up call. It was late 2020. I had just sold my last business. It was called Ubiquity 6. It was a 3D mapping business. It was an AI business that we had founded in 2017. And I felt like a failure at the time because I was in San Francisco.

I was in San Francisco. I just as big picture my life stories. I was born India. I went to high school in Singapore, and I came out of college to the United States at Stanford from my undergraduate degree. And when I arrived at campus in 2011, deep learning had just started taking over the world in Silicon Valley. Andre Carpathie was a computer science TA to Andrew Ng, who was one of the, I would say,

modern founding fathers of deep learning, this idea that you can teach machines to think without having to give them prescriptive rules. And so I went into sort of machine, I got swept up in that moment and started studying. A lot of my coursework was in machine learning. My primary department at Stanford was in bioinformatics, which was machine learning applied to health care. I got sidetracked to a venture firm called Kleiner Perkins for about four and a half years where I got the chance to work for some of the great investors like John Dorr and Mary Meeker. And then I left and started my own company. And as is the case in Silicon Valley when you start, I was 25, I went and raised about,

47 or so million dollars from some of the usual suspects like benchmark and index and so on. I thought I was the coolest kid in town and I got the beat out of me because we we built this incredible technology, which is this AI system that could map any location in 3D and then the pandemic hit. And so location-based mapping, 3D mapping, you can, the only thing you can control is how you react to what happens. And so I did feel for a moment like it was it was. was bad luck and then you just have to pick up the pieces and make the best of it. So I did with my co-founder. We figured it out. It was a tough few years where we had to pivot the business, but we landed the plane. We essentially, a lot of the distributed systems we'd built on the

backend side ended up being quite valuable. We sold that to a company called Discord, which is a chat up for gamers. Yeah, we have a Discord. Discord. Time to plug them. Awesome. Awesome. Our listeners. So about a month after I sold the business, I got a call from some friends who were running research at Open AI. And we'd all been, you know, friends in the machine learning community in the Bay Area. And they said, Ange, you know, we've trained a little model called GPT3. And we think it's the best since. Just a little model. Yeah, nobody really paid attention. They were like, nobody cares, but we think it's the best thing since sliced bread. And we want to leave and turn this into a standalone business. But, you know, it'd be helpful to get

some of your advice on how to do that. And I couldn't really come on board full time at the time with them because I had to integrate my company into the acquire. Sure. But I came on as they're in drill and nights and weekends. I worked with them on the business plan and who we should raise from. That company is Anthropic. Dario and Tom and I started doing these weekly working sessions in early 2021. And yeah, I assumed that, you know, if we went and talked to a bunch of venture capitalists on Sandhill Road,

especially some of the ones who were involved in the biggest hits of the last decade before that, they would get it. These are the creators of GP3, and they were like, we just don't get this. We've heard the whole AI story before. This whole general intelligence thing is a pipe dream. And it was painful. We tried to raise $500 million. We couldn't.

We instead scraped together about $100 million, which I know sounds like a lot, but at the time was a rounding error compared to how much Google it's on the same kind of systems. And it was all angels in that first round, a bunch of cats and dogs, all of us who believed in the mission. And then over the next 18 months,

Darry, Tom, and team put together a plan that we kind of workshopped on getting Amazon involved as a strategic, and that resulted in a $4 billion compute and capital partnership that made me realize infrastructure, especially compute infrastructure, was just a key requirement to create any kind of modern AI lab. And so since then, I've spent the past five, six years figuring out how to unblock that compute bottleneck for research teams. Amazing. Well, obviously, an incredibly well-timed. It just emphasizes how much things have changed, right? Where, like, people are, literally throwing money at like almost any model now versus like a few years ago going like

AGI. Right. Right. I really know. Well, let me ask you this question because this is a very top of mind question for me. And we're getting we can skip around on the timeline here. But there are three labs that are seen as like genuinely at the frontier right now. And that is obviously deep mind within Google, open AI and anthropic.

And then of course, you know, a lot of people say that the Chinese. labs are very close, if not quite there. Maybe they're a few months behind. Is this, is there, you know, when we think about, like, part of your mission is like, you say, okay, a new lab should be able to get access to compute. If you're really bright, like, that shouldn't be the bottleneck. Does that imply, therefore, that you expect more labs to be able to, were they to have access to the compute, also reach the frontier, and that there is something inherent about like this sort of seeming stability or parity that we see among frontier models? So the answer to your first question is, yes, there are many frontiers to be conquered and pioneered.

And it's not just one frontier. I think that's a fundamental misunderstanding people have about the frontier. They talk about the jagged frontier. Exactly, jagged intelligence, right? In a poetic sense, in a historical sense, if you think about the Wild West or the Western frontier, it wasn't just one frontier. There was a frontier of gold and there was a frontier of genes. It turns out Levi's, you know, turned out to be a new modern.

behemoth of a company. I mean, there were so many new businesses founded in the Industrial Revolution. And I think that's the reality is the software engineering frontier, which is where Anthropic is clearly a leader, is one frontier. Yeah. I think the chat frontier,

the sort of consumer chat frontier is another frontier where OpenEI has been a leader. Arguably, bite dances at the video frontier with seed dance, right? Absolutely, yeah. And so I think there's just many, many frontiers to be conquered, or pioneered rather. I think Anthropic is clearly a role model

for the rest of the community on how to do it in an efficient way. They're, you know, I think fewer than 5,000 people, and they've been able to put out state-of-the-art models that, you know, teams like Google, which have 60,000 people, are close to, but not yet quite there. So actually, I don't really agree with your assessment that they're all at parity. If you use the models day in and day out,

they're quite remarkably different in meaningful ways to the person with hands on the keyboard, you know, doing the engineering work. And I think those different, So it reflect the focus of the teams, right? What is the actual mission that the team working on that domain cares about day after day after day. So in the Stanford class I teach, the first lecture was a breakdown of how frontier models

are even created. And it's actually quite simple. The recipe is super simple. There's basically four steps. There's pre-training, mid-training, post-training, and then what we call the continuous feedback loop. So pre-training just says, hey, you collect a business. bunch of data from the internet and train a model to be a generally good pattern recognition machine.

You then do mid-training, which is to say in a particular domain that you really care about, you inject more capabilities. So if you want this model to reason about science or math or physics, then you give it science or math or physics data. And then you get a pretty good model that's specialized in that domain. And then you deploy it to the real world where you have people using it. And the context feedback, which is when the model is able to do a task well or not, and you can verify whether that task was done correctly, gives the model the data it needs to keep improving on that task, on that distribution. Canadian women are looking for more, more to themselves, their businesses, their elected leaders,

and the world around them. And that's why we're thrilled to introduce the Honest Talk podcast. I'm Jennifer Stewart. And I'm Catherine Clark. And in this podcast, we interview Canada's most inspiring women. Entrepreneurs, artists, athletes, politicians, and newsmakers, all at different stages of their journey. So if you're looking to connect, then we hope you'll join us.

Listen to the Honest Talk podcast on IHartRadio or wherever you listen to your podcasts. The Big Take podcast from Bloomberg News keeps you on top of the biggest source. of the day. My fellow Americans, this is Liberation Day. Stories that move markets. Chair Powell opened the door to this first interest rate cut. Impact politics, change businesses. This is a really stunning development for the AI world

and how you think about your bottom line. Listen to the big take from Bloomberg News every weekday afternoon on the IHeart Radio app, Apple Podcasts, or wherever you get your podcasts. This is slightly tangential. But like, I give a lot of feedback to the models because Joe made me paranoid about the basilisk theory. So I want the models to appreciate me once they take over the world. But when you give them feedback, like if they spit out a wrong answer and you say, that's wrong, they immediately apologize and fall over themselves to say that they're sorry. But then you ask them, like, give me another output or like, would you do it again the same way?

And they like they often say yes or they give like a very similar answer. They don't seem to be responding in real time. Correct. So when I say feedback, I mean a very specific kind of feedback, which I call verifiable feedback. So when you say that wasn't right or that was wrong, that's an opinion. Okay. Verifiable feedback is when you can have as close to factual verification as possible. The reason... So what does that actually look like? That's a great question. So let's take reason by example in two or three cases. In the case of software engineering, the way software engineer is actually code is you write a piece of code and then

you submit it to the main code base and then you usually have a peer on your team, review the code and approve it or reject it. And if it gets approved, that's called, that's the first step. That's called a PR, a pull request. And if another human on your team that you trust, approved it, that's one kind of verification of quality. And then two, before that piece of code usually gets deployed to a production system, you have unit tests. And those are, you know, quite objective tests of, is this code performing the function we needed to? And if it passes both those tests, it's a verifiable piece of code that accomplished the goal. So in software engineering, the reason we've seen such a dramatic improvement in capabilities is that a lot of these labs

are using feedback from that verification loop. In the case of another lab I incubated called periodic labs, which we started a year ago, and you should come by sometime. We've got 40,000 square feet in Menlo Park where we've got AI models that are predicting new. The goal is to try to find a room temperature superconductor. And so these models predict... Oh, I forgot about it. A number of room temperature or superconductors. I forgot about that. That was a fun summer. Yes.

This time we will verify there. If we ever put something out, you will know it's not an idea. That's not going to be us. But the AI system predicts new materials candidates. Then we have robots that synthesize the new material in the lab and then use x-ray diffraction machines to test whether the material has the properties the AI said it would. And that's verifiable feedback from reality, from physics. And then we pipe that data back into the data back into the lab.

the training loop over and over again. That context feedback is very factually verifiable. Yeah. And that's where progress is the fastest today, because that feedback doesn't result in the kind of hallucinations that you often experience with these models on more subjective tasks. It's also, by the way, why the models are terrible at subjective tasks, like creative writing. And sometimes it can get quite toxic, to be honest, if you get them down the wrong loop. I don't know if you've been using it as a therapy bot and so on. I have not, just for the record. That's great. It did ask me to defy the laws of gravity at one point because I was trying to create something in my backyard. And I was asking how to do it.

And it was like, then just set this up, like a following way. And I was like, that's not within the laws of physics. Yeah. Yeah. Whatever. Well, what's interesting. This is actually a trillion dollar question from just a very broad standpoint is, as you point out, even prior to AI, the field of coding, had a very systematized approach to the feedback loops already. And so then it's like, AI can sort of replicate that.

Anyone who's any vibe coding can see in the chain of thought sometimes, oh, that didn't work, let me try this. That didn't work, let me try this. Yes. Most fields don't really have that by and large. Journalism doesn't have that. I mean, there are outputs that are better and worse. We don't really have that sort of like formalized approach to the yes, no.

Does that just zooming out, to my mind that would apply that maybe, at least to some extent, coding is a little bit special from a sort of white collar knowledge work that in terms of like, is it going to be as good as, say, I don't know, sales or something like that because it has a coding as a long history of that structured pipeline. Yeah, that's a great point. So where progress will be made most predictably is in parts of knowledge work where the task is essentially a workflow that's fairly structured. Yeah.

And so somebody who spends most of their day inputting cells into an Excel spreadsheet, well, that part of the job will get automated pretty fast because that's actually verifiable. And you know what? That's frankly often the most tedious part of the job anyway. And so I'm quite excited to see that progress because I'm terrible with spreadsheets. And I think if we could free up more of my time and hopefully other people's time to focus on the art of the spreadsheet, not the tedious part of it. The entry, the entry and retrieval.

Yeah. Yeah, exactly. And in journalism, I think it's the same thing. There's so much craft that goes into the verification of a story before it goes out that's not legible to the world. I've had a chance to spend some time with some of the journalistic institutions of the barrier like Cade Mets or Brad Olson at the journal. And as you spend time with them, you realize, I mean, they're verifying every sentence that goes into each article. Fact-checking, absolutely. So fact-checking.

That's an example where I think we should be leaning on these tools and we should, expect more progress. And the parts then that will be more, to borrow your jagged frontier framing there, that's, we will be in a regime of jagged frontier progress where wherever parts of workflows that are verifiable factually will essentially, you'll see progress there very predictably over the next few years. And consequently, wherever that progress, the workflows are not verifiable is actually where humans are going to shine.

And I think that's where parts of the economy are, you're going to see extraordinary gains in the wages of humans who have creativity and craft that are not typically verifiable through traditional objective means. Does that make sense? Yeah, it does. And it dovetails with a lot of what we've been talking about on the show recently. Just going back to verifiable feedback. So, okay, the model spits out something and you can check whether it's right or wrong. Is it important to understand how the model actually got to that answer? Because we have discussions with like. like big bank CEOs who are using more AI.

And their response to this question is always like, well, if we can put restrictions around the AI, if we make sure that it's like released into a sandbox before it's released into the wider world, we're all set from a regulatory perspective. And regulators don't actually need to know what's in the black box model and how it's working. But like this seems a bit concerning to me. Yeah. No, I'm quite strongly opinionated about this one, which is that technical literacy should

be non-negotiable. It's the reason I spend so much time teaching this class at Stanford, putting it up online. And the idea of the frontier systems class is that end-to-end, it's a full, simple, but first principles breakdown of how these AI systems are built from scratch, from land, power, shell, like the energy, where do we get them, the data centers, then how do we train the models? And the final project, the class with the kids, was actually the one-person frontier lab, which is, at the end, they're creating their own models and so on, because the idea is that a person with the right tools today can scale themselves infinitely, but they need to know how to use the tools, what the limitations are, when to lean on them versus not.

And I think this is a generalizable piece of technical literacy that all leaders should have. It's like saying, you know, I, in the 90s, I imagine, if you knew, you could use the internet without really knowing how it worked. But, you know, on the margins when, like, the page doesn't, like, refresh or you're like, this cookie thing is annoying me. Like, over time, people who were more technically literate just realized on some. sometimes you've got to debug, you know, the browser. And those of us who've learned over time to do knowledge work are more adept at leaning on them versus not. Like just now when I was trying to get onto the internet, I realized, okay, there's this, you know, Wi-Fi password, whatever. And then you don't end up relying on them in ways that they can't fulfill your need anyway.

And what's a little bit more dangerous with these systems is because we tend to anthropomorphize them without the technical literacy that I wish all leaders. had about reasoning about how these systems were built, what you end up doing is projecting out in your mind what the capabilities are in ways that are inaccurate. You project out their impact on society that are not accurate. You project out their business models in a way that are not accurate. I mean, the very fact that when you started this conversation, I don't blame you for it. You're like, Ange, there's three models at the frontier. Yeah.

I'm like, well, which frontier? And which three models? Because from where I'm sitting, there's like 17 different frontiers right now. there's four different players in each one, and the businesses of all of them are kind of breathtaking. So I think that technical literacy should always, for leaders, be a basic requirement. And then if you're deploying these systems at Goldman Sachs, you won't oversimplify and get tripped up later when, you know, two years later you realize half your employee base has been leading on this like sandbox framing. When in reality, inside the sandbox, they were doing all kinds of, they were using the tools in ways that were prone to hallucination, into risks, prompt injection, they were leaning on it in ways that were not informed in the

appropriate ways. Is this making sense? Yeah, like at a minimum, they would not be using it in the optimal way. Correct. Or relying too much on it. It's the, you can't outsource your understanding to a model. You can outsource your thinking. You can outsource part of the tedious workflows, but you can't outsource your understanding.

Yeah. And if you create these simplistic frameworks of, oh, here's a sandbox and this is safe, you have to use that sandbox in the right way because if you say, well, now everything that happens in the sandbox is totally fine. If the model says use the spreadsheet, the spreadsheet is good, it's deployed on our servers, but you didn't actually check the spreadsheet and what went into the spreadsheet. And did the model actually understand the particular structure of the business, the physics of the business that you're trying to model out, then you've outsourced your understanding to it.

Does that make sense? Absolutely. Let's talk about AMP. And because you're never going to get the frontier in anything unless you have access to compute. It seems pretty obvious. And there are various arrangements for acquiring compute. You have companies building their own data centers.

You have smaller labs and maybe they use someone else's data centers or a NeoCloud, etc. What are you building at AMP such that at least as part of this story is trying to solve to compute bottleneck specifically. Yeah. We are, it's very simple. What we're doing at AMP, we're doing two things. We are trying to standardize the format for compute, which today is super fragmented. So in the history of infrastructure, if you look at whether it was the industrial revolution,

the internet, streaming, there were usually formats of inputs that were quite heterogeneous. They were fragmented. And then to unlock productivity, you had to standardize a format. So in the case of electricity, until ACDC was standardized, right? Megawatts would just sit in stranded pockets around the United States being unused.

And then once we standardize the format to ACDC, then the question was, okay, great. Now we turned all these stranded pockets of electricity into one sort of interoperable universal format. Now how do we distribute it to everybody who needs it? And we came up with this distribution layer in the United States called the Great. grid. That's all we're doing.

Yeah. You're building a grid for compute. Correct. We're standardize, we're trying to standardize the compute layer today. Different chip types, different manufacturers, different clouds. I mean, it's a complete mess. And if you're- Go ahead.

Say more about how we plan to do this because we've talked before about, you know, there are various people out there that want to create indices of compute, of futures potentially on compute. And the issue that always comes up is fungibility. Right, exactly. So we've got a couple of things. A couple ways we solve the fungibility problem. This is a pretty thorny challenge.

We solve it in two or three ways. The first is we have a system called the grid, which actually makes the compute fungible at a consumption layer. So under the hood, we have a bunch of different chip types. We support various different manufacturers. And there's a system that was built to do this already inside a little company called Google. And one of the technical leads on that project was called Borg, internally at Google, is my co-founder, Sebastian Lobo. He was my roommate at Stanford 14 years ago. He's my engineering

co-founder. And we're building Borg for everybody else, which is essentially a translation layer that says no matter what the underlying chip type is, the machine learning researcher who's using the chip just has to worry about the workload. And we handle everything else underneath the hood. When you say system, is this hardware or software that's doing this? It's all software. Okay. Yeah. So we handle that translation layer in software. And it's a pretty gnarly challenge. But today, we're able to do that in ways that improve utilization sometimes from 50, 60% at labs that we have incubated are on the grid to close to 95, 96%. At Google, the utilization is roughly 99%. When Sebastian arrived at Google, it was about 62%.

By the time he left, it was roughly at 99%. At Google, if utilization is at 96%, that's considered a major outage. Today, the average data center in the industry, in the ecosystem, in the independent ecosystem, is running at less than 70% utilization. The Colossus 2, which is running in Memphis, Elon's 500,000 Gb-300s was running at less than 60% node utilization and less than 11% MFU. Model flop utilization is how much of the chip is actually being used. So there's two kinds of utilization people care about in the data center. First is how many chips are being used. That's the highest.

That's just the most naive measure. If that number is not a 90 plus percent, no excuses. So you have the chips. They should at least be doing something. And then within the chip, how much of the chip is being used, within a workload. That number is usually much lower. I'm very intrigued by this latter point about that even like the chip itself may not be even used at full capacity. Yes.

Because I see these numbers. And you say like a lab has like a, we have 200 chips. We've acquired 800 GPUs, et cetera. And when I see these headlines, I assumed that. optimal utilization techniques must be so good that you can infer someone's capabilities simply by how many Nvidia GPUs they've acquired. But you're saying is that there is actually quite a bit of heterogeneity about the techniques and approaches to getting the most juice out of ad chip.

Yes, you have to measure what matters. And what matters is output. Okay. When anytime I start a new lab with a team, you know, in the case of periodic labs, we started it with Liam Fettis, who was the co-creative Chad GPT, and Doeish Trubuk, who led the physics teams at DeepMind. And when we sat down and we planned out the company's roadmap, the most important thing to us to measure was not the number of chips we had. Yeah. It's the e-val, what we call. So all this chip bragging. They're like, oh, we inquire, it's just a sort of...

It's a lot bravado. Yeah, all right. This is helpful. You don't measure the inputs. You should be using the outputs. No, I agree. Yeah, of course. I'm actually fascinated that, like, there is a software solution to what I perceived in my head as like a very physical constraint. How does this actually work? Like, feel free to get technical here. Like, I want to understand the system. Yes. So let me give you the technological answer and the economic answer. The economic answer actually is a simpler one of the reason about. The way the compute business works today is primarily on the construct of the atomic unit of long-term leases. So I'm a researcher. I need some compute, I show up to a compute provider and say, hello, I would like some compute, please.

And the compute provider says, no problem. Here's 500 AMD chips or Nvidia chips that you can lease from me on various timescales. And you've got to pay for a 24-7. It's like leasing an apartment. And whether you use it or not, that's your problem, but it's $2.50 per hour, $3 an hour. So instead, you take a long-term lease. And now the cloud provider, the compute provider said, Great. I just booked revenue for the next two years that this guy rented time. Now, what happens with that compute? Whether it's used or not is the researchers' problem. They've outsourced that problem. As a result of this wastage that we're talking about, and I'm happy to go into why it's hard for individual teams to utilize most of the capacity. The primary reason is because research is spiky. It's hard to forecast. So you over-provision for your peak, not your base load. because what happens you're researching on these algorithms and the minute, like, one is working, you go, guys, let's scale. We want to ship this thing. So let's improve, throw as many chips at it. And then once we ship it, the needs go down. So between these spikes, there's just huge pockets of unused compute. As a result, the effective price per hour that you're paying is closer to $25 to $28. Whereas the marketed rate that you think you're paying is $2.50.

Yeah. So that spread due to wasted is just insane. So from an economic perspective, that's the wastage. That's the deadweight loss, right? Okay. So now how do we, from a technological perspective, how do we utilize that opportunity? Literally all we do is from a software perspective, we take all of that unutilized compute. No matter what format it is, it might be Nvidia, it might be EMD, we love AMD, might be some other chip.

And we turn it into one fungible resource. And that, we standardize the format on something we call grid credits. So researchers don't even need to think about what chip type is under the hood. You know, they're just paying what they need or what they use. And so from a fiduciary perspective, I'm on seven boards. As an investor, I get very excited when teams switch from this sort of long-term lease model where they're paying $25, $26 per GPU hour. Now they're actually only paying the $2.50 that was marketed,

because everything they're not using gets reallocated to the grid and other research labs can use that resource. Canadian women are looking for more. More out of themselves, their businesses, their elected leaders, and the world are out of them. And that's why we're thrilled to introduce the Honest Talk podcast. I'm Jennifer Stewart. And I'm Catherine Clark. And in this podcast, we interview Canada's most inspiring women.

Entrepreneurs, artists, athletes, politicians, and newsmakers, all at different stages of their journey. So if you're looking to connect, then we hope you'll join us. Listen to the Honest Talk podcast on IHeartRadio or wherever you listen to your podcasts. The Big Take podcast from Bloomberg News keeps you on top of the biggest stories of the day. My fellow Americans, this is Liberation Day. Stories that move markets. Chair Powell opened the door to this first interest rate cut. Impact politics, change businesses.

This is a really stunning development for the AI world. and how you think about your bottom line. Listen to the big take from Bloomberg News every weekday afternoon on the IHeart radio app, Apple Podcasts, or wherever you get your podcasts. The problem you're mostly solving for is the training part, because there's training, right? Training and entrance.

Or is it both? Both. Yeah. So the beauty about having diverse types of compute on our grid is that once you make the resource fungible, you can do any workload. You just fill all the unutilized pockets with inference, and then all the reservations would train. Yeah.

So can you explain, like, why is it that every lab also seems interested right now in customized silicon, including Microsoft announcing a chip that says, like, oh, our new MAI, I don't know, Maya, I don't know how it's pronounced. Oh, MAI, I believe. We had such a class yesterday at Stanford, and he pronounced it as MAI. Okay, their new MAI model. Yeah. And he's like, oh, we also have a new Maya. a 200 chip or something that's optimized with it. Why is it that so many labs or companies that are in a lab, I guess, feel impelled to

like also design a chip that goes along with the model? And long term is what you're doing saying like this really is not necessary to have that sort of model chip alignment. Yeah. There's two technological reasons and two economic reasons. Okay. The first is from a economic perspective, about 80 cents of every dollar. a lab spends today on their R&D,

flows to a chip provider like Nvidia. Okay. And so as a result, your margins are just super, super rough. So from a unit economic perspective, you want more control over your margins. And therefore, when you look at your like unit economics, you're going, wait a minute,

for every dollar we make, there's this massive chunk that's going to somebody else. So instead of spending 80 cents to Nvidia, you spend 78 cents to TSM and keep that two cents for yourself? Well, I think that the better are, software gets, the more that margin should flow actually to the researcher. Okay. Because that's where the value will be captured.

But like, but wait, sorry, you were going to say, what's the technical reason why they're trying to do optimal model chip alignment? On the technical side, the primary reason is you want control over your supply chain. Because today in a compute, well, we've been in a compute constraint world now for at least four or five years. But if you can't get the chips you need, you're not in control of your own supply chain. So you're dependent on compute allocations that the compute manufacturer thinks is optimal, right? By the way, that's how it works at the foundry level.

Today, TSM gets to decide which compute provider, compute provider's business grows or not. Yeah. Because they only have so much production capacity. And so the technological reason is you want supply chain independence. And so when you want economic independence, unit economic independence, and you want supply chain independence, you want as much control over your own chip. But Microsoft doesn't have a fab. That's not what I'm saying. What I'm saying is in inference, for example, Sacha would like more control over his unit economics.

So he's making an inference chip. Right. Because if he or dependent on a third party to give you the inference chip. Okay. And if you don't have an inference chip, you can't sell more product. You want more control. Is it about having a predictable supply of just for you rather than a predictable supply of, okay. Yeah.

So there's a lot of discussion right now about more efficient model allocation. So this idea that like you do not have to be using. the latest model to ask like what the weather is going to be tomorrow or something like that. And you also don't want to blow through your entire one-year token budget in the space of four months as Uber apparently did. So the spikes in usage that you're seeing that allow you to do the system and have grid credits, does some of that go away if people become smarter about which models they're actually using? Okay. So there's an embedded assumption I think I should tease a part in your question.

usage is different from the production of the model. So what's happening, right, in terms of the pipeline is you use the grid to produce these system, the model, and then the model produces tokens. If the end user is only using tokens, then as long as everybody, we have enough diversity in the end user base using models hosted on the grid, things actually even out. Okay. That cyclicality. In the same way, electricity in America evens out if you have enough scale.

At scale, basically, except when like there's a heatwave. There's a heatwave. Exactly. So some of that infrastructure, we are having to reboot. But you can think about AMP in the broadest sense as a utility company. We're what's called an independent system operator of the grid. So we don't own data centers. We don't own our own labs.

But we coordinate the capacity needs across different parties. And at sufficient scale, those usage patterns actually just gets evened out. Does that make sense? Yeah. Well, you're an investor. Well, you are an investor in OpenRouter, I believe, which I think is an interesting company. Do you see, setting aside AMP for a second, do you think that there is at this point still within, say, corporate America, a certain lack of saviness about knowing which model to route to for the query and that there will be an improvement and learning within companies, within users, so that you don't have these incidents for like massive token consumption. because perhaps everyone was using the wrong,

the Cadillac model and the Ford model would have been just as fine for that purpose? Oh, yeah. We're absolutely in the medieval ages of this technology. I think what will happen is increasingly, based on my conversations with corporate American leaders and corporate leaders across the world, they don't really care about the models.

They don't care about the underlying model, the technology. They just don't care. It's like too much complexity. We just want the work done. Yeah. Can you guys please figure out how to get the work done? in the cheapest way, in the most efficient way, in the most secure and trusted way.

And increasingly what you'll find is that which particular model is helping you out in a particular task will just be abstracted. You won't even think about that. It'll just be a companion. You're just going to talk to it. It'll be a companion provided by a brand you trust. And under the hood, they might be using 200 different models

to orchestrate your task. And over time, that efficiency will get better and better and better. And that's why I just don't think there's only three frontier models that are going to win. It's going to be an ecosystem. This is, I know you don't want my take, Joe. No, I actually do. I want your take.

This is my coffee pod theory of AI. I want your take, Tracy. It's all right. I love your take. I'll save it for the outro. Actually, on this note, we have seen some headlines recently. Obviously, there's the Uber one about token spending. And I think it was the COO said he wasn't sure if the ROI was there on Uber's AI usage.

And we've seen there was a good Vox article recently about a corporate reckoning with AI spend. Since you're going out and talking to CEOs, do you see any, like, has anything shifted in the past couple months or so in the way people are thinking about the return on this initial investment or the return on spending on tokens? Yes. I think it's a barbell distribution. So there's two types of CEOs, broadly speaking. The first is the CEOs who are using the tools themselves. And those folks are going, aha, I understand the jagged frontier. When they understand the Jagged Frontier we talked about, their strategies, their questions they ask me are completely different from the CEOs who are outsourcing their understanding. They're not trying the tools.

They're mostly asking their kids like, hey kiddo, this chat GPT thing. Like, it's good, right? And your kid is like, yeah, it's pretty good, dad. And then you go- My kids think it's really dumb, by the way. Yeah, so that's the other thing, right? So the kids are super smart. And they're using the tools.

they're like, it's good at this thing, but not at that. So they understand the jagged frontier part. Actually, you know what? They think I'm dumb for using it. They're like, dad, like, you're not doing anything smart. You know, they don't think the models are dumb. They think it's dumb of me. Exactly.

They might be going, the way you're using it. Exactly. It's not optimal. So the C. No, what I'm saying is my kids are six and they have no, and 10, and they have no idea about anything and they just think I'm dumb. That's the whole point I'm trying to. That's really the only point I'm trying to make.

I see. Okay. Well, you can send them over to me anytime. I'm happy to be the fun uncle. Yeah, that would be great. You can show that actually this is fun to play with. My wife and I are happy to host your kids anytime. That's really what I'm trying to get it.

It's the summer. We have two nieces in London, and we call it Camp Midas-Shan. My last name is Midda. My wife's name is Shen. And so you're welcome to send them to Camp Mirrishan anytime. That's amazing. But that's the bifurcation. As leaders who are actually trying the tools out,

they realize they're extraordinary at some things and not at others. And so depending on whether you get it or not, or you're actually getting your hands dirty or not, I find the questions are completely different. So this has been an incredibly helpful conversation in terms of like understanding, basically the problem of essentially tons of money is being spent.

And your thesis is that it's massively suboptimally used up and down the stack. You mentioned this, okay, you get a credit, et cetera. Like, do you actually see that being financialized in a way I mean, okay, I bought this capacity. I have a lot of unused time. I mean, I don't always have a research idea that is going to require a big model run test. I can resell that.

Is that something that you see, like something that genuinely resembles a financial market? I hope not because when you had speculation to, you know, production goods, it creates scarcity of a different kind, right? Because then you have financial traders and markets trying to trade. the speculative value of the asset, and that's going to hurt a lot of our research teams in technology. On the other hand, I think that creates a need for innovation inside of the research teams. So one of the core operating functions we have inside of our business is a forecasting capability where we have a team that's very similar to actually the kind of forecasting

team you'd have inside of a hedge fund. We're constantly predicting demand and supply, and then we're actually procuring capacity in advance through call options on compute clusters. But our needs are similar to the kind of internal trading desk you'd have inside of a large steel company, right, where they need to lock up iron ore and so on for their production needs. So I'm a big fan of efficient markets. And I'm trying to actively invest in and help entrepreneurs out and teams out who are trying

to drive more efficiency in the service of more productivity in science and engineering. I'm not that thrilled about the financialization of these products. it ultimately results in more speculation. Does that make sense? Yeah. I'm just curious, since you're tracking demand in that way, like if you were going to describe the slope of demand right now versus, say, like, a year ago, is it steeper? Is it starting to plateau? Perpendicular. Oh, wow. Okay. If you look at the compute prices of long-term rentals over the last six months, between January and now, they're trading up to X. So we started, for example, for 26, we started securing our capacity in January at these long-term rates. We could resell that

at a 2x markup if we wanted to. Part of the reason that 2026 has become just totally AI has consumed everyone's mind, I think, is because people got very excited about Claudecote specifically. Yeah. But that was a, that was a breakthrough at the harness level, not the model level, right? Suddenly, like the really excited is like, wow, this is just so fun. It's just so easy of a computer inside your computer. That was a harness breakthrough. Do you see like when you think about investment among AI labs, do you see any shift in allocation away from pure scaling and improving the model towards sort of like tooling and harnesses as a way to get more juice out of the models? No, I'm sorry, I have to correct you there. It was not just a harness

innovation. Those two things go hand in hand. It's a symphony of improvement between, it's a dialectic between the model capability and the harness. That harness was designed specifically for the capabilities that the new model was going to have. And so when you design these things, in the industry, we call this co-design. So you have the harness designed side by side with the researcher who's designed the next generation capabilities in the model. And you get a little bit of visibility in where the model is going to be good, because as I described earlier, the pipeline is actually quite predictable. Pre-training, mid-training, continuous feedback loop. Once you have that visibility, you go, aha, we specifically want to improve the capabilities on this type of task.

It's going to take us about three months to get there. Start designing the harness for that improvement. By the time they show up, then you can have the harness assume that the model will be able to do X, Y, Z on its own, whereas A, B, C, it's going to need third-party tools. So then the harness says, remember that three months ago, you were terrible at understanding a spreadsheet. Yeah. So then we had to go, right, like, go use a third-party tool to use a spreadsheet. In the last three months, what we've done is added the ability to actually reason about a spreadsheet in the model.

In the model, not the-the-law. So now you don't need to use a third-party spreadsheet. Okay. And so then the harness gets updated to say, don't go out and use a third-party spreadsheet, which, by the way, collapses the time required to do that task by, like, sometimes a minute to two minutes. Now suddenly I've improved the user experience. And that's when things really sing. It's when both of those parts, the model and the harness are co-designed to create a symphony.

Does that make sense? Yeah, absolutely. All right, Anjene Midda of AMP PBC. Thank you so much for coming on Odd Lots. Really appreciate it. Thanks for having me. And everyone go out and check out the Stanford lecture series. It's on YouTube, right?

It is. CS153. Stanford.edu. Perfect. I have a big flight coming up, so I'll watch it then. You should download all the lectures. There's quite a few. Thank you so much, Anjane.

That was fantastic. That was great. All right, Joe, that was a great discussion. Yeah. I should emphasize just how big a deal that lecture series actually is at Stanford. Like, students are beating down the door, basically, to get into that. And if it's free on YouTube, you should definitely check it out. I just want to establish that if I had given you the A, the insinuation that I didn't want to hear your take,

or B, the idea that I would have wanted to hear Anjone's take instead of yours. I want to hear your take. No, it's fine, Joe. I realize that most listeners are here for the guest takes. I get it. But I thought his point about the jagged frontier was an important one. And this idea that, like, maybe the future, it's not going to be a winner-takes-all thing in terms of models. You're going to have a bunch of different models doing different things that might suit different companies.

And also the idea that, like, a lot of companies aren't going to care about which specific model they're using. They just want the cheapest one that basically gets the job. done. In my mind, that sounds like more of a, is commodified the right word? Yeah, a commodified market, right? Rather than like, oh, people are going to pay up for, as you said, the Cadillac model. Well, so what I would say is by, in listening to Anjane and AMP, is that people will want a commodified service, but that under the hood, I mean, this just sounds like what he's really trying to solve and it's very interesting. I, as a user or a company, buy a commodified service, but under the hood, the commodity has

an incredible amount of variety of models through which it can route. Sure. Some of which will be the Cadillac. Some of it will be the Currude Coffee Cup. Yeah, absolutely. But, like, my point is maybe in terms of valuations. Sure. Right.

Like, if everyone is assuming that the Cadillac is going to be, like, the one that everyone is going to get and the total available market, the 10. am, infamously is like not just the world, but potentially the universe. Like that seems a stretch to me. Totally. And just generally, I thought it was super interesting. And the idea, we've done a couple episodes recently, specifically learning more about both chip level and box level optimizations, both how many chips you're using and how well

you're using ad chip. Definitely way more to do on that. It still blows my mind that this is a problem that can be solved with software rather than like something physical. Like you just come up with a way to efficiently allocate the compute. Yeah. Because in my mind, like it's such a physical problem

and we've talked to, you know, previous AI market participants like Brandon McBee at CoreWeave. And they talk about like, oh, it's difficult to standardize because of the configurations of chips and things like that. But if you could solve it just through a software system, that's pretty crazy. I guess Google's already done it. Yeah. All right.

Shall we leave it there? Let's leave it there. This has been another episode of the Odd Thoughts podcast. I'm Tracy Alloway. You can follow me at Tracy Allaway. And I'm Jill Wisenthal. You can follow me at The stalwart. Follow our guest, Anjene Midda at Anjane Midda.

Follow our producers, Carmen Rodriguez at Carmen Armid. Dashel Bennett at Dashpot, Kail Brooks at Kail Brooks and Kevin Lazzano at Kevin Lloyd-Lazano. And for more Oddlots content, you should check out our daily newsletter. You can find that at Bloomberg.com forward slash all thoughts. And you can chat about all of these topics 24-7 in our discipline. Discord.g.g. slash oddlots. And if you enjoyed this conversation, then please leave a comment or like the video or better yet, subscribe. Thanks for listening.

The Big Take podcast from Bloomberg News keeps you on top of the biggest stories of the day. My fellow Americans, this is Liberation Day. Stories that move markets. Chair Powell opened the door to this first interest rate cut. Impact politics, change businesses. This is a really stunning development for the AI world. And how you think about your bottom line. Listen to the big take from Bloomberg News every weekday afternoon

on the IHeart Radio app, Apple Podcasts, or wherever you get your podcasts.