On with Kara Swisher - Microsoft AI CEO Mustafa Suleyman on AI “Friends”, OpenAI “Siblings” and Climate Change

Episode Date: October 3, 2024

Will AI “agents” soon be personalized teachers, doctors, companions and even check items off your to-do list on their own? “Agentic” is the latest buzzword in AI and Microsoft AI CEO Mustafa Suleyman says moving beyond the text chatbot to a "smart friend” is the goal. The former co-founder of DeepMind, Suleyman helped grow Google’s AI division before launching another start-up, InflectionAI. Earlier this year, Microsoft paid $650 million for the licensing rights to Inflection, and brought Suleyman and most of his staff on board. Kara spoke to him at this year's Lesbians Who Tech conference about his strategy for integrating Copilot into Microsoft’s existing product suite; why he views OpenAI more like a sibling than a competitor; and why renewable energies (and a lot of cash) will be vital in meeting AI’s massive energy needs. Questions? Comments? Email us at [e-mail] or find Kara on Threads/Instagram @karaswisher Learn more about your ad choices. Visit podcastchoices.com/adchoices

Support for this podcast comes from Anthropic. It's not always easy to harness the power and potential of AI. For all the talk around its revolutionary potential, a lot of AI systems feel like they're designed for specific tasks, performed by a select few. Well, Clawed by Anthropic is AI for everyone. The latest model, Clawed 3.5 Sonnet, offers groundbreaking intelligence at an everyday price.

Claude Sonnet can generate code, help with writing, and reason through hard problems better than any model before. You can discover how Claude can transform your business at anthropic.com slash Claude. Do you feel like your leads never lead anywhere? And you're making content that no one sees? And it takes forever to build a campaign? Well, that's why we built HubSpot. It's an AI-powered customer platform that builds campaigns for you, tells you which leads are worth knowing,

and makes writing blogs, creating videos, and posting on social a breeze. So now, it's easier than ever to be a marketer. Get started at HubSpot.com slash marketers. Support for this show is brought to you by Nissan Kicks. It's never too late to try new things. And it's never too late to reinvent yourself. The all-new reimagined Nissan Kicks is the city-sized crossover vehicle that's been completely revamped for urban adventure. From the design and styling to the performance, all the way to features like the Bose Personal Plus sound system,

you can get closer to everything you love about city life in the all-new reimagined Nissan Kicks. Learn more at www.nissanusa.com slash 2025 dash kicks. Available feature. Bose is a registered trademark of the Bose Corporation. Hi, everyone. From New York Magazine and the Vox Media Podcast Network. This is On with Kara Swisher, and I'm Kara Swisher. My guest today is Mustafa Suleiman, the CEO of Microsoft AI. I started talking to Mustafa about artificial intelligence over a decade ago.

In fact, the way I met him was when I called to tell him that I had broken the story about Google buying his first AI company, DeepMind. I believe the response was, oh, fuck. He went on to work on DeepMind at Google for several years and was critical in the company's AI development. Since then, Mustafa has been dipping in and out of big tech. When I spoke to him last year, he left Google and founded a new startup, Inflection AI. Their personal AI chatbot, Pi, was supposed to be more kind and sensitive, more, quote,

woke, and Mustafa was very startup-y. Inflection AI raised a lot of money, $1.3 billion, a good chunk of which came from Microsoft. So it was maybe not so surprising that earlier this year, Microsoft offered Inflection $650 million for, quote, licensing, and brought Mustafa, his chief scientist, and 60 of the 70 of his best employees on board. They're not calling it an acquisition, but of course the FTC is investigating, because once again, we're seeing massive consolidation of power in tech, and this time it's in AI. Of course, I wanted to grill Mustafa about that, his job as head of consumer AI at Microsoft and the climate change consequences of AI, his take on regulation, and some of the latest buzzwords, such as agentic. I spoke with Mustafa before some big news broke at OpenAI, which Microsoft has a large stake in. OpenAI is trying to convert into a for-profit company,

a seismic shift for a corporation that was founded as a non-profit. Just after the news, Chief Technology Officer Meera Marathi abruptly quit. Chief Research Officer Bob McGrew and VP of Research Barrett Zoff also both out. CEO Sam Altman could potentially get equity. So unfortunately, I didn't get Suleiman's take on all that, but the interview was at one of my favorite conferences, Lesbians Who Tech in New York, where you're in a big outdoor stage overlooking the East River, which had a lovely view. A little less lovely was the helipad nearby, so there is a bit of noise, but it's a pretty cool conversation. I think you're going to like it.

It is on. So Mustafa, thanks for being on On. Thanks for having me. I'm happy to be here. First, congratulations on your new job. I think it's relatively new. This past March, you joined Microsoft as CEO of Microsoft AI. The last time we spoke was June of 2023, which seems a million years ago. And your startup

Inflection AI had launched a personal chatbot, Pi, that was just before you got a $1.3 billion influx, some of which came from Microsoft. Reid Hoffman reportedly called your hire a peanut butter plus chocolate moment. First of all, do you feel like a peanut butter cup? But in all seriousness, do you want it to be a startup at the time? Talk a little bit about the shift to big tech, because Inflection was one of the series of hot AI startups, and you've been around for many years at DeepMind and everywhere else. So talk a little bit about that. At the time I was at Google, this is sort of around 2022. And, you know, Google was sort of

struggling to really get started in the LLM revolution, even though it had been one of the main inventors of the core technology transformers. And early to investing. Yeah, super early. And internally, I worked on Lambda, which was the precursor to Bard, which then became what we have now as Gemini. And we were really trying to get this deployed into search.

And the company was sort of proceeding quite carefully and cautiously and maybe a little too cautiously for a bunch of us. And we just felt that it was the right time to try and get these things into production and get feedback and experiment with them. Why the caution? It's safety, bureaucratic situation. I admire the caution because you don't know what you don't know.

And I think these models are really powerful. They're fundamentally different to the kind of software that we've built in the past. And so you just don't know what's going to happen when they make contact with reality. Right. And, you know, it's a bold step to go first. And ultimately, in the end, it was OpenAI that forged ahead.

And then, obviously, the rest is history. Everyone else has caught up. So when you're over at Google, which is one of the biggest tech companies in the history of the planet, and they're moving slowly and you see something like OpenAI in an area you had been in for a decade, when you were thinking of going, what is that calculus? Because you started as a startup, gets bought, you're in this powerful place, and then you want to go.

I mean, I think what's changed is that the cycles of iteration and innovation are just getting tighter and tighter. So there was this window where it felt like if we could launch first that inflection, then Pi could end up becoming the number one conversational AI. And it turns out that we launched after ChatGPT. ChatGT had obviously exploded, the fastest growing piece of software or experience of all time of any kind. And so, you know, in this market, in this industry, being third or fourth is a very, very difficult position to play from. What I realized over the course of the sort of two years that we did inflection is that, you know, we really have to be first with these things. So, but you had this company and you were sounding like you liked being independent

and then suddenly you're an employee of Microsoft. Well, the mistake that we made at Inflection is that we built our own pre-trained models, right? So when I first launched, part of the reason why we raised so much money is because we built a model that was as good as the absolute cutting edge from DeepMind, OpenAI, and from Microsoft.

And within eight months of us building that model, multiple groups had completely open sourced it. So it kind of undermined our core business proposition. I didn't expect that these things were going to be open sourced at this scale. I thought that these are so valuable, why would they get open sourced? And it turns out there was an entire ecosystem around open source at this scale. I thought that, you know, these are so valuable, you know, why would they get open source? And it turns out there's an entire ecosystem around open source. So what did you say to yourself? Uh-oh.

I was like, eek. Oh, no. That I don't remember. So what did you do then? And why Microsoft? Well, Microsoft was actually our main investor from the outset anyway. I have an amazing, longstanding relationship with Satya and the team there. And you know, they sort of basically said to us, look, we have the resources. We have a big gap in the company. We need a team like yours to come and really lead consumer. Because I think whilst the company in the last decade has been incredibly successful at cloud and enterprise sales and so

on, you know, where the company hasn't really made traction is on mobile, for example, or in sort of really younger consumer more generally. And that's what we're really targeting at the moment. Targeting at Microsoft. But they also had their big investment in OpenAI, which many people lauded as a really smart move on his part. How do you look at that relationship? Yeah, I mean, I think, look, I think that investment in OpenAI is going to go down as one of the great moves in corporate history. It turned out rather well, too. Yeah, it's not too bad. It's not too bad.

Yeah. I made fun of it at the time. I look like an idiot, but go ahead. Well, hindsight's a wonderful thing. I thought this one was smart, but anyway. Yeah, I mean, it was also brave. I mean, a billion dollars in 2019 on a technology that was very, very far from doing anything in practice. It was pure research. So it was really just a kind of trade as like, they couldn't get deep mind, so they had

to bet on something else. And I think that turned out to be amazingly successful. So what is the relationship between them? Are you competitors? How does it? No, no, no. We're siblings. You know, sometimes siblings squabble. Sometimes they squabble, but largely we're on the same team, you know? So, and that's amazing for us. Like, we collaborate closely with them on everything from, you know, the research side to core infrastructure and so on. Obviously we build competing products. Right. And I think that's healthy and a natural part of things. And so it's a good relationship. And they've stepped out with Apple, for example, by doing a relationship,

which you all could do because Microsoft provides software to Apple and things like that. Yeah, I mean, I think one of the things people don't realize is that all the companies actually have deep partnerships with other companies. I mean, we have a deep relationship with Amazon, believe it or not, on a lot of fronts. So there's a lot of connections across the board, it's normal. Right. So what is your purview and goal for what you're doing? You have Bing search engine, Copilot, Chatbot app. What else? Yeah, MSN and also Edge, the browser, as well as our core AI science division.

So you integrate AI into all the products. core AI science division. So you integrate AI into all the products. Yeah, that's exactly the goal. It's that we sort of wanted to build a vertically integrated AI studio. What that means is we have the browser for distribution, we have the search engine for content, we have MSN for news, and now we have the consumer co-pilot, which is trying to produce this really conversational, fluent AI companion. Which is a search engine, really, in a lot of ways. Yeah, I mean, you could think of it as a search engine, but I think it's going to evolve way beyond a search engine. I mean, this is going to become a lasting, meaningful relationship.

People are going to have a real friend that gets to know you over time, that learns from you, that is there in your corner as your support. Yeah, a friend is doing a lot of work there. But the word that Satya used was agentic, I guess. Agentic. Agentic, whatever. You're making up words over there at Microsoft. What does that mean, agentic?

I mean, these are automated agents. How are they different than chatbot or Pi or whatever? I think it's an aspirational term, maybe just like friend. Yeah. That is trying to say like... Because agentic just trips off the tongue. It's an awkward one, isn't it?

Look, it's my agentic. Yeah. People would be like... You know, who used this phrase? Did someone I know use it? Agentic? Yes, Satya, your CEO. Yes.

Oh, no, okay. Yeah. Well, we can't all be perfect. He. Oh, no, okay. Yeah, okay, yeah. Well, we're not, we can't all be perfect. He also put it on social media. It was everywhere. Yeah, I mean, this is the mode we're in. You know, I prefer companion.

Companion, okay. Which is also aspirational. We're not quite there yet. But it's basically getting at this idea that, you know, these AIs are really going to live life alongside you. I mean, they're going to, it's not just a tool. Right. This is something that is going to end up forming a lasting, meaningful relationship with everybody over time. It is going to be your teacher. It is going to be your medical advisor. It is going to be your support network. And ultimately, it's going to take actions on your behalf.

That's what the adjunct piece is getting at. Co-pilot is probably a better term. That's the one that you use at Microsoft. It's interesting because right now, the manifestation of it, and you're starting to see it. I was just at the Apple event in Cupertino. You're starting to see it in little ways.

Most of it is irritating on some level, like Google, for example, with rewriting your emails. When you start to write an email, they're like, can I help you with that? And my email is fully, yes, no, okay. And they're like, can I rewrite that for you? I'm like, no, you can't. And it took me more steps. So it doesn't look particularly helpful yet. That said, a lot of people are using it for summarization, for notes, for ideas and things like that. So there's two versions of this, which is something that is a work helper versus what you're talking about, which is a companion and a friend. So without becoming a Microsoft ad, and you can talk about others, I would love you to talk about others, how are the business and consumer sides connected and where do they diverge?

And where does it leave products and others behind? Because it'll ultimately integrate everything, presumably. No, I mean, it doesn't have to, right? And it's also unclear how... Can I just make an observation? I think search is neat. At the time search was invented, I remember saying to Larry and Sergey, it's Neanderthal.

In 25 years, we'll be like, why did we type words into a box? And they were insulted by that. But nonetheless, they made a lot of money. Well, I think, I do think the new modality is going to be increasingly a voice first modality. Right. Because, you know, we've been kind of crawling up to high quality transcription and voice generation. Very slow. For a decade. Yeah. Dragon systems. It's kind of just been irritating, like you said. It's been irritating, irritating, irritating,

kind of a bit better, nearly there. And I think the last year or so, it's sort of tipped over where it's like, wow, you really can have a fluent conversation with this thing. You can interrupt it. It's super accurate in its transcription. And I think that just unlocks a different way of thinking when it comes to, you know, asking a piece of technology to do something.

Well, the thinking has been there from science fiction, right? If you watch any Avenger movie, when he talks to Jarvis or whatever, that's the kind of relationship you're thinking about. What the experience has been for most people in the consumer space, and I'm not talking about bigger things that it will do, like, you know, cancer research, et cetera, et cetera. It feels like Siri is the dumbest friend you've ever had kind of thing. And I have said that it is. It's still terrible. And Alexa is not too smart either. It's like, what is going on with you men? No, you're right. You are damn right. Yeah, I hear you. Yeah. The problem with technology and the problem with people like me is that we get overexcited and overstate, you know,

where things are about to go as though they're already here. Like, I remember watching like a Sony Walkman advert and seeing the, you know, guy rollerblade down the boulevard with his CD and the thing. And then I got mine and I was like, tried to kind of walk down the street and it just skipped every five seconds. Yes, it did. It was kind of useless. And, you know, if I had it on my desk sitting perfectly flat,

it sort of worked. And so this is the challenge. Like, you know, we have a tendency to overstate things. And at the same time, it always feels like this time is different. I mean, it is profound that you can now have a fluent natural language conversation about any topic. You pick any topic and it's going to be better unless you're on your specialist area of expertise. Right. That's quite a profound thing. So talk a little bit about what, from your perspective, the timeline is for truly integrated where your AirPods talk to you and you have a, you know, not like her, which was, I did not like that movie because I thought if you're that smart,

why are you talking to this dumb Joaquin Phoenix guy? The director, the director of it, Spike Jones, who's lovely. I was, he came up to me and he was, I really want to know what you think, Karen. I went, and he was like, what? And I go, I was like, Larry David. Cause I was like, this is what happens

when intelligence shows up. They flirt. Like, I don't get it. But nonetheless, what do you see the timeline for the relationship developing? I think that next year we will largely eliminate the hallucinations. They will get reduced to a level where you really can use it for practical everyday use cases. You'll rely on it. It'll give pretty good information. Right. And it'll increasingly replace the 10 blue links format. Right. I think beyond that, it's going to have memory. Like that's a kind of missing piece, which means that you don't really like want to invest in it in a lasting way at the moment, because it's frustrating that it forgets that you told it

that you were going out to dinner last night and that, you know, you had an undergrad at XYZ. So the bad assistant. Yeah, it's kind of, that's boring, right? And so you don't bother really giving it your all. Just think about the moment when like, it really does remember everything that you've said and it's able to retrieve from that history

of your conversations accurately. And the timeline for that? I think that we're 18 months away from really getting that to a workable level. What company is critical to that? Apple, I'm assuming, for example, because that's where people have that experience, correct? Well, actually, I think it's mostly about recording new memories that you share with it on the fly rather than accessing existing data. People often think, oh, it's going to sort of plug into every one of your sources. Your apps, whatever. It can do that.

It can do that. But I think what I found is that mostly the valuable data is the fresh information that you impart in a conversation over the last few days. As part of investing in a relationship. It's interesting. And then you choose what to share and how you frame it and stuff like that.

Does that mean, I raise this with Tim Cook, does this mean, I think the apps are going away. You will have a relationship with Uber, but not in the same, because there's friction of opening things up constantly. I'm opening my Uber app, I'm opening my search app. That will go away. I don't think they go away. I think that your AI is just going to learn to use those apps.

That's what I mean, because you're never going to touch them again. Right, right. I mean, yeah. Look, first of all, the efficiency of tapping on something and very quickly, you know, I think ordering an Uber is like five clicks. I mean, that's hard to beat. So it's going to take a few years to really get beyond that.

But if you look out far enough, yes, it's quite likely that your AI is going to learn to click on buttons in your browser, enter text in text entry boxes. It's certainly going to be able to read the page and understand the images and all of the text on the page. And that's going to be a very different, you know, format. I think one of the things that's going to come out quite soon, I would say, is this idea of co-pilot's vision. It's going to really see what you see. And so you'll be able to have a companion that really browses with you, looking at the images.

See what you see physically or see what you see as how you use it? I think at first digitally. Right. But then very shortly after that, there will be a visual awareness, whether it's through earbuds or glasses. Yeah, earbuds with video in them. Yeah, I think that's going to be a new format, is these visual earbuds that are kind of... You saw the Snapchat glasses that came out. Yeah, I think they were cool. They're rather big.

They're a little bit chunky. Chunky, yeah, that's a nice way of putting it. I do think my pivot partner, Scott Galloway, is correct, is that anything that makes you unable to procreate is not going to work for people I think when you get when it gets to be something that's easy like an airpods with video I think is will be something like that and they won't have like exact person recognition they'll have spatial awareness so they'll know like you're outside or you're approaching a car or there's three other people nearby and so on. We'll be back in a minute. Fox Creative.

This is advertiser content from Zelle. When you picture an online scammer, what do you see? For the longest time, we have these images of somebody sitting crouched over their computer with a hoodie on, just kind of typing away in the middle of the night. And honestly, that's not what it is anymore. That's Ian Mitchell, a banker turned fraud fighter. These days, online scams look more like crime syndicates than individual con artists. And they're making bank. Last year, scammers made off with more than $10 billion.

It's mind-blowing to see the kind of infrastructure that's been built to facilitate scamming at scale. There are hundreds, if not thousands, of scam centers all around the world. These are very savvy business people. These are organized criminal rings. And so once we understand the magnitude of this problem, we can protect people better. One challenge that fraud fighters like Ian face is that scam victims sometimes feel too ashamed to discuss what happened to them. But Ian says one of our best defenses is simple. We need to talk to each other. We need to have those awkward conversations around what do you do if you have text messages you don't recognize? What do you do if you start getting asked to send information that's more sensitive? Even my own father fell victim to a, thank goodness, a smaller dollar scam, but he fell victim and we have these conversations all the time. So we are all at

risk and we all need to work together to protect each other. Learn more about how to protect yourself at vox.com slash Zelle. And when using digital payment platforms, remember to only send money to people you know and trust. The Capital Ideas podcast now features a series hosted by Capital Group CEO, Mike Gitlin. Through the words and experiences of investment professionals, you'll discover what differentiates their investment approach, what learnings have shifted their career trajectories, and how do they find their next great idea? Invest 30 minutes in an episode today. Subscribe wherever you get your podcasts. Published by Capital Client Group, Inc.

Support for this podcast comes from Anthropic. You already know that AI is transforming the world around us, but lost in all the enthusiasm and excitement is a really important question. How can AI actually work for you? And where should you even start? Claude from Anthropic may be the answer. Claude is a next-generation AI assistant, built to help you work more efficiently

without sacrificing safety or reliability. Anthropic's latest model, Claude 3.5 Sonnet, can help you organize thoughts, solve tricky problems, analyze data, and more. Whether you're brainstorming alone or working on a team with thousands of people, all at a price that works for just about any use case. If you're trying to crack a problem involving advanced reasoning, need to distill the essence of complex images or graphs, or generate heaps of secure code, Clawed is a great way to save time and money.

Plus, you can rest assured knowing that Anthropic built Clawed with an emphasis on safety. The leadership team founded the company with a commitment to an ethical approach that puts humanity first. To learn more, visit anthropic.com slash clawed. That's anthropic.com slash clawed. One of the interesting things about your job is this relationship with OpenAI. And you and Sam Altman are often pitted as rivals,

but you recently praised him. Talk about what is the dynamic? Because the Wall Street Journal has reported you're building a, quote, in-house OpenAI competitor inside of Microsoft. a quote, in-house OpenAI competitor inside of Microsoft. And talk about why Microsoft is replicating the resources that it also has with OpenAI. What's the strategy there? No, look, it's not about replication.

I think it's more about having a partner on the Microsoft side that genuinely understands the technology that the company is receiving. Right. Because in order to really, you know, use this new clay, it requires a level of technical and scientific craft that just, it comes with experience,

right? So we see them very much as a partner, not a competitor. And I have great respect for what they're doing. So they just released a new reasoning model. It's O1 Preview or Strawberry. Have you been able to look in the hood? Can you explain the difference in the models? And it seems to have a unique ability to scheme, which is pretend to follow the rules, which sounds problematic. It's even worse than hallucinations.

You don't really want a scheming friends. So talk about that. You know, the thing it's really good at is that when faced with a kind of complicated reasoning challenge, it can critique its own answers and iterate to find the right answer before giving you an answer. It's why at the moment it's a little slow. It takes sort of 10 to 30 seconds to give an answer. And that's actually pretty useful because it gets feedback and calibrates and iterates before producing a final output. And that's actually pretty useful because it gets feedback and calibrates and iterates

before producing a final output. And it turns out that that's actually pretty valuable. I mean, we all do that all the time. Like we think in our own heads out loud and we try to reason through a problem before giving an impulsive answer. The moment these models kind of give an impulsive answer, they just give the first answer that comes out, right? Right. You just said something that hallucinations will go away, that it will seem... I think they're going to go below a threshold and make it practical, right? Right now, you know, they're getting pretty good if you use the very best models, but they still make some basic errors. And I think those more basic errors are going to get reduced to an acceptable level. And the highly specific ones will be trained on very clear data, correct, that is accurate.

Yeah, especially if you want to sort of focus them on a domain of expertise, like a legal domain or financial accounting or get to know your company's knowledge really well or medical, exactly. Right, and it has to be. It can't be scheming or being hallucinating for that. Speaking of competition, let's talk about Elon Musk. Reid Al-Burgetti at Semaphore recently proclaimed that Elon is winning the AI race. I wrote him a note and just wrote, ha ha. Because of computing power, he has built a data center in Memphis for XAI, apparently breaking quite a few

rules. What a surprise. Well done, Elon. He says about 100,000 H100 GPUs are operational and another $100,000 in the way. He does make things up. So let's take that with a grain of salt. These are graphic processing units for high-speed processing of large data. He's doing a who's is bigger here thing because he's hopelessly insecure and narcissistic. But would you put that into the scale? Just over a year ago, Inflection AI was building what it called the largest cluster. And that's another word you guys like to use,

cluster, in the world with 22,000 GPUs. Can you talk a little about who's bigger, I guess, if I have to ask? Jesus. You can take it, Mustafa. I would not ask that of Satya or Sundar, but go ahead. I'm going to ask it of you.

Jesus. I mean, look, Elon's a pretty big guy. What can I say? I mean, it's a big cluster. I think everybody else is training at roughly that same size. So it's impressive that he managed to go from a standing start

to now having a competitive cluster with Microsoft and OpenAI and Google and so on. Do you believe he's telling the truth? I don't know. I really genuinely don't know. If he has done it in that timeframe, that will be super impressive. And I think it's kind of unlikely.

But, you know, I genuinely don't know. Yeah, well, he's going to put out that robo-taxi is going really well. Right. He kept saying about robo-taxi, and Waymo has had millions of miles. He's had zero. Yeah, yeah. And the humanoid robot yeah I mean look

we need that kind of vision I love it he told me he was going to Mars five years ago but I wish he would so that would be great good luck godspeed that's what I always say sounds like a great idea for most of the human race although he's you know he's contributed but talk about that competition is that a big competitor how's

twitter going so that said who are the big competitors from your perspective look i mean deep mind is i think the leading lab lab in the world alongside OpenAI. It's an exceptional team. Like, there's no question about it. And with phenomenal resources now, the TPU chip is pretty incredible. Amazon and Meta? I would say, you know, sort of,

I would say Amazon has done well with their new chip, this thing called Tranium, which is pretty cheap, it seems, and has performed quite well and was used to train Claude, Anthropix AI, recently. So they've actually been very impressive, I think. And Meta, which I think is the dark horse here, right? A lot of people feel.

Yeah, I mean, they've done a good job of pivoting away from the pivot, right? And kind of getting into the AI first company. Wait, you mean the metaverse is not where things are going? Apparently, it doesn't seem to be the thing. What? Who'd have thought? I don't know.

I did. You did. Yes, I did. I thought this is fucking ridiculous. I keep thinking they just don't want to die someday. That's what they're trying to replicate themselves digitally. But whatever. But how do you

look at their efforts? I mean, they're the ones open sourcing, right? That's correct. It's kind of interesting to see that they've kind of been playing, because they've been playing from behind, I think their view is like let's, you know, try and galvanize the community

and I think that's going to work out pretty well for them because then they've got a whole ton of open source developers building on their infrastructure. Right. And, you know, we'll see. Which they're used to. They used to be, remember, third party. You know, they're used to that kind of relationship.

So you've pushed back on the idea that chips are the new oil or gold, which most people, you know, now Jensen Huang is apparently a rock star. I remember when he was dull. But yesterday, and not as rich, but yesterday Microsoft and BlackRock announced they're planning a $30 billion fund to develop data centers and power plants for AI. Obviously, we'll get to energy in a second.

Microsoft has predicted it will spend $50 billion this year on capital expense. Wow. I mean, it was high before, but this is astonishing, primarily for data centers. It certainly feels like a gold rush and a lot of picks and shovels. I mean, it was high before, but this is astonishing, primarily for data centers. It certainly feels like a gold rush and a lot of picks and shovels. I mean, it is, right? I mean, the new, like, intelligence, which is the thing that has made us valuable as a species, made us creative,

the ability to accurately predict in a whole range of different environments and produce creative, intelligent things, in a whole range of different environments and produce creative, intelligent things. That is now getting increasingly distilled into an algorithmic construct, which happens to be stored and managed in these huge computers. So, you know, it's the substrate that enables everything else. So it's, of course, there's no changing that. That's going to continue. But at some point, very much, I remember when Cisco was that, was NVIDIA, essentially, in the early Internet age.

Does that what happens to that? It becomes a commodity and then everybody has it? Or what, from your perspective, what happens? Well, the strange thing is that the exponentials are both going up and down. What I mean by that is that, like, the cost of training the very best is higher than it's ever been. training the very best is higher than it's ever been. And yet you can now get GPT-3 entirely for free that runs on a mobile phone. In fact, you can train it from scratch, you know, on like basically a household level system. And that's only like three years ago, right? So that dynamic is almost certainly going to continue. So if you roll that forward to 2030, then you've got real value,

like real capability that is fundamentally free, or if not zero marginal cost and available to everybody. And that's what's really fundamentally different to different waves, past waves of technology. And then the services start, the services that everyone will be paying for, where the business, I know now OpenAI is doing pretty well with subscriptions at this point, but eventually it'll be services. Yeah, and I think those service models are likely to be, you know, impact driven. So like once you're able to like prove some value in a workplace or when your AI has really, you know, up-leveled you in some way or given you a skill or, you know, sort of managed some complex task in your personal life, you know, after you've observed the value, then you'll basically pay it a cut. It's not just chips. You need content. And Grok is being trained on content for X, right? Which

depends on whether it's going to be good. No one really gave permission for Elon to do that necessarily, although it probably is somewhere deep in one of the terms of service. OpenAI and other LMs were originally trained by scraping the Internet, not a new fresh thing for tech companies to scrape things. A bunch of lawsuits are pending. Now they're paying third parties for their data. Going forward, what do you think AI should be trained on to avoid biased disinformation and how closely should the inputs be monitored and paid for? Yeah. So I think one of the things that's become clear

in the last three or four years is that as the models get larger, they actually get easier to control. Right. So some people feared the inverse would happen, is that they would get bigger and that they would get sort of more dangerous or more prone to hallucination or more full of bias.

And as they've got larger, they in fact do a better job of following the behavior policy that you put in the prompt or in the fine-tuning process. And that's actually very good news because it means that although your underlying data is inherently full of all kinds of biases, I mean, if it's trained on X or it's trained on the open web, it's going to just internalize, you know, information from the entire world.

Yeah, garbage in, garbage out. Right, right, but not garbage out. I think that's the key thing is that, in fact, you can... You can put garbage in and a beautiful cake comes out or what? Well, with a little tuning, yes. You know, it basically gets better at following instructions. And that's good for safety. It's good for product making. It makes them reliable and consistent. They don't do random things as much. And in a way, you know, we want them to hallucinate, right? Like we want them to be creative. They're not a

one-for-one relational database like old school technology. Sure, sure. So creative in that they think for themselves. I mean, not in the human thinking, because I think that's difficult for people, but thinking in some fashion, correct? I mean, they do kind of think. The variation in their outputs, the fact that they don't reproduce precisely the same thing every time is a strength. That's kind of what we want from these models. It's just they have to do so within some bounds of acceptable behavior. And that's what we craft when we're doing sort of prompt design or prompt iteration. We'll be back in a minute. Support for this show is brought to you by Nissan Kicks. It's never too late to try new things, and it's never too late to reinvent yourself.

The all-new reimagined Nissan Kicks is the city-sized crossover vehicle that's been completely revamped for urban adventure. From the design and styling to the performance, all the way to features like the Bose Personal Plus sound system, you can get closer to everything you love about city life in the all-new, reimagined Nissan Kicks. Learn more at www.nissanusa.com slash 2025 dash kicks. Available feature, Bose is a registered trademark of the Bose Corporation. dash kicks. Available feature, Bose is a registered trademark of the Bose Corporation.

Support for this show comes from Constant Contact. You know what's not easy? Marketing. And when you're starting your small business, while you're so focused on the day-to-day, the personnel, and the finances, marketing is the last thing on your mind. But if customers don't know about you, the rest of it doesn't really matter. Luckily, there's Constant Contact. Constant Contact's award-winning marketing platform can help your businesses stand out, stay top of mind, and see big results. Sell more, raise more, and build more genuine relationships with your audience through a suite of digital marketing tools made to fast track your growth. With Constant Contact, you can get email marketing that helps you create and send the perfect email to every customer.

And create, promote, and manage your events with ease, all in one place. Get all the automation, integration, and reporting tools that get your marketing running seamlessly, all backed by Constant Contact's expert live customer support. Ready, set, grow. Go to ConstantContact.ca and start your free trial today. Go to ConstantContact.ca for your free trial. ConstantContact.ca Do you feel like your leads never lead anywhere? And you're making content that no one sees?

And it takes forever to build a campaign? Well, that's why we built HubSpot. It's an AI-powered customer platform that builds campaigns for you, tells you which leads are worth knowing, and makes writing blogs, creating videos, and posting on social a breeze. So now, it's easier than ever to be a marketer. Get started at HubSpot.com slash marketers. Going back to data centers and energy, I'd love to talk about this for a minute. You've talked about the benefits of AI, including solving some of the world's biggest issues,

like the climate crisis, except it's causing the climate crisis. The Guardian just came out with an analysis indicating there's a significant gap between the official numbers and the actual emissions. For example, Microsoft's in-house data center emissions were at least 21 times higher than the company reported. You know, this could be a climate disaster. Now, many, including Sam Altman, are working on alternative energy. this could be a climate disaster. Now, many, including Sam Altman, are working on alternative energy. I think Larry Ellison, the expert in nuclear fission, just discussed creating nuclear

plants, which Larry Ellison and nuclear plants are not two words I would like to see together ever, but here we are. So let's talk about the concerns. You have to be responsible in your energy use if you're going to cosplay that you care about the environment, which Microsoft and Apple and many other companies have done. You should be concerned, by the way. I do think it's important to point out, like, Microsoft is on a path to being fully renewable. Right. They pledged carbon neutral, right? Right, right. And in fact, it'll be neutral and it will be net negative in some few years to come, right? So I don't think any of

those commitments are going to get unpicked here. You know, I sit in some of the meetings, you know, every week and I see huge investment in fully renewable energy across the board. So I don't think that's going to change. So you all have to get in the energy business or in the renewable energy business. I mean, in some ways, the company is already in the infrastructure business. This is buying vast amounts of land, building huge, huge data centers. And Google has done this with Thermo, with all kinds of things. I recall there were a lot of kooky schemes, but some very smart ones around energy of the server farms, correct? Yeah. I mean, I actually worked on that at Google in 2017, 2018. And, you know, we have the most efficient data centers in the world at Microsoft and Google.

We measure this thing called PUE, power usage efficiency. So how much of the energy that goes into the data center gets used to actually run the servers, how much is wasted. And that got down to like 1.1, 1.2. So you think between crypto and this, which is a huge energy. Well, the crypto guys weren't committed to using renewables. No, they were not. So that's pretty different.

Right. But when you're committed, what does that mean? And what are the power needs in the midst of a climate crisis, a very obvious climate crisis? Yeah. I mean, well, I mean, I think all the energy that we consume,

almost all is fully renewable. In a few years time, the entire supply chain is going to be fully renewable. And I think by 2030, it's going to be net negative. Including these developers? Including all the AI, everything that the company does. Is there one kind of energy you're more focused on? I think fusion is a little way away. I mean, Microsoft does have multiple bets in the fusion space, investments in the fusion space. And where do you see it right now? I think that for a long time to come, we're going to be using hydro and solar.

There'll be some nuclear in the mix. But I think fusion is not going to be a relevant part of the energy mix until post-2030. Nuclear? Nuclear to some extent. We already used some nuclear. You know, someone named Bill Gates told me that renewables aren't going to cut it. In an interview, he talked a lot about the idea that he's a big nuclear person. When you have discussions like that, ultimately there's going to be a choice, correct, of where you're going to do this? Look, I think solar has come down exponentially in the last decade. So when the costs come down, there's a natural market proliferation of technology

because it's in our rational interest. What does government have to do here to push this? Well, the tariffs with China are not going to help, right? I mean, the fundamental reality is that China produces, you know, the vast majority of the world's solar voltaic cells at an amazing price. And the last thing we should be doing is considering putting tariffs on those

because we want them to spread around the world. Yeah, I've got some news for you. One of the people who's running wants to put tariffs on everything. I've never heard of him. Yeah, yeah, 100%. It's crazy.

You think it'll hinder economic growth. I mean, there's no question about it. That doesn't make any sense. Yeah, so does every economist, apparently. Speaking of which, Karen Howe of The Atlantic recently wrote, Microsoft marketing AI technology to big fossil fuel companies

like ExxonMobil and Chevron helped them find and develop new oil and gas reserves and maximize production. Big tech, big oil together is, how do you feel about that? I mean, do you want to talk about the FTC then? Okay. That's next. Spin the wheel. What do we have next? Well, FTC is sitting right here. Look, I think they have made big commitment. The oil companies have made big commitments to transition,

and they need to be held accountable, both by us. We're a partner of them, and everybody else is too. We're stakeholders in their future. And I want to see them move much faster, of course. They should be adopting 100% renewable commitments like us. Yeah, BP, Beyond Petroleum. I always love that. What a gall.

Anyway, a couple more things. The FTC has launched an antitrust probe. The government is, there's lots of probes, not just Microsoft. Google just got, just lost its case. Obviously, there's another one coming. But there's, around this particular deal, let me see if I got this right.

You and Inflection co-founder, it's Karen Simonian and Reed, yeah, chief scientists and Inflection both left for Microsoft. 60 out of 70 of your Inflection employees left with you. Microsoft paid Inflection $650 million as part of a licensing thing. Microsoft is claiming it wasn't an acquisition. My partner, Scott Gallery, claims it is. I mean, it wasn't an acquisition. My partner, Scott Gallery, claims it is. The FTC is investigating. Talk a little bit about this.

How is this not, is this an acqui-hire model or an acquisition? No, I mean, as I said earlier, like the, you know, we got to a point where it was pretty clear that we couldn't win the consumer battle. Right, right. But the company, so Reid Hoffman, our third co-founder, and he's still running Inflection and they're pursuing, you know, the enterprise mission. They actually are now

using other open source pre-trained models, fine tuning and adapting them for a whole bunch of other businesses and pursuing B2B. I'm not a B2B person. I didn't want to do that. And that's why we took up

the Microsoft offer. I mean, you know, it's also worth adding both the, you know, the English regulator, the CMA and the Europeans briefly looked at the case and then dismissed it and aren't taking it anywhere. So both of those have been given green light. And the FTC, because, you know, FTC is still in process. I mean, they're moving a little slower, but I mean, Europe and the UK have come to that conclusion. So the model, it's not, others are doing it, Character AI went to Google, Adept AI struck a deal with Amazon,

all without FTC approval. There is a move of the consolidation of power that for the first time, I am seeing regulators, including conservatives, there's conservatives who like Lena Kahn, that includes J.D. Vance and Matt Gaetz and others. There's unusual bedfellows happening in Washington around that idea. Is that good to have the government looking at this from your perspective? I think it's great. I think it's great.

Like, I think I've always said, I think we need a set of regulators that deeply understand technology, are very close to the companies, and really understand how we operate in order to hold us all accountable. And we need that healthy conversation and that feedback. I always welcome it. Whenever I travel, I always make time to go see regulators and the politicians.

Do you sense a more intense government on these things across the world? Because they did miss out on the first one. They didn't do any regulation on the first iteration of the internet. Do you sense a different tone from government? You see Pavel Durev arrested in France. These are all different things, but these are all actions

that are of a more aggressive government stance towards the impact of tech on society. I think we've got to just recognize that governments will get it wrong sometimes. Regulators will make mistakes. And that doesn't mean that they should slow down, that we should be forgiving of them, just as we should be forgiving of companies and, you know, every other organization, if they sincerely made a mistake and need to rein it in. Because what we need is more rapid experimentation among the regulators, rather than saying they have to get

it right every single time. That just leads to, you know, stasis, and they end up taking far too long to push these cases through. A lot of the, you know, European regulatory cases have taken a decade, whether on tax or on monopoly markets or whatever. And, you know, so I think that that is, you know, these are very complicated issues, so it does take time. But at the same time, we need them to move fast, right? This is a moment where the technology is changing super quickly. And, you know, we want people to be involved in the process. So do you welcome it from states doing that? California has been particularly aggressive. Gavin Newsom just signed a bunch of things.

I think there's a lot of sensible things in the privacy bill that was signed a few years ago. And I think states should play the same role, like active, engaged, in the public interest. All right, I want to end by talking, then we'll get a couple of questions from the audience. Before we go, zoom out a bit and get your take on the future of AI. You've advocated in the past for more transparency, independent third-party red teams doing ethics reviews, which is a great idea, preventing recursive self-improvement so AIs aren't learning from themselves. Do you think these are essential? And what is the most important thing that has to be done in the short term to make sure we don't drown in this tsunami of change if we don't know where it's

going, right? What do you think needs to be done? What's the most important thing for government and companies to do in the short term? Well, one of the things that's already happening at the UK AI Safety Institute is that they've embedded UK AI Safety Institute people inside of the companies that are really like tracking how big the models are, what capabilities they have, you know, and what the roadmaps. What's in them. Yeah, inside the companies, right?

So there's like plenty of detailed interactions between the four or five big companies and that safety institute, which was also backed by the Biden administration last year at the big White House thing. So, you know, I think it's headed in the right direction. This is a kind of unprecedented level of scrutiny and

collaboration before there has been any material harm, right? And everybody is learning the lessons of Web 2 and, you know, the lessons of the cigarette lobby and oil and so on and so forth. And I think there's a genuine desire to try to get it right. It doesn't obviously mean that we aren't going to make all kinds of mistakes, but I do think that it feels slightly different this time around. It's much more proactive when these things don't really even work that well yet. So we're already seeing AI impact this election, and often in a strange way. Taylor Swift posted her endorsement for Vice President Kamala Harris after Trump shared an AI-generated image of her endorsing him. Nice move, Donald. Talk about what you thought about that because, you know, she called it out specifically. A lot of the things that get

called out are the negative impacts of AI as people go to the polls. Right now, it looks like maybe not as much as people had feared. What is your biggest fear in that as a citizen and as someone who's developing these technologies? I think that we are going to have photorealistic, super high quality, video generated impersonations. And that is going to destabilize things for a while. We have in the past had moments in technology where things get really uncertain and we struggle to adapt. And then generally speaking,

we always find a way of solving that problem. In this case, it will be coming to rely on content that is produced by a human and being able to cryptographically verify that it's been produced by that account or by that person. Therefore, rendering all of the other content at least open to cynical speculation about its source and validity.

And that will be a very big shift and has a lot of other consequences. But I do expect that we'll end up in that kind of regime in three or four years because it will just be zero marginal cost to produce perfect content. What is your scariest scenario and what's your best scenario with these generative technologies? My scariest scenario is that it just happens too quickly and just runs away from us. And I think that sometimes, you know, the consequences take a while to settle and for us to really observe.

You know, it's taken 80 years or 100 years for us to have a robust vehicle regulation regime. If you think about it, there's driver training and seatbelts and airbags and, you know, street lighting and vehicle emissions, and somehow they all kind of fit together. And I think this time around, it's not going to be like one agency. There's going to be a ton of different pressure testing on the system to figure out where it's weak and where it has these third degree effects. And as things are moving so quickly, I think it's going to be tough to kind of catch up to that.

But I think the upside is, you know, everybody is going to have an incredible personalized tutor in your pocket. Like it is going to be as smart as the very best educator that, you know, any of us have ever had. Not better than Tim Walls. But anyway, Coach, I know you're more of an optimist in tech than a pessimist. In April, you said that AI could be described as a new digital species.

From your perspective, if this digital species represents the best of us and not the worst, what does that look like? Because, you know, we finally solved these issues of racism and anti-Semitism. Wait a minute. No, we haven't these issues of racism and anti-Semitism. Wait a minute. No, we haven't, right? It just keeps coming back. I think the reason why I tried to introduce that metaphor was because I'm concerned with the idea of containment and the limitation of the downsides.

And so, you know, it's best to frame it in that potential so that we can think of it just as we think of one another. Like it will have agency, it will have creativity, it'll have memory. And so once you kind of apply that frame, you can then start poking at all the potential weaknesses and the strengths. And I think the aspirational strength is that, you know, these models won't need to be bound by the same limitations that we have as a species. They won't have guilt. They won't have shame. They won't have, unless we program into it, ego. They won't be chippy and anxious. You know, we will have, some people will choose to program some of those traits into them. Right. But once we overcome some of their weaknesses, which memory

and factuality and hallucinations and so on, you know, then it really has the potential to be the very best of us, right? And you don't think shame is part of that, being the best part of us? Because some people are shameless, really. Can have a place, but it could also, those emotions tend to manifest in pretty destructive ways sometimes. And I think the challenge that we all have is that as personality engineers, which is this new... Wow, that's another word. This is a new competency, right? We get to imbue these models with emotional intelligence,

with kindness, compassion, sensitivity, and that's the new craft.'re in like day dot of that experimentation. And I think that, you know, that's an amazing aspiration. If we could like put agents or AI companions or co-pilots with love and kindness into the world, that would be an amazing contribution. All right. So my last question then is, I just have one more quick one, is that if that's the case, personality engineers, shouldn't they look like the rest of us?

That's my issue is that they don't. But they don't. They don't. The personality engineer is an Elon Musk, not a particularly good engineer for personality. Personality disorder engineer, certainly. So, I know that's good. You can use it. How do we make sure that happens? Because, again, we're back to the same old, same old in a lot of ways. Yeah, I mean, we have a lot of work to do. I think that this is a space that needs to be owned and co-opted by everybody who is going to be a user, which is all of us. And I think that there's much more that we can do as a company that I often try to do to make it a welcoming and inviting place for everybody to be and to create and make. You know, we've had this problem in Silicon Valley

for decades where, you know, technologists end up being nerdy, typically white guys that make technology with our own blind spots imbued in them because we're solving problems for ourselves. We're about to change that because these technologies can now be programmed by anybody. And they're now entering the realm of emotional intelligence, not just utility. Previously, technology software has been about doing something more efficiently, getting a payment in, you know, connecting people in a marketplace. Whereas now we're actually creating experiences. They're emotional, they're lasting, meaningful relationships. And so that's a moment when we can like reset the

counter on who gets to be involved in making them. All right. Well, excuse me if I don't take your word for it. Mustafa Suleiman, thank you, everyone. Thank you. take your word for it. Mustafa Suleiman, thank you, everyone. Thank you. On with Kara Swisher is produced by Christian Castro-Russell,

Kateri Yochum, Jolie Myers, and Megan Burney. Special thanks to Kate Gallagher, Kate Furby, Kaylin Lynch, and Claire Hyman.

Our engineers are Rick Kwan, Fernando Arruda, and Aaliyah Jackson. And our theme music is by Trackademics. If you're already following the show, you are your own agent.

If not, you're still aspirational like a friend. Go wherever you listen to podcasts, search for On with Kara Swisher and hit follow. Thanks for listening to On with Kara Swisher from New York Magazine, the Vox Media Podcast Network, and us. We'll be back on Thursday with more. Reimagined Nissan Kicks is the city-sized crossover vehicle that's been completely revamped for urban adventure. From the design and styling to the performance, all the way to features like the Bose Personal Plus sound system,

you can get closer to everything you love about city life in the all-new, reimagined Nissan Kicks. Learn more at www.nissanusa.com slash 2025 dash kicks. Available feature. Bose is a registered trademark of the Bose Corporation. Support for this podcast comes from Stripe. Stripe is a payments and billing platform supporting millions of businesses around the world, including companies like Uber, BMW and DoorDash. Stripe has helped countless startups and established companies alike reach their growth targets, make progress on their missions and reach more customers globally. Thank you. management needs. You can learn how Stripe helps companies of all sizes make progress at Stripe.com. That's Stripe.com to learn more. Stripe. Make progress.