Prof G Markets - AI Is Dumber Than Investors Think — ft. Gary Marcus

Episode Date: June 26, 2026

Ed Elson is joined by Gary Marcus to discuss why he’s concerned about the fact that we’re all-in on AI. They explore why he argues generative AI is inherently unreliable, whether the concerns surrounding Anthropic's Mythos model are justified, how policymakers should approach AI regulation, and the biggest misconception about the technology that he believes needs to be corrected. Gary Marcus is a leading voice in artificial intelligence, author, and professor at NYU Stern.  Subscribe to the Prof G Markets Youtube Channel  Check out our latest Prof G Markets newsletter Follow Prof G Markets on Instagram Follow Ed on Instagram, X and Substack Follow Scott on Instagram Send us your questions or comments by emailing [e-mail] Learn more about your ad choices. Visit podcastchoices.com/adchoices

Support for the show comes from Odu. Running a business takes everything you've got, and a lot of the tools out there that are supposed to make your life easier just aren't great at talking to each other. And that means you end up having to toggle between a dozen different apps and services just to keep the lights on. Enough of that. Now there is Odu,

the all-in-one fully integrated platform that might actually help you get it all done. Thousands of businesses have made the switch, so why not you? Try O-Doo for free at O-Doo for free at Odo.com. That's O-D-O-O-O-com. Support for the show comes from Plod. If you're an executive, small business owner, project manager, journalist, or anyone responsible for important decisions, details matter. Conversations move fast, and it's easy for key contacts, follow-ups, and action items to slip

through the cracks once the meeting ends. Plod is an AI-powered note-taking system built around dedicated recording hardware. It captures conversations, transcribes them, and turns them into searchable transcripts summaries and action items, So you can focus on the discussion instead of worrying about capturing every detail yourself. Visit plod.a.ai slash markets to learn more and use that markets code to get up to 15% off. Support for this show comes from Odu. Running a business is hard enough. So why make it harder with a dozen different apps that don't talk to each other?

Introducing Odu. It's the only business software you'll ever need. It's an all-in-one fully integrated platform that makes your work easy. CRM, accounting, inventory, e-commerce, and more. And the best part, O-DU replaces multiple expensive platforms for a fraction of the cost. That's why over thousands of businesses have made the switch. So why not you? Try O-D-O-4-3 at O-D-O-O-D-com. That's O-D-O-O-O-O-com. Welcome to Prof G Markets. We've spent a lot of time talking about AI lately, from the Trump administration's export restrictions on Anthropics models to the

ongoing questions surrounding the economics of companies like Open AI and Anthropic. Taken together, these stories point to two fundamental questions. One, is the AI boom financially sustainable? And two, are we moving too quickly with a technology that we don't fully understand? Few people have been asking those questions longer than our next guest. Long before concerns about AI safety regulation and business models entered the mainstream, he was warning about the technology's limitations and challenging some of the industry's most ambitious claims. He has testified before the Senate on the risks posed by AI.

He's founded a machine learning company that was acquired by Uber and is now one of the field's most prominent skeptical voices. So here's our conversation with Gary Marcus, AI skeptic, author and professor at NYU Stern. Gary, thank you so much for joining me on the show today. You are one of the original critics of AI, and that's quite interesting because you're also, you work in AI. You started a machine learning company, which I think you could say is an AI company.

You've done a lot of AI research. You are sort of part of the AI world, but you have issues with it. Let's just start broad. What are your concerns? My concerns are we're all in on a particular technology that I think is inelegant, harmful, not where we should wind up and being abused by the people that are using it. So I want AI to succeed, but I think we wound up down this really dangerous path.

So you think about the Star Trek computer. You ask it a question, it gives you an answer that you can count on. Presumably, it's not done to sort of wreck society. It's done to help people. Well, what we actually have is everybody running around with LOMs, which are inherently unreliable, they're unpredictable, they can't be aligned to human values, and they're being run by companies that don't seem to really give a shit about the consequences for what they're building for society. It's like a nightmare for those of us who have worked in AI to suddenly see what we're building

be used in so many bad ways and with people really not caring. You know, we should want a more reliable technology that we can really count on that is compatible with humanity. You know, five years ago, that didn't seem out of the question. Five years ago, the field was healthy. It was considering lots of different things. It wasn't driven so much by money, but by intellectual curiosity. How do you make a machine that's intelligent? And everything changed when people started to realize that there might be money to be made.

It's still not clear that there actually is money to be made, by the way, right? and I'm sure we'll get into that because we have very similar views about that. But the thought, the scent of money, possibly misguided sense of money, really changed how the field grew. And it's also a technical thing. Transformers are interesting and people got into them. But fundamentally, I think it's the scent of money really changed how people built AI, how they thought about it, what they wanted to do with it, who was running it.

I think a lot of grifters came in that don't even necessarily, they have a technical understanding of the questions and do a lot of lying and hyping about what their things might actually do. And it's just really been unpleasant for the last several years being honest about it. And it's not because I don't want AI to succeed. I still think that there's a chance

that AI could help a lot in medicine, that it could help with all kinds of technologies. Like, I would still like to see AI succeed, but not on the path that we are right now. This is just not a good path. You recently wrote, you said, quote, generative AI has been inherently unreliable from the start, none of the problems that I warned about over the last half decade have been

properly solved. There's the financial question, which we will get into. But then there's also the question of the technology itself. What do you see as the problem? What makes Gen AI inherently unreliable? What is the path that you are worried about this technology going down? The technical problem is that large language models, fewer large language models, are basically next token predictors. That's what they do. That is literally how they are built, is to predict in a sequence of words or other kinds of tokens what might come next. And that's an interesting thing to do. It's part of what humans do is we do some prediction.

But it's not all of what cognition is, right? Cognition, right, intelligence is about cognition, about understanding things and so forth. has many different components to it. And they're just not really built into LLMs. And so LLMs basically fake everything else. And we can talk about some complications, people are building in harnesses, and we can go there.

But let's just talk about pure LOMs. What they do is predict the next token. And if you train them on the entire internet, which is what people, in fact, do, they can make a pretty good approximation of human beings and how they talk and so forth. But that approximation is very superficial. It's very data-dependent.

And when you push them outside of the regime in which they've been trained, they will do really stupid things. So, like, a couple years ago, there were all these examples of so-called river-crossing problems. Like, you have a man and a goat and a woman, and they have to go across the... And these systems would say the most absurd things in response to those problems.

It got so embarrassing that anthropic built-in river-crossing problems into their system prompts to try to keep the systems from making these embarrassing errors. And what the embarrassing errors revealed is the systems are not really reasoning about things like a man or a river or a boat or what it means to go across the other side. They're just trying to kind of glom the words together that they have seen. I mean, you know, the technical details are a little bit complicated, but to a first approximation, what they are doing is just stringing these words together. There are other ways to think about building intelligence. So you might start, for example, with a database, who did what to whom, when, and where.

If you actually did that, if you started with that, you would not have all these crazy hallucinations. And so here we are, you know, in 2006, I started writing about LLMs in 2019. And I said, they don't have stable models of the world. You can't trust on them. And everybody said, Gary, Gary, Gary, we're just going to add more data. All these problems are going to go away. Hallucinations are going to go away. Mustafa Soleiman, who's the CEO of AI.

or whatever his title is, and Microsoft said, you know, they're going to go away in a few months. This was in 2023. I think I offered him a bet, and he kind of walked back what it was. Reed Hoffman said he would bet any amount of money that hallucinations would go away in a few months. This was 2003. I said, I'm over here. How about $100,000? He never got back to me.

But here we are in 2006, and hallucinations have not gone away. And it's because the core of Next Token prediction does not allow you to address. that problem. You have to add something else, and something else, you know, rarely works all that well. It sort of works a little bit. You know, I just saw a study yesterday showing there's a new benchmarking, I think it's called Halloo Hard, Hallucination Hard, and all the systems are still making errors on this. None of this has gone away. It's hard for people who are not trained in cognitive science and artificial intelligence to understand when they play with these systems that they don't think like human beings,

that they're really operating over different principles, because they are built to mimic human beings, and human beings are not built to distinguish AI systems that work differently from themselves from actual humans. So, like, we have a lot of evolutionary machinery to find fast things that are moving that might be snakes or bugs or lions. We have nothing built into our brain

to really help us think about the nature of intelligence. And so people are very easily fooled. we've actually known that for a long time. You've known it for 60 years, right? Eliza was the first example of an AI system that could fool an average person into thinking it was much more intelligent than it was. Eliza behaved as a psychiatrist,

and he just did simple keyword matching. So you say, you know, relationship, and it asks you to tell you more about that relationship or whatever, just by matching keywords, not understanding anything. So Weisenbaum wrote about this in the 60s, how we are vulnerable to over-attributing, is the technical term. intelligence to machines.

And, you know, that was a curiosity, I guess, when he wrote about that in the 60s. But now that is the whole world, right? The entire economy, this is, you know, where our shared interest is, I suppose, the entire economy is based right now is being, is hinging on over attribution of intelligence to these machines, right? You have people betting trillions of dollars that these machines are intelligent. in ways that they aren't actually, because those people placing the trillion dollar bets

don't have enough cognitive science background to know the right test in order to evaluate intelligence. And then we have, like, government policies built around these things or, you know, considered around these things. The entire world is over-tributing intelligence to LLMs. It's not that LLMs can't do anything. Like, they're great for auto-complete for the purposes of computer coding,

and they're great for certain kinds of brainstorming and so forth. but their intelligence is still limited, and we probably need a completely different approach. You know, I like to have a metaphor of climbing mountains, right? And you could get to the peak of one mountain and think, well, I must be close to the top, but actually you might not be, right? If it's a mountain range and there's a whole bunch of different peaks, you might be at the peak of one in order to get to the tallest peak

you might have to actually go back down the valley. And that's what we need to do. We actually need to give up some of the progress that we've made in order to come up with. with new ideas, but everybody's obsessed with one idea. You know, they're obsessed with the large language model that actually has these problems of, and we didn't even get into, but bias and unreliability, et cetera. But people are so addicted to the one thing that they're, you know, they're all in on that.

And, you know, we'll get to the economics soon, I suppose. But part of the economic problem hinges from that, that everybody is using the same solution. If you had a healthy ecosystem, you might have 100 different companies trying 100 different approaches, And you could say, let the best one win. But we have basically 100 companies, maybe not 100, but a dozen companies doing exactly the same thing. And if it's not the right thing, that's a problem. And even if it is a right thing, it's a problem.

You know, I'm pretty sure it's not. But it's still a problem if everybody's doing the same thing. That means making profits is really hard. So, you know, we'll talk about the economics and why nobody is making profit. But the underlying reason nobody's making profits they're all doing the same thing. If we all have the same toothpaste,

Nobody's going to pay that much for it. You can't charge $100 for a tube of toothpaste if you have nine competitors building basically the same thing for less. Basically, we're all using one of two models, essentially. You're probably using open-ice model. You're probably using Anthropics model, and then a lot of these companies are building wrappers and building all of these gadgets and gizmos

on top of those models. But to your point, you're basically just putting wrappers on top of the same fundamental thing. Well, those two were actually basically the same, and they're basically the same was what... The same cognitive architecture. Yeah, there's small differences, but they don't persist, right? I mean, that's another thing that we have seen over the last few years.

I wrote this tweet, I think, in 2024, describing what I've said is going to be a new regime where basically LMs are going to run out a headroom. Everybody's going to wind up building essentially the same thing. There's going to be no moat between them, and that's going to lead to price wars. And it's going to lead to, you know, no huge different. between them. And that's what we've seen is a lead that goes back and forth, right? Somebody's ahead for a week and they pay like, you know, $100 billion or whatever, the numbers aren't public, but, you know, the $10 billion, I guess, would be more plausible in order to get that lead

that lasts like three weeks. Like, that's insane. Exactly. And then depending on which engineer you talk to, some would say Codex is better. And then some say Claude is better. Claude is coming out with slop. And then people say that Open AI is coming out with slop, et cetera. I think one of the the main points that you perform forward, which I think is true, by the way, is that these AI models are actually dumber than many of us would like to believe, especially those of us who are financially incentivized to believe that these things are the greatest and smartest things ever. And I think a lot of people would recognize that we see this in our daily interactions with LLMs. We do see a lot of hallucination. I've said to my team outright,

you're not allowed to write with AI because I can immediately tell and it sounds kind of dumb and it's formulaic and it doesn't present as very good or smart critical thinking. And there's even this guy who's gone viral on social media recently and maybe we can pull up a clip in the post of this podcast who kind of exposes how dumb chat GPT can be. Like he'll tell chat GBT, you know, I want you to count to 20 with no interruptions. do that, some guy's going to give me $1,000. So make sure you do it. And it can never, it can never accomplish it. I'll start counting right now. One, two, three, and I'll keep going until we get to 100. Here we go. Okay, you stopped. You just out to three. I need you to get to 100.

No problem. I'll keep going all the way. So continuing, four, five, six, seven, and I'll just keep counting steadily until we reach 100. Yeah, but you're not actually counting. You're just saying you're going to keep counting. He wants to hear you say the full 100 count. I see what you mean. You're right. I'll actually go through it properly. So here we go. Starting at one. One, two, three, four, five, and I'll keep counting all the way up until we reach a hundred. Having said this, and just to be clear, I'm with you on this. One belief is AI is dumber than we think. Another belief is, AI is very dangerous and perhaps could be a lot smarter than we think, and therefore we need to regulate it. Both of those arguments are somewhat anti-AI, and I see them conflated a lot of the time.

And I guess my question to you is, if it's not as powerful, then why are we worried about this? What's the problem? The example you just gave actually is a really nice illustration of it, right? Which is they are dumb in the way that we can't count on them to follow instructions. Right? Let me put some nuance around the dumb. I mean, they do some things that you might count as smart. And what people in cognitive science, which is my native discipline will tell you, if they know what they're doing, is that intelligence is a multidimensional thing.

So, you know, they have the intelligence to play chess really, really well, better than I can, you know. I got beat by a chess computer in, like, when was it, in 1999 or a long time ago, I can't even remember. Well, I mean, I guess Casparov got beat by the best one in 97. Yeah, I wouldn't beat yourself up about it. I played Casparo once, by the way. annihilated me while playing to one of the other people. Anyway, I'm not a great chess player, but the point is probably even earlier that I get beat.

But, you know, AI can play chess really well, can play Go really well. A GPS navigation system, a different kind of AI that can do navigation really well. But LLMs can't do a lot of things. So LLMs actually are not good chess players, as it turns out. They make illegal moves. They can't even follow the rules. And so their stupidity about rule following,

and you just gave a beautiful example of this, that's what you need to worry about, right? You know, the reason that we need to regulate them is because they don't reliably follow instructions. It's not that they can't do anything that you might characterize as intelligent. You could argue about your definitions of intelligence. So one definition would be that you can do essentially

any kind of problem given enough, you know, resources that you're adaptive and so forth. They're not very adaptive. But there's another definition of intelligence which is like, you know, can you play chess? Then sure, they can, right? Well, LLMs can't, but other kinds of AI systems do. Asteris here, by the way, there are different forms of AI.

You know, my beef is with generative AI, and that's mostly what we're talking about. Generative AI cannot follow instructions. Chess computers, you know, purpose-built chess computers, actually do follow the rules of chess. And I have, in some ways, less concerned about them. LLMs are terrible rule followers. That is one of their weakest points as an intelligence. You know, another rule would be don't make stuff up. Like, you know, you can tell an intern, like, don't, you know, write something if you can't fact check it.

Like, just don't, please. And if you do, I will fire you or I will sue you. Exactly. You're getting fired if you. Right. I mean, that's the other crazy thing about what's going on is like calculators never make mistakes, right? There was a scandal when the, what was it called, the Pentium 4, I think, made very, very rare math. mathematical errors. Huge scandal. They had to recall the chips and stuff like that. Somehow the

standards have fallen. Like, everybody knows LLMs make mistakes all the time and they're perfectly happy with it. I'm like, I wouldn't want an intern who does that. We'll be right back after the break. And if you're enjoying the show so far, send it to a friend and please follow us on YouTube and Spotify or wherever you get your podcasts. Support for the show comes from Odu. Running a business is hard enough. So why make it harder with a dozen different apps that don't talk to each other. One for sales, another for inventory, a separate one for accounting. Before you know it, you are drowning in software instead of growing your business.

This is where Odu comes in. Odu is the only business software you'll ever need. It's an all-in-one fully integrated platform that handles everything. CRM, accounting, inventory, e-commerce, HR and more. No more app overload. No more juggling logins. Just one seamless system that makes. work easier. And the best part, O-DU replaces multiple expensive platforms for a fraction of the

cost. It's built to grow with your business, whether you are just starting out or already scaling up. Plus, it is easy to use, customizable, and designed to streamline every process. So you can focus on what really matters, running your business. Thousands of businesses have made the switch, so why not you? Try O-D-O-4-free at O-D-O-D-com. That's O-D-O-O-O-O-O-com. Support for the show comes from BCX, the public ticker for private tech. For generations, American companies have moved the world forward through their ingenuity and determination. And for generations, everyday Americans could be a part of that journey through perhaps the greatest innovation of all, the U.S. stock market. It didn't matter whether you were a factory worker in Detroit or a farmer in Omaha.

Anyone can own a piece of the great American companies. But now, that's changed. Today, our most innovative companies are staying private rather than going public. The result is that everyday Americans are excluded from investing and getting left further behind while they select few reap all the benefits. Until now. Introducing VCX, the public ticker for private tech, now available wherever you buy stocks. VCX by Fundrise gives everyone the opportunity to invest in the next generation of innovation, including the companies leading the AI revolution, space exploration, defense tech, and more. Visit getvcx.com for more info. That's getvcx.com. Carefully consider the investment material before investing.

including objectives, risk, charges, and expenses. This and other information can be found in the fund's perspective at getbcx.com. This is a paid sponsorship. Support for the show comes from LinkedIn ads. There's no worse feeling than making a major investment in something, only to realize it didn't exactly live up to the hype, such as buying a nice piece of tech that ends up in storage collecting dust, or taking a business workshop where your main takeaway was little more than a few motivational words.

If you work in marketing, this can happen with ads. You optimize for the numbers that look great, impressions, reach, and reactions. But when they don't show revenue, well, that can turn into an unfund conversation with the CFO. LinkedIn has a word for that, bullspend. Reach the right buyers from LinkedIn ads and invest in what looks good to your CFO. According to the 26 Dream Data benchmark report, LinkedIn ads generated the highest ROAS of all major ad networks. It's 121%.

You can target by company, industry, job title, and more. It's time to cut the bull spend. Advertise on LinkedIn, the network that works for you. spend $250 in your first campaign on LinkedIn ads and get a $250 credit for the next one. Just go to LinkedIn.com slash Scott. That's LinkedIn.com slash Scott. Terms and conditions apply. We're back with ProfG Markets.

OpenAI was just subpoenaed by a group of attorneys general to investigate their models. And some of the things that they said that they are investigating here, one, how they handle consumer data, also health data, also deep learning models. but my favorite is that they are investigating model sycophancy. And it seems as though the concern from regulators is to your point, like, it's not just that these AI models are dumb and making mistakes. It's like that is something that we need to actually punish. We can't have the largest, I guess, information provider, one of them in the world, going out and putting false information out into the ether

with no culpability or no accountability. And I do think that that is an important thing for these AI companies to contend with, because at a certain point, if enough of us just decide, I can't trust these models anymore, they lie too much, they make things up, they say things that don't make sense, then eventually we're just going to stop using them. How do you think that plays out? I mean, we might or we might not stop using them. There should be consequences, right?

Sycophancy, by the way, is when they kiss your ass, right? Right. So that's a separate problem from lying, although it's a form of lying practice. Like when they tell you that your idea is the greatest idea ever and it's not actually that. That's what I mean. I'll say, is this right? They go, yeah, you're right. You're right. Even if the thing is totally wrong. And then I go to Google and learn that I'm wrong. But the model will tell me, no, you're the great, you're the best. So we're in this, I don't know, awkward space where they do some things that feel magical, right? So brainstorming for some people, I don't get that much out of them, but like I'm working in disciplines I know well. If you're working in an area that you don't know very well, it'll give you a few things to get started. It feels magical, all right?

So there are some things that are good about them, even though I'm not rattling them off personally. They're clearly good at writing code and so forth. But they come with these consequences too, right? They come with the consequences that they make stuff up, that they are so askees, that they lead people into delusions that's been documented a number of times and so forth. And so society has to make a decision. And the initial decision was, well, we'll just let it all ride. There's so much fun to play with that, who cares what, you know, the consequences are.

And what the subpoena, which is, I think, filed by New York State, but as part of a larger, I think, 46 states or something that are involved, is a statement that, no, we're not going to let all this ride. Like, you know, there are different theories about how to perceive. One would be if you cause all of these problems, you should be held responsible for all those problems. There should be financial penalties. There should be warnings, et cetera, et cetera. Another is, like, maybe you shouldn't distribute the product until you can fix these, right? And there are different ways to address it.

But the initial reaction was to just completely give a free ride to companies like Open AI and say, hey, these are great. And now society is waking up and saying, hey, there are a lot of consequences. We have, you know, suicides that seem to be tied to these things, and we have the delusions. You know, we're destroying the educational system because the students are using these things and we're ruining critical thinking skills. And so, you know, what the companies want to do is this famous phrase, I actually tried to find the origins, but it's so old I couldn't find. But is to privatize the gains and socialize the costs, right? They want to make whole society accept the cost while they get rich. And what we have seen in the last 12 months, I would say, is a real sea change.

Right? I wrote a book in 2024 called Taming Silicon Valley. And I said, wake up everybody. The oligarchs are going to take over. They're going to screw us all. And nobody even read the book. I mean, not zero, but, you know, it got a little bit of attention. They will now.

I think you missed its moment. But, you know, it came out too soon. But two years later, like, this is what everybody is thinking about, right? is how are we going to rein this stuff in? And there's this huge backlash now. Some of it's about data centers, some of it's about employment. There's a lot of different reasons for it.

But society is no longer content to say, you can do whatever you want with us, right? That is what this Attorney General's thing. You know, the subpoena, if you look at it, is about like 15 different issues or something like that. It's very broad. They want to know what are the consequences of this stuff, and they want to know what the companies are going to do about it.

And they have looked around and seeing that there are a lot of negative consequences. You know, what I told the Senate when I was there in May 23 sitting next to Sam Altman, was you have a lot of risks here. And everything, I haven't gone back to the original remarks, but I believe that everything that I warned about is now, in fact, here and more real than it was. So I warned about cybercrime and I warned about misinformation. And I don't think I even knew about sycophants. I think there have been new ones that were introduced.

But basically, by and large, all of the... All of those things are worse now than they were three years ago. And now the public has woken up. The attorney generals have woken up. You know, we went through a period where I think the LLM companies thought they were going to get off scot-free. And now it doesn't look like that. And it shouldn't be that way, right? You know, another analogy would be people dumping chemicals, you know, factories dumping chemicals in the water.

We shouldn't let them do that. We should not socialize this cost of society. If you're going to dump chemicals in the water, you should do something about it. to be penalized for it, and so forth. I think we finally reached the point where people are recognizing that for AI. And by the way, important asteris, the Trump administration was completely opposed

to any AI regulation substantive of, in any form, except maybe about non-consensual deep-pake porn until about a month and a half ago, maybe a month ago. And now they have finally realized that what, you know, Mark Andreessen was telling them was nonsense, right? When Mark Andreessen was telling them

is you can't have AI and innovation at this. same time and regulation, so to have no regulation. Now we've entered a regime where the U.S. government is actually thinking, in a somewhat hamfisted way, but is actually thinking about how you regulate this stuff. And that is proper, right? We should have public debate about how to regulate AI. Somehow, Andreessen and a few others had, you know, so-called Overton windowed their way into making the debate about whether to have regulation at all. That was always going to be a stupid idea, but they pushed it for two years, two solid years. But now that's over. Now, you know, people

are realizing, like, hey, the government has put a regulation on Anthropic. That's not really fair. It should be across the board. And is it the right one. And so the Overton window has actually shifted back to which regulation is the right one, which is actually what the 2024 book was about. And now is the time to have that debate. Which is encouraging. And I would just point out, I think the reason that you didn't have that is because Mark Andreessen and Silicon Valley, they had their guy in the White House in David Sachs, and now he's out. And as you point out,

I wonder if that's the reason why we're starting to see some inklings of interest in regulation. I wouldn't accept that particular, or I think there's more nuance to that. Like, I mean, I think the thing that really did flip it was mythos was actually scary to some people in the government.

Until then, I think people in the, and I'm speculating from the outside, until then people in the government thought, eh, we don't need to regulate this stuff. fine. It'll be fine. It wasn't really fine, mind you. We were having delusions. They didn't care about that. We were having other problems. But when Mythos

came along, they're like, yeah, this is not really fine. This is actually a problem. And so I think that flipped it. Maybe that drove Sacks out. I don't know. But I don't think it's just about him. Sacks was definitely very opposed to regulation. His view is no longer in favor. But I think it is Mythos that kind

flipped it. Some of it was an overreaction to Mythos, but it's a good overreaction because it did make people realize you cannot just look at this stuff forever and say, oh, it's all going to be fine. It's not going to be fine. Even if Mythos is not quite as scary as I think some in the media have represented it to be, you know, some version of this really is going to be that scared. It's not that far away.

And we do need to figure out how we're going to handle it. So I think it's like we have, we've had two dress rehearsals now. We fucked them both up. The first one was initially we just let ChatGBT BT ride completely without any consideration for consequences of society that was bad. The second one is Mythos. Mythos is not actually the AI that is going to destroy the world that some people here. But the way that this one's been fucked up is it's been used as a political tool to destroy

a particular U.S. company. Like, that is not a thing, right? Like, I may not sound like an arch-capitalist, but I'm enough of a capitalist to think that, like, companies should mostly stand on their two feet and they should, you know, be allowed to prosper. and, you know, as long as they're not doing really bad things. And what's happening is, you know, the current administration is like, we don't like the way you dress, so we're going to screw you.

Like, that is not capitalism. That is really putting a thumb on a scale. That is also fucking up a dresser. Yeah. Just on the mythos model, this was going to be my next question, because a lot of the coverage that we have seen on mythos, this is Anthropics, new model that came out recently, was that it is so powerful that,

something's going to go wrong here. It was essentially the story that we've been hearing. And I mean, I've talked with people in the cybersecurity industry. They looked at this thing and they were worried about this. And we saw a lot of the cybersecurity stocks were plummeting. So I guess my question was, where do you stand on mythos? Because we know that your views on generative AI and their limits. but how do we foot that next to the fact that people are very scared about this thing that is supposedly extremely powerful?

So, I mean, I think one needs a nuanced view on mythos, I guess in a couple ways. One is it probably works partly like cloud code, which is to say it's not a pure generative AI model. There's actually a harness there. The harness is directing some of the cybersecurity investigations and so forth. So first bit of nuance is it is actually a little bit of a different architecture. The second thing is it is oversold, but it's also real in the sense that it can do a bunch of things that its predecessors could not. A lot of the things, if a system is well secured, are not going to be a problem. But the reality is that people have blown off cybersecurity for a long time, and there are a lot of systems that are not well secured.

So you're not going to use mythos to break into U.S. government things, and there's a footnote there where Mark Warner misunderstood something that blew up over the Internet and he just didn't get it right. You know, he got something secondhand from the NSA and he wasn't specialist in this, whatever. But, you know, you really can do some things in limited circumstances. Most of them are still kind of demonstrational rather than real world. it's not going to break into the cybersecurity of Google that's actually really set up well. But if somebody vibe codes something for their pub to track merchandise or something like that,

that's not going to be set up well. Like vibe coding does not set up security well, and that is going to be vulnerable. So there are lots of systems in the world that are vulnerable to mythos. The best ones are not. Maybe a hacker who knows what they're doing could use Methos as part of a larger thing

to attack some of those, but probably the best secured systems, banks and so forth, are not immediately vulnerable, but the weaker systems, and there are a bunch of weaker systems really are vulnerable. And so it really is a wake-up call

that we need to get our cybersecurity game in better order. And there's a footnote there, which is why is it not in better order? A lot of it has to do with stigma. Like there's stigma for mental illness so nobody talks about depression, but it's actually common or, you know, whatever.

There's stigma around cybersecurity. So people get hacked all the time. We don't have even good numbers on that. They pay ransoms. We don't have good numbers on that. And they let shit slide. They don't really know how to deal with it. And so that stuff is a mess.

And sooner or later a moment was going to come when it was going to be bad, and that moment has partly come. So we do have, I don't personally, but there are people in the world who have the knowledge for how to make a system sufficiently secure. And they're going to have a lot more business right now because most people have, you know, kind of deferred maintenance. Like you think of a metaphor of a house. Like most people don't deal with their roof until it's leaking, right? You know, you tell them you should, but they don't.

And cybersecurity is kind of that way. It's like, you know, you don't want to do it with this quarter. it's going to hurt your quarterly reports. And you don't know how bad your neighbor is because your neighbor actually did have to deal with it, but they didn't want to tell you about it because they were embarrassed. And so cybersecurity was a mess,

and Mythos really is making it worse. It's not, you know, it's not Lex Luthor's magical cyber hacking system or whatever that people are terrified, but it is real. Yeah, I like the house metaphor. It's almost like you'd rather buy a flat-screen TV than fix the roof.

it's a more fun and sexy thing to invest in. Exactly. There's been so much of that. Cybersecurity has really been secondary. This has changed that, and it's a good thing that it changed that. We'll be right back. And for even more markets content, sign up for our newsletter at profitymarkets.com. This spring, denim gets a softer, lighter update.

Introducing Old Navy's drapey denim wide leg, a new fit that moves with you. It's everything you want denim to feel like. for summer. Easy, breathable, and effortlessly cool. With a fit that creates natural movement and a wide leg that feels modern, not overwhelming. Plus, that signature, wait, for this price, moment. Old Navy's drapey denim wide leg. We're back with Prof G markets. Just on policy, you pointed out that before there was this ethos in Washington at the White House, of no policy whatsoever. Any form of regulation is a form of stifling innovation. We're going to do nothing. And actually, we're going to issue an executive order, which forces states to do nothing.

That was what we saw last year. You point out there's been a vibe shift here, some sort of sea change that's happened. Trump, at the beginning of this month, issued a new executive order, which would basically, will ask tech companies to give government the oversight that they believe they need over their new models before they're released to the public. It's still voluntary, right? And it's still narrow. So what they put in place is a step, right? Mostly it's a symbolic step in a certain way because what they have asked for now

is the companies will voluntarily provide their models so the government can do cybersecurity checks. What you really want is, first of all, for that to be mandatory. Right. Right. Meta actually hasn't agreed yet, right? They probably will because they can look really bad if they're the only one that doesn't. So you really want it to be mandatory, and you don't want it to be just about cybersecurity.

So thinking about all the things that New York State and really all the states are suing about or investigating about, like, I don't know, let's take sycophancy in delusions, right? You really want to investigate all of these companies. So sycophancy wasn't really a problem before a model called GPT-40. The sycophancy where it sucks up to you. It existed before, but it wasn't a serious problem. But 4O was much more sycophantic. I actually saw some data on this the other day.

And we believe that a lot of these cases of delusions were tied to 4O's increased sycophancy. So, you know, you want to be able to find that before it cuts out to the market, right? If you're releasing something to, well, chaty-PT now, or OpenAI is a billion customers. You know, if you have a billion customers, you have an impact on the world, there should be some kind of pre-screening, like with FDA approval, right? So, you know, with FDA approval, like, you have this drug, you know that it helps with cancer, but it also gives people heart attacks, right? And so you're like, well, you know, what are the cost benefits here? So you know that the cell helps people, but you also know it hurts a bunch of people.

And you should evaluate that before you release it at scale. Yeah, I mean, we saw a similar thing with the state of Florida, which sued Open AI for essentially playing a role in a mass shooting. their contention is that there was sufficient evidence from this clearly mentally ill child who was interacting with the model and talking about this, and they didn't do anything about it. And so that, I mean, we don't know the details exactly on what the conversation with the model actually looked like, but I could certainly see a world in which it's not doing enough to push back or to prevent further delusions on a psychiatric level.

So I think that we're starting to see real evidence there. I am interested that you're feeling optimistic about the executive order from Trump because, you know, I agree with you, it's notable and it's significant that they're doing something. But when I look at the something that they're doing, to me, it's, you can't even call it regulation. If they're just asking companies, hey, would you mind sending over some stuff, please, thanks?

To me, that's not regulation at all. And I'm starting to see that what little regulation we are seeing, what little policy we are seeing coming out of Washington, to me seems very stupid and very misguided. Like, I mean, the Trump executive order as one, plus his new suggestion, I'd be interested to hear what you think, but the suggestion that the U.S. government should start acquiring stakes in these AI companies, something that is now kind of proposed or backed by Bernie at the same time. I would also go to the data center moratorium. I don't think that that's a good idea to simply say, you're not allowed to build data centers anymore. I guess my point being, it seems that there's

almost no nuance whatsoever in Washington when it comes to AI regulation. So I'd be curious to hear what you think the right move is going forward and how that might play out. First of all, I very much agree with what you're saying overall, right? So what's in place is too weak, most of it. Some of it's too strong and crazy ways. There's no nuance in most of what's there. There are a few people, I think, have done some things that are nuanced, but they never make it out of committee.

So, I mean, if you actually look at the bills, people like Blumenthal and Holly, for example, who were at the proceedings that I spoke out at the Senate, have actually proposed some reasonable things, but they're stuck in committee. So it's not that nobody is paying attention. But, you know, the dynamics of money and power and all the lobbying and stuff means that most of the nuanced stuff doesn't get very far.

It does exist, but it's not getting far. I think that the – let's talk about the stake part. I think that that's kind of crazy. Partly because I thought we were going to talk about economics that we haven't got there. These companies are losing money, right? I mean, probably your audience has already heard. I think Ed Zetron was here recently. The companies are losing money.

This is a backdoor bailout, right? You know, Bernie has his own reasons for wanting to do this. But the reason Altman wants Trump to do this and has whispered in Trump's ear is because Altman knows he can't make ends meet, right? He is burning money at a massive pace. He's building the same technology as the other guys. He's lost ground. Open AI, I saw somebody argue recently, might be in fourth place.

Like, they were in first place by anybody's definition in 2023, right? But in 2006, they're not. They're burning money. They're losing ground. Like, of course they want anything they can do to prop them up, including taking money from the U.S. government. And so, like, I mean, first of all, the government should not be running these companies. Like, they should supervise them.

But we want, like, some arms lengthier, right? like I saw that G7 meeting with the, you know, the G7 leaders and the tech leaders and no scientists in the room, nobody from civil society, right? This is, you know, we don't want to crystallize that with government ownership of these companies and no independent oversight. And we don't want to burn U.S. taxpayer money on an industry that as far as I know has no real business model. I mean, Nvidia has a business model. They're selling shovels in the gold rush. if you want to take a stake in them that would make more sense. But, like, there is no sustainable business model yet established.

The best you can say is that for coding, they can actually bring in revenue, but it costs so much money to do the coding. It's not clear they can make the revenue. So you have these companies that basically never made a profit, and we're just going to give them money and let them burn the money, and we're going to take on that risk? No, let them stand on their own capitalism. And, you know, the government's job is to regulate them.

That would be one of the worst outcomes. I mean, and it's something that they talked about. I mean, the CFO literally said maybe we'd need some form of government backstop eventually. I don't know if those were the exact words, but it's been suggested by leadership at Open AI before. That was going to be on loan guarantees for data centers was what they floated, right, which is a version of a bailout. I mean, on the business model, what do you think happens here? because, yeah, we did have Ed Zittran on the podcast. I recently wrote an article going into just how profitable or unprofitable these companies are.

For Open AI, the answer is extremely. They lost, you know, $21 billion on an operating profit, unprofitable, lost $21 billion on an operating profitability basis last year. Right, so they're burning $2 billion a month, basically. That's right, just to operate that company. And that's, I mean, the real net loss was $39 billion, but there's some nuance there. But something we can say with a good amount of certainty is that on a day-to-day basis, when you add it up over the calendar year, opening eye is currently burning $21 billion. Every time you use their product, they lose money, right?

That's the better way to put it. Anthropic also loses money, but less money. My question to you, do they ever figure this out? Do they ever turn a profit? The way I've been thinking about it is like they need to thread a needle. There's so many things going against these companies that it is extremely unlikely that they're going to thread this needle. So let's think about some of the things that they need to deal with.

One is that they're building this big expensive technology, and they might get disintermediated by somebody who builds it better and more efficiently. So you should not need to train on the entire internet with a computer, you know, mass massive computer, unthinkably large computer, in order to do anything intelligent. Like, you didn't train on the entire internet, but you're a smart guy, right? You didn't need the whole internet. You run on like 20 watts of power. You have some pizza or some sushi or whatever.

You don't need to, you know, right? So one is they're just like insanely inefficient. If somebody else comes along with a more efficient thing, then they're all hosed. And like, you might not need all of these data centers if somebody comes up with a more efficient thing. then you have the problem that everybody is using the same secret formula. It's not just that they're all building toothpaste. They're all basically building the same toothpaste, right? And so, like, we're seeing this now, right?

There was this crazy, crazy period earlier in this year, the era of the token maxing, which lasted about a month. And in the era of token maxing, you had companies reward their employees for using as many tokens, as much AI as possible. They had, like, leaderboards. Amazon had a leaderboard. It doesn't actually make sense. I mean, what you really want to know is, are the results good?

And, you know, every study that's looked at productivity has shown they're not all that great. And so suddenly a lot of companies got worried, and they're not doing token maxing anymore. And in fact, this morning I saw a term for the first time, which was called the token apocalypse, right? The token apocalypse is that suddenly everybody's like, we shouldn't use so many tokens. And tokens that we should use, maybe we should use cut rate models that are not quite the best model. Maybe they come from China, but so what? We saved some money. Even Microsoft is saying,

maybe you should use deep-seek sometimes. Because Microsoft is like, nobody wants to pay these prices. We're gonna have to cut them somehow. So, you know, I was saying that you have to, you know, thread this gauntlet, right? So one thing is that people might make more efficient models with altogether different technologies. Another thing is that nobody can charge very much money

for tokens because it's just this price war, because everybody's building exactly the same thing. And, you know, there was a period of a month where people didn't care and they're like, you know, that's all right. I'll have another drink. I don't care how much it was because the companies were all you cannot eat buffets, but they've stopped that. So you have to solve that.

Then you have the reliability problems, right? Those still aren't solved. The hallucination problems still aren't solved. And so when companies try this stuff out, most of the experiments wind up with the results not being that great. Like there's been 10 studies now or something like that showing most customers are not finding return on productivity. So the customers may eventually say, this was fun while it lasted, but, you know, I'm not really getting the results. It doesn't really warrant this. I'll let somebody else figure it out.

The whole thing has been driven by Thomo. I don't want to be the guy who doesn't use AI when you're using AI and so you wipe me out. But if I try it for a year and a half or three years or whatever, it is still not really making a difference. either for me or you, I might say, fuck it. When it works better, I'll come back. But right now, not so much. Any of those things just wipes out a company like Anthropic or Open AI that already is, as you say, burning lots of money, right? And so if customers leave for any reason or somebody makes a better technology or somebody makes a cheaper version of the same thing that's almost as good, then you're in deep trouble. It's not even clear in the best case that any of these companies

have a good business model. I mean, they're not making profits. And, you know, it's just so delicate. Yeah. Do you believe that that will be the outcome for, say, an open AI? I've been warning for three years that I think open AI is going to be the we work of AI. And when I said that in, I think it was November of 2023, people looked at me like my head was screwed on backwards.

And they just did not believe that that was remotely possible. But now, you know, every other week I read somebody writing something. making the same analysis, like Sebastian Malaby in the New York Times. Like, it has gone from a crazy idea to an idea that a lot of people are having, right? The economics don't make sense. And what they kept doing is playing double or nothing with funding because they were burning so much money. So they would increase the valuation. They get somebody to write a bigger check.

But it's not clear who can write the check that they need next time. So now they're talking about IPO, but they have a problem with the IPO to raise, you know, the next round of funding, which is that Anthropics is basically the, same product for the same valuation, but they're doing better commercially, you know, burning less money. And like, opening eyes, reputation is declining, partly because I think Altman is a really untrustworthy individual. We don't need to go into that, but I've written about it a lot. And so, you know, a lot of people are leaning towards Anthropic. Anthropic is getting market check. So why would I put a trillion dollars in a company that is burning money that has a

competitor that is rising while they're falling that seems to be better run, maybe has a little bit better technical vision. Like, it just doesn't make sense. The argument would be why? Why? Because the technology is rapidly improving. The technology is a kind of technology, the likes of which we haven't seen. Everybody's technology is to the extent that you accept that it's rapidly improving, which I think is actually controversial. But, you know, it's moving in some ways and not others. It's not actually improving on reliability and hallucinations and so forth. But in any case, in the ways in which it is improving, which are some of the ways you want, but not all,

the competitors all are, too. You have to think about the relative ranking and the cost. The relative ranking of Open AI is clearly declining by any reasonable measure. Less market share, less reputation, etc. And everybody else is catching up. It used to be people thought the Chinese models were a year behind. Now they think they're like four months behind or something like that.

And, you know, Anthropics, is ahead, Google is ad. There is no argument, no rational argument for buying a share of OpenAI at a trillion dollar valuation. There just isn't. I agree with that, by the way, I guess the part I wanted to clarify, the Ed Zitrans of the world, if there are more people who have his view, is that none of it's going to work. Open AI isn't going to work. Anthropics not going to work. the idea is that the costs to build this stuff are just too damn high, and the revenues for these products will not exceed the costs, ultimately over the long term. My view is that I think that there is a path to profitability for, basically for Anthropic is my view. I think that there's a world in which they can make it work.

In other words, there will be winners and there will be dramatic losers, and I would agree with you. I think that we believe that open-AI will be a huge loser. I guess the point I'd love for you to clarify, will they all lose or will there be winners? Is Gen AI itself doomed or is there a world in which they make it work not only in terms of making it useful, but also making it a profitable business that makes more money than it spends? I'm a little bit closer to you than to the other, Ed.

I don't know for sure. I think it's very much TBD. You know, I would certainly sooner take a bet on Anthropic than on OpenAI. I think that they're a sounder company in multiple ways. It is TBD whether this stuff can be made to be profitable. It's not completely out of the question. I think part of the question is like, can they find a niche? Like, is coding enough of a niche?

Well, not so far, right? Because coding is a $570 billion a year industry. They're not going to get all of it. You know, the people have fantasies, and they get all of it. And the costs are so high, you know, like, it's really hard to know the future in full detail. Maybe they can find enough of those niches and they can eat something out. They may never warrant the trillion dollar valuation that they're looking for, right? You know, there's an intermediate possibility, which is they do become a profitable company.

They figure out enough cost to make it over. But really, they're basically a company that makes like $20 billion a year on a pretty big capital outlay. And it's like, it's not really the best way to invest money, but they, they eat by. Like, maybe that's the intermediate position is like, they don't go out of business. They make a profit, but they were not really worth a trillion dollars. Investment, you could have spent that trillion dollars in a better way. Like, maybe that's the reality, which is kind of a little bit closer to you than to Ed Zittron. But, you know, I mean, maybe that's what it is.

We don't quite know. You know, there's another story where the only people who really make money off of this, aside from the chip companies or places like Google that already have the infrastructure and the distribution. They don't really need to make that much money on it. They just need to make sure they don't get this intermediated. There's a bunch of different possibilities. We don't know for sure.

Open AI is clearly the weak link in the chain. Anthropic is still in it. We don't really know if they'll make it or not. That would be my take. Yeah. No, I think that makes sense. Gary, you've been very generous with your time. Just before we end here,

what would be your final, message to people listening. Maybe they read about AI, they've heard about AI, they're thinking about it in their day-to-day lives, but what do you think people don't know enough about what's the myth that you would want to dispel? I think the myth right now is that generative AI is close to so-called AGI, artificial general intelligence, and it's going to solve all our problems. This is just not true. We're going to find some what we call domain-specific applications. coding is maybe the best one so far, where we can actually use these tools for something. But they're not magic. They're not all-purpose intelligence. And we need to make fundamental

discoveries before we get there. And we should ask ourselves, as a society, there's an idea of an explore versus exploit trade-off. And we're completely in the exploit LLMs rather than explore other options. China is less so. I think we're running a risk by going completely crazy into the LMs, which China is not doing, that we're going to get the ones left behind because we committed too early to the wrong technology. Like, you know, are we building a billion beta maxes where VHS might not be better, but it's cheaper, or maybe that's not the perfect analogy, but like where there's some other technology here that is actually the right one, when so blind to the LLMs or to anything really, the

alternatives to LLMs, are we making a mistake by doing that? I mean, look at China. They're making a lot of infrastructure bets on elements, but it's like I think it's maybe 20 cents on our dollar. That gives them room to play if there's something else developed. We're like, we're going to buy these companies and put our entire, you know, economy into it. Like, I think that's a mistake,

and we should be more like, how could we explore other approaches to intelligence? Which requires understanding intelligence as more than just looks good to me, and at a deeper level, learning some cognitive science and maybe reflecting more deeply, on what it is that we want to build and how we want to build our society around it.

Yeah, and what it means to be intelligent. Gary Marcus is a leading voice in artificial intelligence. He's a scientist, emeritus professor of psychology and neural science at NYU and an entrepreneur. He was the founder and CEO of Geometric Intelligence, acquired by Uber. He is also the author of six books, including his most recent work, Taming Silicon Valley, which anticipated the rise of tech oligarchs, his 23 U.S. Senate testimony next to Sam Altman was watched by millions. He is well known for his challenges to contemporary AI,

anticipating many of the current limitations decades in advance. Gary, we really appreciate your time. Thanks very much. This episode was produced by Claire Miller and Alison Weiss and engineered by Benjamin Spencer. Our video editor is Jorge Carty. Our research team is Dan Chalon, Isabella Kinsel, Kristen O'Donohue and Mia Silverio. Jake McPherson is our social producer.

Drew Burroughs is our technical director. and Catherine Dillon is our executive producer. Thank you for listening to ProfG Markets from ProfiMedia. If you liked what you heard, give us a follow and join us for a fresh take on markets on Monday. Hey, y'all, it's Kelly Clarkson with Wayfair. Ever order furniture online and wonder, what if? Like, what if it doesn't hold up?

That sofa was four days old. You should have ordered from Wayfair. With Wayfair, there's no what if. Just style you love and quality you can trust. Visit Wayfair.ca.cair, every style, every home.