The Current - Historian Yuval Noah Harari on the “unknown unknown” of AI

Episode Date: December 23, 2024

Historian Yuval Noah Harari says AI is the first technology that is not just a tool, but “an active agent” doing things we didn’t anticipate and might lose control over. The bestselling big thinker spoke to Matt Galloway in front of a live audience in Toronto this September about AI’s possible consequences, and why humans are smart enough to put a man on the moon but too stupid to achieve peace on Earth. Listen to the rest of the conversation here and here.

In 2017, it felt like drugs were everywhere in the news, so I started a podcast called On Drugs. We covered a lot of ground over two seasons, but there are still so many more stories to tell. I'm Jeff Turner, and I'm back with Season 3 of On Drugs. And this time, it's going to get personal. I don't know who Sober Jeff is. I don't even know if I like that guy.

On Drugs is available now wherever you get your podcasts. This is a CBC Podcast. Hello, I'm Matt Galloway, and this is The Current Podcast. If you've ever wondered why human beings are so smart that we can send men to the moon, but so stupid that we can't make peace on earth? Well, you have something in common with Yuval Noah Harari. It is a question that has consumed him, inspired a series of best-selling books,

and as you're going to hear, he thinks he has the answer. Matt Galloway spoke with Yuval Noah Harari in September as part of the Toronto International Festival of Authors. Harari's latest book is called Nexus, a brief history of information networks from the Stone Age to AI. And here is part of that conversation. This is a book of our time in many ways. It is about something that is arriving, something that is here, something that people believe will reshape the world, something that people are already terrified of.

It's also about information networks more broadly. And you are very specific in terms of defining that term. What do you mean when you say an information network? What to you is an information network? Well, the key thing about information is that information is not truth. As many people tend to think, especially in places like Silicon Valley. Most information in the world is fiction and fantasies and delusions and so forth. The main function of information is to connect a lot of individuals into a network,

an army, a church, a tribe, a state, a corporation, a market, they are connected by information. Sometimes the information that connects them is the truth, but not always. Very often it's easier to connect people with fiction. If you think, for instance, about images, portraits, what is the most common portrait in the world? What is the most famous face in history? It's the face of Jesus. There are billions and billions of portraits of Jesus created over the last 2,000 years in churches and cathedrals and private houses, and all of them are inauthentic.

None of them is true, because we have no idea how Jesus actually looked like. There is not a single portrait that survived from his own lifetime, if anybody made a portrait, and there is not a single word in the Bible which describes him. Not a single word in the Bible which describes him. Not a single word. There is one description of his clothes but not of the man himself. We don't know if he was tall or short, thin or fat,

if he had black hair or blonde hair, if he was bald, nothing. All the portraits are fictional and nevertheless, they have been important in creating one of the most influential and powerful networks in human history

why is it important that we think of that as a social phenomenon that information is a social issue because currently we will talk about social networks but this is not something new the way that information has been used, and you're kind of hinting at it there, but the way that information used over time

has been a social issue. Because we are a social species. Our power in the world doesn't come from our individual genius or our individual abilities. It comes of this very unique ability, really unprecedented in the history of the world, to connect unlimited numbers of strangers, individuals who don't know each other personally, into a network that can accomplish things, you know, from flying to the moon to deciphering the secrets of DNA,

waging wars, but also building hospitals. It's all done by these social networks of individuals, which in turn are based on information. And it's amazing to see through history how even seemingly small technological changes in information have their far-reaching social and political and economic consequences. If you think, for instance, about one of the first revolutions in information technology in history,

the invention of writing in ancient Mesopotamia, what is today Iraq, about 5,000 years ago, in technical terms, it was so simple. It was based on taking clay tablets, and clay is basically mud, so taking pieces of mud and taking a stick and imprinting signs in the mud to preserve information.

And these were the first written documents. And they created an immense change in human ability to organize themselves. And to take just one example, if you think about something like ownership, what does it mean to own something, to own a house, to own a field? So if you live in a preliterate society, a society without documents in the ancient Middle East or in ancient Canada, to own a field means that your neighbors agree that you own it. It's a community affair. They agree, yes, this field is yours, so they don't graze their goats or sheep there and they don't go to pick fruits there

without your permission. But since ownership is a community affair, it also limits your private property rights. For instance, you can't sell your field to a stranger, to somebody else, without the agreement of the neighbors, because ownership means the agreement of the neighbors. It also limits the ability of some distant king or dictator

to create a huge centralized kingdom, because to create such a kingdom, you need soldiers. To pay the soldiers you need taxes and to levite taxes you need information on who owns what. And nobody in the king can't remember who owns all the fields in all the villages in the region. So before writing you don't see any large kingdoms or empires anywhere in the world. And then you have these pieces of mud. And the mud changed everything. Suddenly, to own a field means not that the neighbors agree that this field is yours,

that there is a piece of dry mud which says that this is yours. And it sounds funny, but it's the same today. I'm not sure about the legal system in Canada, but in most places, to own a house or a field means that there is a piece of paper. Okay, so we advance from mud to paper. And in some places, it's even digital. But it's basically the same thing.

To own something means that there is a document that says this is yours, which on the one hand hand increases individual property rights, because now you can sell your field to someone, a complete stranger, without the agreement of your neighbors. Because to transfer ownership to that stranger, you just need to transfer that piece of mud or that piece of paper. Even if the neighbors don't agree that this field belongs to that person,

you can use the court and the police to enforce the claim. You also suddenly have the ability for a distant king to create these centralized authoritarian systems because the king creates an archives with lots and lots of these clay tablets or papers or whatever, and now the king knows who owns what over thousands of kilometres and creates a taxation system and pays soldiers and creates a kingdom.

So even such a small change in information technology completely revolutionises the way that you can construct human networks. And also it's very telling that it never has just one possible outcome. It often has contradictory outcomes, the same technology, for instance, strengthening both individual rights and the potential for authoritarian control at the same time. What is it that scares the hell out of you when it comes to this latest bit of information technology and artificial intelligence? Because we have seen a series of

escalations when it comes to how technology can be used and but there is something the way that you talk about it in this book there is something that is different about AI absolutely everything you see you know from the invention of writing and then you have the book and then you have the printing press and then you have radio and each of these information revolutions had its own consequences. But in essence, it was more of the same. AI is totally different. Because it's the first technology that can make decisions by itself. And that can create new ideas by itself. A clay tablet could be used to impose taxes.

But the clay tablet could not decide how much tax to take and who to tax. It was always a human being. The clay tablets were essential, but all the decisions were made by human beings. And the clay tablet could not invent a new tax. The same with the printing press, the same with radio. Radio cannot decide what to play. It's humans making the decisions. And radio cannot write a new symphony or a new political speech.

Only humans can. AI changes that. It's really, it's the first technology which is not a tool. It's an agent. And clay tablets, radio sets, they are tools in our hands. We use them. AI is in a way like us. It is an active agent doing things in the world

which we often didn't anticipate and might lose control over, which is why I think, you know, I don't think about AI as artificial intelligence. You call it alien intelligence. I think the better acronym is alien intelligence. What does that mean? Alien, not coming from outer space, of course. It's created here on Earth by us.

But artificial implies that this is an artifact. We create it. We control it. And with each passing year, AI is becoming less and less artificial and more alien in the sense that it makes decisions and invents ideas which are really different. Is it really that yet, though? I mean, it's not writing symphonies yet, is it? It's perhaps improving things. It's writing bad symphonies. It's writing bad symphonies. But this is the... I can't write even a bad symphony. You know, it's a huge thing.

We keep moving the signposts that, you know, a radio couldn't write a symphony. Now we have something that writes bad symphonies, and we are barely like 10 years into the AI revolution. We haven't seen anything yet. And I don't understand music very well, but I understand texts. And when I look at the texts being produced

by the very, very primitive AI of today, ChatGPT is still very primitive. It's just the first baby steps. I'm amazed that this is not some glorified autocomplete. It has an understanding of the semantic field of different words. It knows how to string not just sentences, but paragraphs and entire stories.

Again, you can say the stories are not good, or the essays are full of hallucinations and lies. And this is true, and this is also true of humans in many cases. But the mere fact that the AI can create a long text which makes sense. Again, you can like it, you can not like it, but it's a real story or it's a real essay making coherent arguments or a coherent narrative. And again, the key thing to understand is we haven't seen anything yet.

This is just the very beginning. In 2017, it felt like drugs were everywhere in the news. So I started a podcast called On Drugs. We covered a lot of ground over two seasons, but there are still so many more stories to tell. I'm Jeff Turner, and I'm back with season three of On Drugs. And this time, it's going to get personal. I don't know who Sober Jeff is. I don't even know if I like that guy. On Drugs is available now wherever you get your podcasts.

I spoke with Yoshua Bengio, who is known as one of the godfathers of AI, just in the last couple of weeks. The timeline that he talks about when it comes to AI taking control, being smarter than us, and the fact that there's not kind of a hidden switch or a plug that we can kind of yank to take the thing out of power. He talks about it in years, not in decades, but in years. Is that, you would nod, Is that kind of the scenario? Because the outlook that you portray in this book is, it's a pretty grim world ahead. Before we get into this, is it bad or good, the doomsday scenario, the wonderful promises,

first of all, let's understand what is happening. You know, when I wrote Homo Deus, and I started becoming interested in AI, it was 2015, 2016. So people like Nick Bostrom, they talked about this development of super intelligence, which will get out of our control and might basically take over the world and take over history. But this was seen as a kind of philosophical idea that might be relevant, you know, in centuries, in generations. And you look

at the pace that this thing accelerated over the last 10 years and now a lot of very serious people like Joshua Ben-Jor, like many others, they are really talking in terms of a few years by the end of the decade maybe.

Within 10 years. And I'm not in a position... My training is a medieval military historian. This is what I bring to the table. I'm not really a computer scientist. My expertise is to try to understand, if these people are right, what are the consequences for religion? what are the consequences for religion? What are the consequences for warfare?

What are the consequences for politics? But certainly, a few years ago, almost nobody was talking in those terms. And at present, there is no consensus, but a lot of extremely serious people, some of the brightest mind, the godfathers of AI, they are talking in those terms, just a few years. What are you most worried about when it comes to AI? Because it's an agent and not a tool, what worries me most is the kind of unknown unknowns. The things that by definition we cannot anticipate how it will if you can anticipate it it's not really an ai again there is now because everybody's now caught on to this oh ai is very important

there is a hype around it because of that it's becoming difficult to know what ai really is because everything is ai whenever they try to sell you something they tell you oh it's an ai these are ai shoes and these are ai socks and like they tend to sell you something, they tell you, oh, it's an AI. These are AI shoes. And these are AI socks. And they tend to sell you a coffee machine. Oh, it's an AI coffee machine. So let me have two minutes just to explain very briefly. How do you know if something is an AI?

Somebody tries to sell you a coffee machine and tells you this is an AI coffee machine. So if the coffee machine cannot teach itself anything new, it just does whatever it was pre-programmed to do, like it was pre-programmed to make you an espresso cup, you press the button, an espresso comes out, it's not an AI, it's an automatic machine.

However, if when you approach the machine, before you even press a single button, the machine tells you, I've been watching you and millions of other people, and based on everything I've learned about you and other members of your species, I predict that you would like an espresso, and I took the liberty to already prepare you a cup. Sounds pretty good. That's an AI. And it's really an AI if it then tells you, actually, I've invented a new drink, which I call Bespresso, which is, I think you would like it even better than espresso.

Here, try it out. So this is an AI. It changes, it learns by itself. Yes, it was originally created by human beings, like Joshua Bengio, but you can think about it like some kind of digital baby, that you create something which its main ability is to learn and develop by itself in directions that you can't anticipate and control. So when we think about the dangers of this thing, the dangers we can anticipate are in a way the lesser evil. Like it's easy to imagine a scenario when a government,

let's say some paranoid dictator somewhere, gives an AI system control over nuclear weapons, and then the AI does something that maybe even the dictator didn't expect, and which results in a nuclear war. Similarly, we can anticipate that a group of terrorists would use AI to create a new super virus and create a new super pandemic.

It's more difficult to anticipate scenarios like AI causing a financial crisis much bigger than 2007, 2008, but in many ways this is more likely. Finance is the ideal playground for AI because it's a purely informational field. It's just data. You know, when you ask yourself, why don't we have self-driving cars all over the place despite all the hype and promises? Because cars are messy.

They need to deal with traffic, with physical world, with potholes, with pedestrians, with cats, with dogs. It's difficult. Finance is just information. What happens if we give more and more control of the financial system to AIs, which invent not just new financial strategies, but new financial devices? The history of finance, again, it's out, basically. It's humans inventinging creating very imaginative things

like money in ancient Mesopotamia and then at a certain point they invent checks and credit and all kinds of ETFs and CDOs and whatever and the big financial crisis of 2007-8 it started when a few financial wizards

in Wall Street invented this new financial device, CDOs, collateralized debt obligations, I think it's called, which was so complicated that almost nobody understood it, certainly not the regulators. For a few years, everything looked wonderful. They made billions and billions of dollars, and then everything collapsed. And with hindsight, people realized, oh, we didn't regulate it properly, partly because almost nobody understood what it was. What happens now if we give AIs the power, or they take the power,

to start creating new financial devices, which for a few years create immense wealth. The markets were never better as in this new wonderful AI era. And after, say, five years, there is a huge financial crash, much, much worse than 2008, and not a single human being on earth has any idea what is happening because the A has created financial devices which are orders of magnitude more complicated

than what the human brain can comprehend. It's really beyond our capacity. That's something which you don't often see in Hollywood science fiction movies about AI. And then there are all the things that we just can't anticipate, because our brains are organic, our imagination is limited by organic biochemistry, and AI is not organic. It thinks in a fundamentally different way

than any organic entity. And what's out there that we can't anticipate, this in some ways is the most frightening thing. I mean, the question that is at the center of this book, which is at the center of much of what you've written, is how is it that human beings can be so smart and yet so stupid at the same time? Have you figured out the answer to that? What is wrong with us?

Yes. You have. Yeah. The problem is not with us. It's with our information. I mean, in many theologies and mythologies, when this question is raised, if humans are so wise,

why are we doing so many stupid and self-destructive things, then the basic theological or mythological answer is there is something faulty, there is a deep flow in human nature which makes us self-destructive and like we are evil in some way. And I don't think that's true. I think the vast majority of humans, of people, are good. They are compassionate. Yes, they have dark sides, but they also have very good sides.

The problem is not in our nature. It's in our information. And we became very powerful, not as individuals, but as these large networks of cooperation. And unfortunately, it's much easier to make people cooperate using fiction and fantasy and delusion than by using facts and truth. And to take an example, if you want to build an atom bomb, so you need to know some facts about nuclear physics, otherwise the bomb will not explode.

But you also need to get millions of people to cooperate on this project, to mine uranium, to build reactors, to feed all these millions of people. Like the Manhattan Project directly employed hundreds of thousands and indirectly millions more. Now, how do you get all these people to cooperate? If you just tell them the facts of thousands, and indirectly millions more. Now, how do you get all these people to cooperate

if you just tell them the fact of physics? E equals mc squared. So what? This doesn't motivate, this doesn't inspire. And unfortunately, history shows us that the easiest way to inspire people is with fantasies and mythologies. So in the end, the atom bombs, the decisions about how to use them,

are not in the hands of experts in nuclear physics. They are in the hands of experts in various mythologies. That is part of Matt Galloway's conversation with the Israeli historian Yuval Noah Harari, recorded live on stage in Toronto last September. And if you'd like to listen to more of that conversation, go for it. Go to our website, cbc.ca slash thecurrent, or you can find it on our podcast.

For more CBC Podcasts, go to cbc.ca slash podcasts.