Technology, Connected - Nexus: Yuval Noah Harari on AI, Digital Empires and the Silicon Curtain

Episode Date: March 18, 2025

In the final chapter of Nexus, Yuval Noah Harari argues that artificial intelligence could divide the world into competing digital empires, each built on separate hardware, software, data and political systems.Mark and Jeremy examine Harari’s idea of a “Silicon Curtain” and what it could mean for the future of AI, geopolitics and the internet. Rather than producing one shared technological future, AI may accelerate the fragmentation of the global information system.In this episode, we discuss:What Harari means by the Silicon CurtainHow the United States and China are building competing AI ecosystemsThe role of data colonialism in the development of artificial intelligenceHow avatars, deepfakes and digital identities affect our understanding of realityWhy digital personas may become more influential than physical individualsHow software, data and computing infrastructure are reshaping geopolitical powerWhether societies can build more reliable information networks in an increasingly divided worldThis episode isn’t primarily about artificial general intelligence. It’s about who controls AI systems, how those systems shape perception, and whether a shared global information order can survive.Please enjoy the show.–Chapters(00:00) Disruptors and curious minds(01:54) My AI Epiphany(02:54) The precondition for cooperation isn't similarity. It's the ability to exchange information(04:28) AI Pessimism - How many nefarious bad actors do you need(05:50) The Liverpool Manchester Train moment for AI(06:48) Google was an AI company in 2002(08:21) Cats and consciousness(08:55) ImageNet Large Scale Visual Recognition Challenge(11:50) The mind-body problem and online identities(15:26) Veneer theory and realism(17:22) Data colonialism and digital "cocoons"(21:58) Epilogue: Create wiser networks(24:13) How to be wise in the age of AI--Read books with us: www.thinkingonpaper.xyz

disruptors and curious minds, CEOs, founders, book lovers. We have reached the end of Nexus by Yuval Noah Harari. We have a new book, we'll tell you at the end. But before we get into that, I... I've always thought that maybe this book would end with a big Hollywood twist, that it would go into a last chapter of happiness and optimism and hope. But instead, Yuval notches it up and notches it up. and notches it up, but he ramps it up. Digital Empires, turning points, the end of civilization, nuclear war.

Cats! I blame cats. It's all the cat's fault. Chapter 11, the silicon curtain is closing. Could it have been Schrodinger's cat responsible for this at some point? Yes. Whenever we're connecting dots between technologies, there's always a bloody cat involved. Seriously. So, all right, friends, we're turning... Oh my God, you have a cat. Okay, so we're about to turn this up to 11, as they say in Spinal Tap. The final chat. The final chapter, Mark alluded to the interdependencies between all of these different things, being really interesting, but not also being exacerbated by AI. So societal interdependencies, one of the first sentences in this last chapter I thought was really interesting.

So you have all of these interactions between all of these different entities. And doing the interactions manually takes a really long time. Doing the interactions by connecting these information networks, AI does that very, very well. This will create more independencies, interdependencies. It will also create more walls. Society's already walled off. He talks about the silicon curtain is going to wall us off. And this is just going to make, it's going to make those walls harder to navigate,

easier to manipulate and able to spin that stuff up in a worse way, I think. It's just it's gas on the fire, right? Gas on the fire. And just when you were speaking, I had my epiphany. Well, this is I've had my AI epiphany. And he mentioned. it on the first page of chapter 11. Computers are not yet

powerful enough to completely escape our control or destroy human civilization by themselves. As long as humanity stands united, we can build institutions that will control AI and identify incorrect algorithmic errors. I don't need to read the rest, do I? Of that sentence.

Because we've proven to be so good at that. Universal coordination, global coordination. AI is not the problem. That's a fact. AI is not the problem. That's what this book should be renamed. AI is not the problem. The problem is us. Well, he references how we handle the industrial revolution and that led to imperialism, but also led to some awesome stuff eventually. And he talks about, hey, we muddled through, I think was the term he used. We muddled through the industrial revolution.

What's going to happen if we muddle through AI? But let's look at the bright side. I'm going to bring a bright side into this, Mark. I really love this. This was a little bit later in the chapter. Sorry if I'm jumping, but this quote was really cool. The pre-judition... You've earned the right to jump now, Jeremy, after 11 chapters, jump away. I'm jumping. I'm jumping. The precondition, this is a quote, the precondition for cooperation isn't similarity. It's the ability to exchange

information. So let that settle in there for a minute. We think we have to cooperate with those who are similar to us. Doesn't that just perpetuate the echo chamber is what I think about. And we have to get up another level

of abstraction reference to Paki McCormack, most human wins, tying that in, if we get up to a higher level of abstraction with this, cooperating on a global level, thinking bigger than the idea of nation states, because guess what? Nation states are intersubjective realities. Let's point back to chapter one, two, three, right? You know, so, so. I would have loved if Zelensky would have gone to Jamie, Matt. Nation states and intersubjective reality. That would have been fantastic. That would have been fantastic. But I think about that that's the root of all of this. It's not about A.

It's about us and our inability to cooperate in a way that serves the greater good of all of the people, of all of us. And I know we have to protect ourselves. I know we need to find secure ways to move about our day. But we kind of start thinking higher levels because AI is going to change the game very, very quickly. and yeah. I'm going to pour a big dolep of pessimism on your hope there with a simple question. And I think that superintelligence by Nick Bostrom,

obviously Nick Bostrom has mentioned quite a few times in this book. We've mentioned his book quite a lot. And how many bad actors do you need? How many defarious bad actors do you need, Jeremy? Not many. You just need one. Yeah. Yeah.

So if you're going to get humanity singing from the same hymn sheet, it's going to be quite difficult when borders don't matter, when there is no, like, it's either the all. of us or it's none of us? Well, think about the nefarious bad actors. Like, he mentions another thought experiment. Imagine in 20 years when there's a new group of leaders coming out that, you know, maybe today or eight, when they start becoming, when they become a leader, they're 28.

And maybe they haven't been so careful about what they post on social media or what is, what gets captured of all the eight billion cameras in the world capturing you doing things that you don't want captured, right? And now someone has control over all of those things. If they have control over all those things, I would say they have control over you as well. Another point of manipulation that, again, wow, as if we couldn't live deeper and darker into, you know, the rough side of this. Well, that's a nice segue into digital empire. So he speaks about digital empires.

He talks about tipping points, turning points, says the Liverpool to Manchester train, which nobody thought trains were going to do much. and then the private sector built this train between Manchester and Liverpool and that catapulted Britain to the forefront of everything and then a few decades later we own half the world and we have all these dominions everywhere. He likens AI and the race to develop AI to the race to develop trains the private industry, the private sector versus the government public sector and who was the most successful. And he likens the private AI developers of Open AI, etc. of today versus government and how it's mirroring that development of infrastructure.

And we're at this point where we're just waiting for this turning point, this Liverpool to Manchester train equivalent in AI. And then I'm sure you noticed the Kevin Kelly quote. I did see the Kevin Kelly quote. That was interesting. We'll have to ask him about that the next time we're on the show. But the idea behind it was he was talking to Larry Page and was like, hey, you know, free search, free web search? What's the, what's? In 2002, you know, web search for free,

where does that get you? And page responds, we're actually making an AI. They're ahead of the game, aren't they? They're like those train builders. They're like the bloody train builders, and they know they've got to build this train. All right, so the train builders, but then you have like the rest of the people. And I think there was a reference to why should a, you know, a rice paddy farmer care about a new network of trains going through their area and what does that mean from a bigger picture perspective. Here's where you start to worry about that a little bit because here's the question, are smaller cogs in the wheel capable of bigger picture thinking? When I say small cogs in the wheel, I don't mean to diminish who they are or their potential or their intelligence, but the

idea that these smaller cogs in the wheel haven't achieved the levels of Maslow's hierarchy of needs, that they're focusing on, yo, I got to have food right now, food today, where these other people are like, hey, let's draw these train maps all over. And here's how what, here's it's going to happen and here's what we're going to be able to do. And before you know it, you blank and something new has spun up that is yet again controlling a large, large... To answer your question, why should the rice farmers of the Irrawaddy basin in Burma or the Yanksy basin in China concern themselves about the Liverpool-Manchester Railway?

Because by the end of the 19th century, they were all conquered or indirectly exploited by the Great British Empire. And could the same thing happen with AI? Could it, where he poses that question, Jeremy, could it? Yes. I think you just answered it. Yep, for sure. Let's talk about cats.

You showed me your nice cup, your tea cup with a cat on it. Can I just ask you a question on consciousness? Because I didn't actually think about this cup. I just chose it at random to make my cup of tea before the show. I've obviously done my notes. So there was some kind of thinking cats must have been there simmering away under some things. So I chose this cup. I have a lot of cups.

Milk in your tea or no milk? What do you think? I'm going to say milk. Of course milk in my tea. Except if it's green tea or herbal tea, I must like, you know, for the year. Europeans, English tea we're talking about. Yeah. All right.

So a fateful day, September 2012, ImageNet, large scale, visual recognition challenge. So we started putting together a bunch of cat images to teach computers how to recognize a cat. Throughout this challenge and this

experiment, it turns out now cats can recognize, or humans, hold on, pause, me, bell, computers can recognize cats better than humans. We're like, okay, well, That sounds pretty cool, you know, cool, meow cat generator. We get cute cats.

You look at the surface level of this seems very innocent. Not that, you know, the cat recognition immediately had the idea to be turned into something nefarious, but that technology, these little fun little experiments that we're having today could be turned into not so great things down the road because this cat generator, part of the tech of Meow Generator, we look at, you know, Red Wolf, Blue Wolf, Wolf Pack, right? Which are these technologies that are perpetuating the terribleness between Israel and Palestine? Right, I'm going to back up there before we get to Israel and Palestine. Can an algorithm identify a cat? The competition was about identifying pictures of cats, and algorithms were notoriously bad at identifying cats, until Alex Nett, who was incredibly good at identifying cats, because we, I assume it's because we actually flooded the internet with cat pictures

for billions and billions and billions of cat pictures. And it just became really good at identifying a cat. And it demonstrated to experts the potential for rapid progress in certain AI domains, essentially image recognition. And let me back up a hair because I jumped quite a bit ahead there. I'm not saying the meow generator developers knew that similar technology would be used to do other things. You know, it's just really what hit me was the experiments that we're making today could have effects, beyond the result of the immediate experiment was where I was headed.

Yeah, secondary downriver, second order effect. What he does to say here is, because obviously on my first read, it was like, they can only identify 96 out of every 100 cats. These cat images must be very hazy or blurry or manipulated, or the cat must look like something else to actually fail. Very good point. How is your cat recognition capability? Well, if it was just 100 cats,

Cats, by the way. I know we're talking on cats. We've got a show coming up with World Ideen. We're talking about proving you're a human in the age of AI. Thinking on paper X, Y, Z to check that out. But cats, no. The cat ain't falling for no bot. That could be really interesting.

So not just cat identifier, but how do we organize cat's behavior, cat's tendencies, cats' superpowers? How do we think like a cat? I don't know that I want to do that. All right, here's another question as we move through this chapter. we have this idea of like supercharging the mind-body problem, which again walks us back through the history of certain religions and, you know, the Old Testament referring to humans as merely physical bodies only. The kingdom of God is of flesh and blood until right around 1274 where the approach became a little bit more dualistic.

I don't want to go too deep in the Christianity side, but the reference is rooted in an interesting question. What is the relationship between our physical bodies and our online identities? When we start doing more things online and we start doing things online through avatars or online personas, well, we care less about the problems that affect the physical world. So if we're online all the time, Mark, are we going to become less concerned with things like global warming, less concerned with things like communities that get taken advantage of, co-opted? Really interesting. So lastly on that, before I let you riff, is if society moves from identifying humans as physical bodies,

if we spend so much time online that we start thinking of online personas as human people without physical representation, could we then start beginning to think of AI more as a human? That was a really interesting thread in the latter part of this chapter. I agree. I agree with you. And it takes a couple of pages to get to that question. And while I was reading the two pages to get there, I almost stopped reading. I think this isn't really going anywhere. But then he poses the really interesting question at the end.

Quote, this debate could shape attitudes not only towards organisms, but also towards digital entities. As long as society defines identity by focusing on physical bodies, it is unlikely to view AIs as persons. But if society gives less important to physical bodies, then even AIs that like any corporal manifestations may be accepted, as legal persons enjoying various rights, which is what you said. And yeah, it makes you wonder quickly we get to that point because people use avatars now,

I think most of the world probably don't. Most of the world don't use avatars. And as gaming proliferates, as the next generations come through, they will. And you have your online persona. You have your real war persona, and they merge and they merge and they diverge.

And yeah, and that, it raises the question. at what point are AI is accepted as one of us? I accept you with your strange avatar that there's no resemblance to you in real life and I accept this thing over here which I've given up asking whether it's AI or not. I'll just accept it for one of us. Well, have you heard the latest news on Sesame AI

who's working on voice creation from AI, human voice replication and creation? Tell us more. I didn't get a chance to dig in too much but they've created these voice identities as an interface to these AI systems, right? So instead of typing into the chat, you're actually talking through it. And they've been very successful in testing the relatability of these AI human voice recreations. You have that human voice, that could eventually be manipulated.

And then you think about fishing scams today that are bad with the Prince of Nigeria asking you to let his son out of jail or whatever the heck it is. What if you have like your grandmother's voice call you and be like, Mark, I need some help. That gets, that gets a little funky. Let's see. Fortunately, most people don't answer their phones anymore. I don't. That's a fact, yeah. What did you think about, what did you think about the veneer theory?

So I studied international relations at university, Jeremy, and I was a 21-year-old realist. Oh, yes, I'm a realist. we suck. So let's talk, so you see you had, when you were going through school, so realism, as I understand, realist thinkers, it's all about power, right? It's about the attainment of power in order to achieve control. Is that a core theory that I picked out of this? Well, I think that he explains it with Hans Morgenthau and John Meishina

have argued that an all-out competition for power is the inescapable condition of the international system. The veneer theory takes it to the next. next step, it argues that at our heart humans are Stone Age hunters who cannot but see the world as a jungle where the strong prey upon the week. And where might makes right. On my bad days, of my

cynical bad days, like Friday, for example, about 7pm GMT, I think that but on my brighter, more optimistic days, I like his quote, as for Stone Age hunters, they were gatherers as well. I mean, they're only gathering to feed their killing sprees, but, you know,

I think, on my brighter days, I don't Subscribe to the veneer theory, no. Well, dot, dot, dot, the jungle code, so the strong over the weak, the strong controlling the week to manipulate, control jungle theory. But guess what? This was really cool and bright at the end of this. Real jungles are full of cooperation. Let's point back, hey, just a little ding ding, mycelia networks, right?

Mycelium networks. You got these little strands between the fungi that connect the dots and the trees and the enable it. It's a fiber optic network of the forest. So when you really think, about it, scientifically based, there's tons of cooperation in that where I would debunk from principle, from theory, the jungle code, but unfortunately some people decide to live by the jungle code. Yeah. So after the jungle code, he gets into populism again. I'll let you, if you

want to take that, you can take that. Could I just ask you a question just before that about your thoughts on data colonialism and the cocoon? Yes, there was a quote. The reference to take over a colony, you don't need gunboats anymore, you just need to extract data. So if you can extract data, you can control things. And he talks about these cocoons being these little digital empires where instead of having two, three, four superpowers in the world, we could have 12, 14 superpowers in the world that activate their wishes upon the others in a digital way, instead of a physical way. The Cold War analogy was great. A hyper-rational game of chess. If I push this button, everybody dies. It's really kind of easy and straightforward. But when we have these multiple

digital empires that are able to interact and cause chaos digitally, it's harder to track that. It's harder to see what's going to happen down the road because, number one, you can't spot all the interactions. Number two, you can't see where they're going to go because a lot of them are emergent. And it's hard to predict the end outcomes. That, that again, damn it, we're like in scary land again. Well, yeah, because if the big red nuclear button, you press it, you know what's going to happen. And so mutually restored destruction permeates and nobody presses it because they know what is going to happen. But if you have next to it a little green button because it's AI and it's painted it green, so you know blue, so you trust it.

So you press it. And it sends some kind of AI virus somewhere. But you're more likely to press it. A government is more likely to press it. And leaders are more likely to press it because they don't know the outcome exactly. Well, within these cocoons that he calls, you're going to have the people and you're going to have AI in the form of bots or personas or whatever the heck it is. And just millions upon millions upon millions that are, once they're unleashed and their ability to interact, the possibility of doing things is kind of endless. But here we go.

It goes back to this as humans. Hey, let's just all get in a room together. Let's work it out and make sure we protect everybody. but, you know, the likelihood of that doesn't seem... Question for you, Jeremy, going back to nation-state is just... Subjective reality. My favorite term coming out of this book. Into subjective reality.

So, question for you. Imagine a situation. In 20 years, when somebody in Beijing or San Francisco possesses the entire personal history of every politician, journalist, colonel, and CEO in your country, every text they ever sent, every web search they ever made, every illness they suffered, every sexual encounter they enjoyed, every joke they told, every bribe they took.

Would you still be living in an independent country or would you now be living in a data colony? So, all right, let's extend that a little bit. I think data colony. So let's extend that. You and the 12 people who lead a particular country are going to a global meeting to figure out AI. And you're like you have these grand ideas

about, hey, we're going to do X, Y, and Z. You know, we really need to lead the charge in this and blah, blah, blah, and then the dude that has all that information is going to go to you be like, Mark, remember this? Remember this? Are you sure you want to lead the charge in that way?

Because, you know, if you can, we're getting very black mirror again on a lot of this. The way you say it, and if we follow on from the question quote that I just asked you, it doesn't seem like a very difficult conclusion to come to, though. The way he paints the picture,

all of our deepest, darkest fears about AI, suddenly come into the light very, very easily. It's almost like a script where this A follows B, follows C, follows D, and the big climax in Acts 3 is the end of the world. But he says multiple times in this book, technology is rarely deterministic, but that sounds very deterministic.

He also numerous times says the AI isn't a technology like we're used to. He goes to pains to point out that you can't compare it to the printing press. You can't compare it to the Industrial Revolution. You can't compare it to those. It's not like for like.

Everyone says, oh, the printing press was amazing. AI will be amazing. Yes, you pointed out that the printing press gave us the witch trials and mass murder. The Industrial Revolution gave us huge, huge shifts to our political, social economic, millions of people died. Like, there's a downside to all of it. And he says AI is not a normal technology as we're used to it. So you can't have both arguments. Yeah, well played.

So let's tiptoe into the epilogue here. Spoiler alert. Spoiler alert, if you didn't read this entire book, all we have to do, all we have to do is create wiser networks. That's my takeaway. We just got to create wiser information networks. Bam, done. I'm out. Bye. Yeah, rather to create wise networks, we must abandon both the naive and the populist views of information, put aside our fantasies of infallibility and commit ourselves to the hard and rather mundane work of building institutions with strong correcting mechanism. That is perhaps the most important takeaway.

this book has to offer hashtag wishful thinking. Wow. Well, hang on to your self-correcting mechanisms. Make sure important things aren't excluded from your AI models. Like the exclusion of the acts of Paul and FECLA that led to the giant growth of misogynistic views over time. That's pretty important. Something gets left out of a universal book because some guy doesn't believe in it. Look at the repercussions.

Be careful what goes into your models. friends and neighbors. Be careful what you wish for. Jeremy, last question for you. So you're going to go meet some family, you haven't seen it for a long time, meet some friends for a beer. They're going to go,

what are you reading at the moment, Jeremy? And you're going to say, oh, I've just read Nexus. And they're going to say, could you summarize it for me in like a bit of a sentence? Because I can't be bothered to read 400 pages of AI doom and gloom. What are you going to say? Wow, put me on the spot. I would say it was the study of information networks,

how they were used over time, how we can point back to historical outcomes based on information networks and what different technologies created deeper, faster, quicker information networks and how the patterns we've seen in the past are shaping what we see in the future, which is not any grand, grand aha moment. But now you look at the ability for all of these things to hold together. It comes back to me, the idea of controlling something like this or getting our hands around it, you've got to have the self-correcting mechanisms.

We've got to have these things that will autocorrect. instead of lockdown or block down all the blockades that keep you in the center of the lane as you're driving. I think that was the biggest thing. And I love the idea. You said it in jest earlier, but creating wiser information networks. What does it mean to be wise? I think to be wise is to seek true understanding, no matter your current position. And that requires a little bit of empathy for other people as you hear their view. And it also requires a little bit of vulnerability, knowing that, hey, what I know today about this subject may not be all there is to hear. It might just be a small piece of it and I need to decouple from this belief that I'm just driven to defend it. It's basic. It's not even about

tech. It's about that. That's what it's all about to me. I like it. Good answer. Nexus, Uval, Noah Harari. I think that, I think it is an important book. I said this in the last show and I say a lot of things in Jess, but that's just my way of interpreting it. We live in a, going to live in a post-AI world, I think the world before AI and the world after AI, regardless where you are, what you think, where you are on the spectrum, whether you're an optimist, whether you're a dunger way, somewhere in between. You cannot deny that the horse has bolted, the horse is out of the shed, that the AI horse is rampaging across the earth and it's going to change entertainment, it's going to change

work, it's going to change our relationships, it's going to change the internet, that it's going to change finance, it's going to change how we interoperate with each other, is going to change relationships, it's going to change culture, how it changes, what changes and to what degree? We're going to find out. And I think that you said, you asked about what it's like to be wise. And I don't know the answer to that, but I do know that you've got to read both sides of the story to get anywhere near being wise. And I think that this book pays a very good picture of one side of that. And hopefully we'll read a book about, optimism next. The big exhale from the book and I enjoyed it as much as we're like, man,

this thing was so dark that the, what we saw, the case he laid out from history and stair step in all of these different regimes and talking through it as information networks. I thought it was great. I definitely, I definitely enjoyed it. I'm glad I read it. It was dark. I feel like I can exhale a little bit, but now I'm moving forward with a little bit of it. But you're moving forward, like you're thinking about it. You're engaging with material. This is point of thinking on paper and if you're listening to this that's the whole point of what we're doing some of this is it's kind of not scripted we take notes and then it emerges as we talk and that's why we do it because it helps us to understand and it will help you to understand that's why

we do it and come and join us what are we reading next so out of the frying pan into the fire I'm really excited about this book it's by Federico right here so he's Italian so we're going to say fagin, I think of that, jeez, like a je, a jeepagin, I don't know. Can you pronounce a title? He's definitely not coming on the show because he, you can't pronounce his name yet. We need to work on that. I'm going to edit the update the update out. He is going on the show.

Wonderful. Let's do it. So this talks about consciousness, life, computers and human nature. Mark, when you shared this with me via text the other day, it was such an immediate yes. I was just like, yes immediately. So I'm not sure where the adventure will take us. Come on it with us. Read this with us. Buy the book. Jump in, pop into one of these squares. We've been asking you guys for quite a while.

We've had a few guests come on. We'd love to unpack this with you in real time. Help sharpen our minds. We can help sharpen yours. Do you know where I got this book idea from? You'll like this. Talk to me. It was from a tweet from a man who writes a newsletter about being human. So Pachie McCormacky. Back to Packy. Back to Packy. Packy. You've got to come on the show. He posted a picture of this book.

and I clicked on the book and like you, it was an immediate... Hell yeah, we've got to read this book. Federico Fajin is one of the greatest luminaries of high technology alive today. A physicist by education, he is the inventor of the microprocessor and the MOS Silicon Gate technology, both of which underlie the modern world's entire information technology. See, we're always tying books together. It's just magic. It's thinking on paper magic. Frederico now turns his attention to consciousness and the nature of reality. reality, bearing with us his profound insights on the classical and quantum world,

superficial intelligence, life, and the human mind. Buckle up, kids, buckle up. So get outpaper.xyZ, we have two ways we push stuff out to you guys. One is this book club where we actively read, critically think, think on paper together. This is participatory. This is not designed to be a broadcast. This is designed to be a book club. Come in and read it with us.

Hello at Thinking on Paper.xyZ. Drop us a note if you want to join in. Secondly, we interview some of the most interesting technological leaders, cultural leaders in the world that are painting a new picture of the way the world's going to look. We bring you backstage to ask the questions that you care about to try and understand what it all means. If we can get better at something, send us a note, drop us a comment. This is for you. This is not for us. Off we go.

I'm going to add to that because people talk about connecting the dots. We really are connecting the dots of emerging things. technology. In the last two weeks, we've spoken to people putting data centers in space, building ion-trap quantum computers, using AI on satellites to get real-time analysis of natural disasters. A blockchain 500 brand advisor advising the biggest brands in the world what to do about AI and blockchain. Robot pills in your stomach. Robot pills in your stomach. We've got world ID coming to speak about proof of humanity. It's there. And all you just connect the dots, thinking of paper.

by X, Y, Z. That's it for today, guys. Be curious. Stay disrupted. Keep thinking on paper. And we'll see you next time. Yeah. Goodbye.