Technology, Connected - Tech CEOs Are Selling You the Future

Episode Date: April 24, 2026

Carissa Véliz joins Thinking on Paper to examine how AI forecasts, platform algorithms and prediction markets can influence the future they claim only to predict.Predictions aren’t always neutral d...escriptions. When they come from powerful technology companies, executives, platforms or financial markets, they can change investment, policy and public behaviour. A forecast may become a self-fulfilling prophecy because people act as though its outcome is inevitable.The conversation begins with a broader question about the good life, curiosity and what the analogue world offers that digital systems often remove. It then turns to the institutions increasingly making predictions about people and society.In this episode, we discuss:How AI predictions influence human behaviourWhy forecasts can become self-fulfilling propheciesHow technology executives shape expectations about the future of AIWhether AI hiring tools reinforce existing assumptions about workersHow TikTok and other recommendation systems direct attentionWhy engagement-maximising algorithms reward harmful contentHow prediction markets such as Kalshi and Polymarket workWhether prediction markets measure beliefs or help create outcomesHow platforms exploit the human desire for certainty and securityWhat the Molly Russell case reveals about algorithmic recommendationWhy comedy and serendipity resist predictive systemsHow citizens can make more deliberate choices about technology and beliefWhat Epicureanism offers that digital optimisation cannotCarissa argues that people should treat influential predictions as interventions rather than passive forecasts. The more reach and authority a prediction has, the greater its ability to reorganise the world around itself.This conversation examines how to resist technological prophecy by preserving uncertainty, curiosity and the freedom to choose futures that algorithms haven’t already selected.Please enjoy the show.--Thinking on Paper is a technology podcast about AI, computing, science, and the systems shaping the future.📺 Watch On YouTube: 🎧 Listen to every podcast⁠📺 Follow us on ⁠Instagram⁠🏠 Follow us on ⁠X⁠🏠 Follow Jeremy on ⁠LinkedIn⁠To suggest guests or sponsor the show, please email: hello@thinkingonpaper.xyz--CHAPTERS  (00:00) Intro(01:00) What is the good life? (02:00) Why knowing yourself matters more than strategy (04:44) The analog world vs the digital world (06:45) How prophecies exploit our need for security (08:47) Ancient Rome (10:11) The illusion of safety (12:27) When predictions work(15:00) Altman, Amodei, Huang(28:29) How to resist prophecies (29:53) Prediction markets(31:49) TikTok, algorithms, and the Molly Russell case (36:08) Engagement algorithms(40:54) Self-fulfilling prophecies (43:44) Comedy(46:59) Seinfeld (52:16) Karikó (53:40) Serendipity (56:13) Why Epicurus beats the Stoics

Transcript
Discussion (0)
Starting point is 00:00:00 The company Clarnah, the payments company Clarna, fired 700 people quite publicly not too long ago. And the idea was that AI was getting good enough to replace them. About a year later, they were hiring 700 people. So often we idealize AI, even when we're working with it, and it's not working, when we hear that it's going to work perfectly by these guys. We tend to believe that. We tend to be optimists when it comes to technology. Prophecy, prediction, power, and the fight for the future, from the ancient oracles to AI.
Starting point is 00:00:38 Prophecy by Carissa Velaz, one of the most important books you will read in the next coming years. It talks about the effect of these technologies on us as humans. It's what we talk about it thinking on paper every day, the intersection between technology in humanity. This is explored in a most profound way, a most accessible way. you will not want to miss this episode. Please enjoy our discussion with Carissa. What is a good life? What is the good life?
Starting point is 00:01:09 That is a very philosophical question. Perhaps the most important philosophical question. It's the kind of life where when you get to the end of it, you feel it's worth the ride. I've done my bit. It's worth thinking about because sometimes we're so caught up the logistics of life, that we forget what it's all about. And I think the good life is one that you have written yourself, that you didn't receive a script from someone else that you just
Starting point is 00:01:41 followed, that you feel that you thought about what you think is important, and that you got to know yourself well, and got to know how you can contribute to the world, what are the things that you enjoy and that you find meaningful, and that you went for it. that you gave it your all, that hopefully you left the place better than you found it, and that you had a good time and great friendships and meaningful relationships in the journey. Talk to me about how knowing yourself is so powerful in this self-directed life because I think people get caught up in this business of busy and this cycle, this whirling of things that we should, we think we should be doing. How is knowing yourself help with that
Starting point is 00:02:34 piece? We're very anxious creatures and that's partly because we're smart and we're smart enough to get a sense of everything that can go wrong. But that anxiety is tricky because while it can be a useful engine and it can be a motivator and I think it's a good sign if you have anxiety about the uncertainty of the future, it means that the future is not written, that you can intercede, that you can have a chance to change it. Because we are such anxious creatures, it can lead us to look for answers from someone else. So sometimes when we're so scared and anxious,
Starting point is 00:03:10 it's easier to go to someone and tell them, tell me what I should do. Tell me what the future looks like. Tell me what path to follow because I don't know. And that makes us very vulnerable to people exploiting that situation for their own benefit and for their own agenda. So part of what it means to know yourself,
Starting point is 00:03:27 is to be able to sit with that anxiety of knowing that you're a mortal being and that a lot of things can go wrong, is sitting with that fear about making a mistake, making an error, but also knowing what attracts your attention, what makes you curious about the world, what makes you joyful, and what you're good at in a way that you could contribute to the world that maybe nobody else can. And one of the things I get most asked by both students and parents of students and readers from around the world, people email me asking me, what should I study or what should my child study? And my best advice is, look, the future is completely unpredictable. And if you're going on strategy, you're probably going to get it wrong and you're probably going to be miserable in the process. So follow your curiosity. Your curiosity tells you that there's something there that needs to be looked at twice.
Starting point is 00:04:31 And one of the things I want to encourage people to do is to be bolder about life because it's not worth living a life that's not yours. When you're bald and when you find what you're interested in, you're much more likely to find a different angle that nobody has seen before. Because nobody's like you. Nobody has your background. Nobody has even your perceptual sensitivity. And it's finding that which others are not seeing
Starting point is 00:04:57 that can give you more of a competitive advantage, ironically, than trying to predict what the future will be and trying to follow that script to play it safe. Playing it safe is also a bet, and safe bets are not always as safe as I might seem. You're painting a very analogue experience. Are you suggesting that for the most part we should remove digital experiences from our existence, or can there be a dual digital analog experience
Starting point is 00:05:31 to the human condition? I think we should have two tracks, the analog and the digital, but I do worry that recently we focus a lot on the digital, partly because the digital has better marketing teams right now than the analog world, and we forget that the analog is... Anologue has nature. Surely nature is the best advertising marketing department there is? You think, but if you grow up far away from nature and you've never had that contact and you just see it in documentaries, that's not enough. Nature is something, we're such cultural creatures that nature, even nature is something that you have to grown accustomed to and learn to appreciate and learn to see. So, yeah, I think that we forget that the analog is the primal experience, that everything did. depends on the analog. We lose the analog. We lose everything. We lose the digital and we lose a lot,
Starting point is 00:06:27 but we don't lose the analog. And so there is an asymmetry. I do think that analog is more important than digital. I don't want to do away with it digital. I just want to put it in perspective. And that if we focus too much on the digital to the detriment of the analog, we are heading towards ruins. You're not going to hear about athletic greens. We have no electrolyte drinks or AI agents to sell you. The show you're listening to, Thinking on Paper, is funded entirely by me and Jeremy. So this is an advert for us because we sponsor ourselves and we need your help. So please subscribe wherever you're listening or watching Thinking on Paper. And if you're feeling really generous and kind, please leave a comment.
Starting point is 00:07:12 Now, back to Carissa. You've set us, set this conversation up in the context of we as human creatures are highly, perceptible or highly influenced or highly vulnerable to the mechanism that you're talking about in prophecy that pulls us away from who we are and into a situation that's controlled by others directed by others. So how does the manipulation of our need to feel secure, how does that make us vulnerable to these prophecies that you're talking about? Can you walk us through that a little bit? Essentially, we're all worried about tomorrow because that's where we're all being spending the rest of our lives. And we're wondering, are we going to have a job?
Starting point is 00:07:59 Is our family going to be okay? Is war going to break out? Are we going to be able to navigate climate change? There are all these questions. And we want answers and we don't have them. And so we have a tendency to ask other people about it, often experts or very rich people. I think we associate wealth with success. Surely if this person became so wealthy, they must know something that's.
Starting point is 00:08:21 I don't know. And so we asked them, you know, tell me what the future holds. I get, I get asked all the time, what's the future of privacy? As if I or anyone else knew. And then predictions become the arena where fights about the future get played out. So if you go to a tech executive and you tell them, tell me what is the future, what do I need to know to be able to navigate the future well? What is he going to tell you? Of course it's going to tell you that the future is his because that's that's in his interest he has a company has a product to sell and he wants the future to be to be that and it happens to line his pockets as well and so we end up believing as facts what very often are power plays in disguise or marketing or some other kind of power related move often it it is a
Starting point is 00:09:18 self-fulfilling prophecy it's that there there's many ways to class predictions, but one way to do it is one thing is to predict something about objects. So if I predict whether it's going to rain tomorrow or not, it has no effect on whether it actually rains because their clouds are not listening to this podcast. But if I predict the future about human beings, and in particular, when I predict the future about one particular human being, that can have a huge effect because it changes expectations. It changes how we view the world and how we treat other people. And it's a mistake to think about predictions as facts.
Starting point is 00:09:55 Predictions are never fucked. At best, they can be educated guesses. But facts are only about the present and the past, never about the future. However, predictions are very misleading because they do sound like fox. It sound like fox about the future, but that actually doesn't exist. The future is unwritten and it hasn't happened. For example, in ancient Rome, it was illegal to predict the death of the emperor. for the very simple reason that that kind of prediction
Starting point is 00:10:20 ended up with a dead body of an emperor. Talk to us about this illusion of safety and illusion of security that when you rely on these predictions, because we're so busy, we're running and gunner, we're doing so many things, we need shortcuts. Like, that's what ends up happening. But talk to us about what we're actually getting
Starting point is 00:10:40 for that exchange. So one very common use of AI is to minimize risks for companies. And that can look in different ways. One is the use of AI to hire people. And so the idea is if we use data to see what kind of person has been successful in the past, we can predict who's going to be a good employee and thereby lower your risk when you hire someone that you might be hiring a walking disaster.
Starting point is 00:11:09 Another use is AI for supply chains to try to predict what are going to be the bottlenecks or challenges or demands. And in each case, AI and other kinds of prediction give us the illusion of safety. Because part of what gives us that illusion is quantifiable. So if you can put a number on it, you feel like you're closer to knowing more about whatever that is.
Starting point is 00:11:40 So if I tell you, you know, this person has a 70% chance of not repaying a loan, As a bank, you might feel like you are safer than if you didn't have that number to rely on. However, one of the limitations of any kind of prediction, including AI, is that it assumes that the future will look like the past. And for the most part, that's actually true. But it's not always true. And what's most interesting about reality is that the most important events of your life,
Starting point is 00:12:16 of your individual life, but also of our collective lives, are the most unpredictable ones. So it's very easy to know what's ahead when the road is completely straight. It doesn't take a genius. It's the curve. Stack the odds in some domains in our favor with predictions, though. So you mentioned supply chain and logistics and hiring. What about drug discovery or climate monitoring
Starting point is 00:12:42 or one day, perhaps in the future, the ability to predict earthquakes? and tsunamis and natural disasters. So there is a yin to the yang. There is a positive, a net positive for humanity to predict, isn't there? Absolutely. And it's partly what has given us a competitive advantage as animals. We're pretty weak physically. But we have language.
Starting point is 00:13:04 We have opposable thumbs. And we have this ability. If we can predict how an animal is going to behave, it's easier to hunt them. If we can predict the seasons, it's easier to use the sky as a map. if we can learn how to read the stars and so on and so forth. But it's interesting that the three examples you gave were examples of predicting things about things and not about people. So that's the first point to note.
Starting point is 00:13:29 And the second thing is, yes. Individual genetic disease. Okay. So but even then, I mean, that is a kind of a mixed case because there's a lot that human beings can do to affect which genes get activated. But even then, you might argue. that there are some genetic diseases that if you have it in your genes, you're going to get no matter what. And so it's not really about human behavior.
Starting point is 00:13:54 But the other point to note is that even when, yes, prediction gives us a competitive advantage when it works well, because the future does look like the past, it works even better if we are aware of its limits. So one thing is to use prediction and just note that the future, that the future, there is a premise there that is implicit. If the future looks like the past, then this is going to happen and always be aware that sometimes the future doesn't look like the past. For example, with climate change. So our predictions about climates are very limited because there's no database about the future. It's as simple as that. And we will be exposed to things that haven't happened before in the face of the earth. And there is no way to predict them for that reason.
Starting point is 00:14:41 And when it comes to human history, events are much more unpredictable. Even when it comes to the physical reality, we don't know whether one day we're going to be hit by a meteor. And we can't to a certain extent predict it because that's a trajectory and so on. But there are physical events that are hard to predict or impossible to predict. But when it comes to human history, that's even more the case. Because one of the annoying and wonderful things about humanity is how we tend to surprise ourselves. And when it comes to things like many wars are started by a particular murder. And whereas we can predict roughly with some accuracy how many people will be murdered in a
Starting point is 00:15:24 particular year, who will be murdered and who will do the murdering is absolutely unpredictable. Can I read you a couple of quotes, Carissa? So the first is from Sam Altman. Anyone in 2035 should be able to marshal the intellectual capacity equivalent to everyone in 2025. Dario Amodi, I simultaneously think that AI will disrupt 50% of entry-level white-collar jobs over one to five years, while also thinking we may have AI that is more capable than everyone in only one to two years. Jensen Huang, AI agents are the new digital workforce. The IT department of every company is going to be the HR department of AI.
Starting point is 00:16:11 AI agents in the future. What do you think? What do you want to tell me and Jeremy when you hear quotes like that, predictions like that? I fear that the way they're interpreted is as a fact. These guys know a lot about AI. They're making AI. If they're saying this, this is going to happen. And I better act in accordance with that.
Starting point is 00:16:31 So if every company is going to have an AI department, then I better start building that now. So I don't fall behind the curve. And what I would like people to hear is, okay, this is a very rich guy with huge financial interests who is selling this product. He's made predictions in the past. Have those panned out? Not that great. All right. Is that the future that I want?
Starting point is 00:16:58 Is that the future that I want to see? And if not, what am I going to do to build the future that I want to see? We're in this conversation about our susceptibility to being pulled into these predictions. And it's not, it's not just that we're gullible people or we talked about the whole fear, security thing. I ran across something, and you referenced Bays quite a lot in, you know, when you set the stage for probability theory and all of that and how that rolls into it. I ran across a guy named, I don't know him, but I ran in my research, Carl Friston, who has this theory called the Bayesian brain, which essentially says that our brains are prediction machines. Whether that theory is right or wrong, there is some research going into that.
Starting point is 00:17:47 And if that is even a little bit the case, that's a pretty tough situation because we have to kind of think about ourselves as being somewhat disadvantaged in this situation. So what are your thoughts on that theory, number one? And how does that apply to your thinking? And, your message to everybody that, hey, this might be the case, but we are kind of fixable if we think about it a different way. Yeah, that's a great question. So part of the answer to the previous question is how things can go wrong if you are too gullible with these predictions. The company Klarna, the payments company Klarna, fired 700 people quite publicly not too long ago. and the idea was that AI was getting good enough to replace them.
Starting point is 00:18:34 About a year later, they were hiring 700 people. So often we idealize AI, even when we're working with it, and it's not working, when we hear that it's going to work perfectly by these guys. We tend to believe that. We tend to be optimist when it comes to technology. When it comes to thinking about the human mind as a predictive machine, There is a long history of coming up with disciplines, with artifacts, with theories, and then coming up with the piggybacking theory that the mind is like that.
Starting point is 00:19:12 So we saw this with the rise of computers. We never really thought about the mind as a computer. But then suddenly we came up with these artifacts that were quite impressive. And we started thinking, oh, maybe that's how the mind works. So I think there is a bit to that when it comes to theories about the mind. I think that, yes, that is one thing that the human mind does. We do predict in a kind of roughly Bayesian way in which we have a hypothesis and then we test it out in the world, both influencing the world but also just getting
Starting point is 00:19:50 information from the world. And then we revise our views accordingly. I think that is very reasonable. But I think what makes human beings especially impressive is the ability to do causal reasoning, to think about how things connect with one another, even in the absence of data, of previous data, about that particular thing.
Starting point is 00:20:16 And when we rely so much on AI and we lose sight of these causal reasoning, we risk creating blind spots for ourselves. So one of the classic examples in philosophy is that of the turkey. So there's a turkey and it's being fed every day and taken care of. And each day that passes, the turkey grows more and more confident that it has a great caretaker. And of course, every day that passes, Thanksgiving is nearer and nearer and their life is more at risk. And I think something particular to human beings is that ability to think
Starting point is 00:20:54 even if I don't have data to the contrary, I can understand how things work and what are the motivations behind people and how this could go very wrong. What should that turkey do if someone propheses and says you have a less than zero chance of being eaten for dinner in 25 days? What should the turkey do? Escape. Is there not something similar with some of these?
Starting point is 00:21:20 prophecies being right enough that we should act upon them? Well, it depends on the kind of prophecy we're talking about. Because there's also the opposite effect. So Hannah Arendt has this point about how it makes no sense to argue with a murderer who's saying that their victim is going to be dead. The only appropriate response is to rescue the victim. So the prediction that the person is going to eat the turkey is presumably a prediction of an outsider who has no stake in the matter. But most predictions that are about human beings out there are predictions that are not without self-interest and they're not without that kind of reflexive quality.
Starting point is 00:22:09 So presumably the prediction that the turkey is going to be eaten in 25 days is not going to be eaten. is not going to affect the behavior of the caretaker. Unless, you know, turkeys could talk and then if they had learned that I had told the turkey that is going to be killed in 25 days, the caretaker might kill them tomorrow and then freeze them, right? And so the point is that it's not about, okay, is this prediction going to be right, and therefore, how do I act? The point is that the future is unwritten, and when we predict things in ways that, affect expectations, that changes the outcome itself. You write about truth-like things and you write about when numbers are attached to things,
Starting point is 00:22:56 they make them seem more believable even though numbers are a human invention in a sense, which I thought was a very, very clever view of it. Let me read a couple of quotes and then I want to ask you a couple of questions on these quotes. So quote, fiction is simplified, custom design to affect us in a particular way while the truth is messy and unwieldy, hard to appreciate in all its richness. Another quick quote, not enough of a market for the truth.
Starting point is 00:23:28 That one hit hard. With stories hitting harder than facts by design, how could we give ourselves a leg up to know that that's a mechanism being manipulated? It's a bit like encountering junk food. I think now it's a very well-known fact, but hearing it for the first time that, you know, junk food is actually designed for your taste buds and not for your body. And there are reasons why you crave so much sugar and butter or fat because it was very hard to come by thousands of
Starting point is 00:24:00 years ago when our bodies evolved. And now when we have it available all the time, it can really poison us. And it's a bit like that. It's being aware that your mind is naturally drawn to well-rounded stories that make sense that are uplifting or that are terrifying. And being aware that in the long run, you are much better served by actually being interested in the truth, even when it's uncomfortable, even when it's hard to understand, even when it yields a lot of uncertainty and maybe some more questions and answers. it's having that second level reflection on why am I attracted to this story, how much of it is true. And if I believe a false story, how you're going to eventually crash into a wall?
Starting point is 00:25:00 Feels like you're asking a lot of people. It's not, basically, this is not a passive movement. You're going to have to be active if you want to fight the prophets. Before I ask my next question, could I ask you who are your favorite or who are the most dangerous, memorable prophets from your book first? That's a great question. So one very famous prophet is Rasputin. And he's really an interesting guy because he came out of nowhere. This was a person who didn't even know how to read.
Starting point is 00:25:35 He came from a very rural place, not well connected at all. And it's very interesting to see how he gains following, how his eccentricity attracts people, but he's also very well versed in power and manipulating people's views. However, when somebody lies and lies and lies and lies, it's a matter of time before it all comes crashing down because they start earning enemies and people become resentful. One of the things that he did was he was a sexual predator. And when he had harassed enough women, people started getting angry about it. Not only women, but also the men in their lives.
Starting point is 00:26:19 And eventually, you have this incredibly tense situation in which the soars are completely dependent on this person and have made him almost part of the family. but it's really endangering their rule because they have lost legitimacy in the face of the people and at the end it becomes their downfall. So that is a particularly interesting case. Okay, well it's interesting that you use the word followers. Elon Musk has 230 million followers on Twitter.
Starting point is 00:26:56 Big Tech has the ear, just like Rasputin did, of the leaders of the free world. They sat on the front row of the inauguration. They have instant access to literally a billion people, something that Rasputin and the other prophets in the book never had in such a short, tiny time frame. What does that change about modern day prophecy? How is modern day prophecy different to the days of Rasputin and the Oracle of Delphi. In some ways, it's the same. In the sense that the political role that it can play
Starting point is 00:27:37 is the same and the danger is the same, that a prophet with enough power can utter a prophecy, and if people believe it, it can become true. It creates its own momentum. In another way, it's a more complex story because there is a lot of tension between the administrator, sorry, the administration and some of these profits we've just seen a fallout between anthropic and the Trump administration. And so there is somewhat of a mutual dependency there, among other reasons, because governments are using big tech for infrastructure in a way that the czar arguably could have gotten rid of Rasputin without much consequence. And so it's a much more complex picture. but the fact of the matter remains the same
Starting point is 00:28:30 that if one powerful and influential person says something about the future and enough people believe it, there is a momentum for it to become true in a way that if they hadn't said that, maybe it would have never happened. And that makes it a very sensitive issue and something that we should be very skeptical about
Starting point is 00:28:50 and very responsible as citizens to not just peddle any kind of prophecy because somebody influential has said it. You almost want to shout it from the mountaintops, but people might be too busy to believe it. How do you connect to the people who are too busy or who can't afford the time to think so actively about it? One of the pleasures of writing this book
Starting point is 00:29:17 is that it changed the way I see the world, and I hope that it changes the way other people see the world. And just this very simple thought about Be careful with predictions. They're not facts, and they can become self-fulfilling prophecies, has changed the way I interpret most news I come across. And in a way, that's enough. It doesn't take that much.
Starting point is 00:29:40 In a way, you know, the overachievers can go all the way and say, okay, how am I going to be the author of my own life? But even a very passive take of just being a little bit more skeptical about prophecies would change massively the quality of democracy and public. discourse that we have. And as predictions markets become more popular, this is going to be more of an issue. And I've shared the book with a few friends. And it gives you a lot of pleasure when they send me an email saying, oh, today I run across
Starting point is 00:30:14 this news and I just saw it differently. Thanks to just that kind of snippet, it's a kind of lens that makes you look at the world and new. mentioned Kaoshi, Polymarket, according to the Wall Street Journal, they're both, they have their eye on 20 billion valuations, whatever that means. People are predicting, betting, making money on war, leaders, health, murder. In your book, you ask Tim Harford whether you should ban prediction in light of Kauci in Polymarket. What did you mean by that? So I was asking Tim Hartford about whether we should have some limits, whether we should have some guardrails, and in particular with predictions about elections.
Starting point is 00:31:01 And his first response, which I think is the right one and a very sensible one, is, well, wait, we don't want to ban anything that unnecessarily, right? And let's think this through because in liberal democracies, the default is not banning. But when you start to get into it and you say, well, okay, election polls are mostly for entertainment. And if you cash it out that way, you say, well, are they worth the risk, though, for entertainment? Like, there are some good movies that I could recommend that are very entertaining. And so they only become justifiable if there isn't a risk of them distorting democracy. But when there is, then you start thinking, well, maybe we should put some limit. here. And something that Jeremy said is a great point about scale. Predictions right now scale more
Starting point is 00:31:53 than ever, both in the sense that an algorithm that's making a predictive guess can reach millions, if not billions of people. Very often we're using either the same algorithm or a version of the same algorithm. And in the sense that someone with high following on social media can reach billions of people. And that has never been the case in the past. which makes predictions more dangerous than ever. This is pretty timely, and it's also top of mind for me as a parent. I have four kids. We do our best to maintain their relationships with technology.
Starting point is 00:32:30 But, you know, every now and again, you know, I catch them kind of in the loop, which is such a difficult balance to run. But let's talk about algorithms and let's talk about the ownership of one of the most powerful algorithms in the United States at the moment that is that is owned by Oracle that is owned by Silver Lake MDX this group I'm talking about the TikTok US algorithm right and I want to get your thoughts on what effect that pathway that pipeline that accessibility to the younger generation what effect could that have the straight answer is that we don't know We know it has an effect and we know that large segments of the population and in particular young people access this kind of content and are influenced by it.
Starting point is 00:33:28 But it's a hard thing to quantify and to get a good sense of, among other reasons, because the data from platforms is proprietary. So it's not easy to research. However, when we see either academic research or research in courts, it does suggest that these platforms have a huge effect on how young people look at the world and how anyone looks at the world. And one example that comes to mind that is particularly concerning. And it comes to mind not only because it's concerning, but also because it was studied. in depth is the case of Mully Russell. Molly was a teenager in the UK who was exposed to content related to self-harm and who ended up ending her life.
Starting point is 00:34:26 And there was a trial about it. And there's a documentary called Molly versus the Machines that I highly recommend about the trial. And it was the first time that in a country like the UK, it was established that the platform in that case, Instagram, have played a significant role in the death of this very young person. And if that is true, then the same goes for TikTok, of course, because they're very similar platforms. I'm going to let that one breathe a little bit in my brain. Yeah, there's a lot to think about there. And the black box nature of those things and the proprietary nature of those things, like you said, make it very difficult.
Starting point is 00:35:08 And we may not know for 50 years, 30 years, 25 years, what kind of influential effect this has on short cutting, critical thinking, short-circuiting, how people acquire their beliefs and hold their beliefs. Part of what makes a democracy functional is a certain kind of trust in the system, in both your citizens, but also in the system and trust that there hasn't been foul play of some sort. So one concern is that these platforms could be used for political fearmongering or different kinds of propaganda. And one concern is that even if they're not used that way, that they could be used that way. And we have no real way of knowing whether they're being used that way introduces a corrosiveness in the gears of democracy doubt about whether it's working well enough. And that's already bad enough. So we have a lot more work to do to figure out how to better protect democracy from these potential interferences. In prophecy, you say towards the end that in fact this is a book about AI ethics, but not about AI ethics.
Starting point is 00:36:19 So what can governments do, for example, with TikTok that's easy to execute to make these platforms less. dangerous? Is there anything that can be done by the end of this year, for example? Yes. One thing that could be done is forcing platforms to give up data to academic researchers, so we have a better sense of what's going on. Another thing that can be done is to ban algorithms that are designed to maximize engagement. Every algorithm is designed to do something. And we know that maximizing engagement is an incredibly toxic thing to do. It's the equivalent of putting cocaine or another kind of very addictive drug into food to make people buy that food. We don't allow it. We don't allow addictiveness to be part of the design of food. And in the same way, the algorithm
Starting point is 00:37:19 should maximize something else, should maximize entertainment or information or whatever else. But to maximize engagement, we know, leads to some of the most toxic patterns that we've seen. I think people are so confused about what an algorithm is. And when people say algorithm, they're just like, oh, it's just like the magic that lives in the hardware and the software that makes this stuff happen. Like, what would be a way to explain algorithms like through story? An algorithm is an instruction. And you ask it to do something for you. In some cases, it's to identify dogs in a bunch of images,
Starting point is 00:38:02 which can then be, of course, used to change that instruction and to identify a tumor for the purposes of medical healthcare. And when it comes to social media platforms, what the algorithm is built to do is to get you to stay on the platform for as long as possible, because that way platforms get more data out of you and then they can do things without data. And what do they do?
Starting point is 00:38:34 Mostly they try to predict what you're going to do next, which is, of course, influenced by the kind of content that you are exposed to. So one way to put it is that the easiest way to predict the future is to determine it. So if I jail someone, of course it's very easy to predict where he's going to be the following day. I jailed the guy.
Starting point is 00:38:53 In the same way, when a platform influences your behavior, it makes it easier to predict what you're going to do. Chatbots, by the way, are also built in this way. So they are built in a way that will make people want to use it more. And that is what makes them very sycophantic. They have this feature whereby they basically validate you as a user. And so they'll tell you that you're brilliant, that you're so insightful. And of course, validation is a pleasurable thing. And in small doses, it can be important and encouraging.
Starting point is 00:39:34 But too much validation makes us lose complete perspective. The idea of banning those types of algorithms to my cynical brain seems, I'm going to make a prediction almost zero, at least in the foreseeable future. It's a very top-down approach. What does the bottom-up emergent solution to this problem look like? It looks like either consumers being more demanding and saying, well, I'm not going to use this thing. That's just exploiting me. I think we're seeing some of that.
Starting point is 00:40:11 Granted, my students are not particularly representative of the population, but I have more and more students who don't use social media and who don't want to use social media. Also, companies doing better. So realizing that there's a competitive advantage in offering an actually safe product and an actually respectful product and people choosing it. And something important to bear in mind is that most of humanity's breakthroughs, whatever you consider a breakthrough, from coming up with the airplane and actually making the thing fly to human rights of all kinds,
Starting point is 00:40:50 including workers' rights and universal suffrage and all of that seemed absolutely impossible quite near to when they were achieved. That's something really interesting to think about. So coming off the algorithm discussion, we referenced self-fulfilling prophecies a bit. Let's go a little deeper into that because that's a big piece of this book. Tell us about that in general and what people need to know about how the future can actually be written by the people predicting it if we let it happen to us. So one example that is very relevant for the times that we live in is that once a population
Starting point is 00:41:32 starts thinking that there will be war, they start preparing for war and they make war much more likely to happen. Because of course it creates a reaction in the adversary of preparing as well. If you see that your neighbor is getting armed, then you arm yourself. And then once everybody's it makes a war much more likely. The self-fulfilling prophecies are all around us. There is, and there are many studies that validate how powerful they are. So one of the studies that I cite is one in which researchers get given mice and half the researchers get told that their mice are stupid for traversing a maze.
Starting point is 00:42:13 and the other ones get told that their mice are geniuses. And the researchers who have the geniuses end up having actual geniuses. But in fact, they got produced because all the mice were exactly the same. So the expectations of the researchers that their mice would be dumb or brilliant influenced how they treated them in a way that ended up fulfilling the label that these mice had. another example that is very real is how predictions can affect the outcome of health in medical settings so medical resources are always scarce and we have to make a decision of where they go first and this is made on the basis of predictions and if somebody predicts that a person
Starting point is 00:43:07 is less likely to benefit from resources the research would be allocated to someone else. And we will never know whether that person might have lived if we had given them the resources. And I tell the story of one case in which there was an organ donor in Spain who had spontaneous heartbeats of meaning that they weren't completely dead. But because they had already been branded as an organ donor, it was too late to do anything about it. And that example is particularly good because not only does it exemplify how high the stakes can be, but also how self-fulfilling prophecies tend to be the perfect crime, because they cover their tracks. We will never have that data.
Starting point is 00:43:56 And in the same way, when we use an algorithm to predict it's going to be a good employee, and we never give the job to certain kinds of people, we will never know whether they might have been brilliant employees. and so we get the sense that the algorithm is very accurate when in fact we don't have the data that would be most important to prove it wrong. I want to speak about my favorite part of your book, Povercy. Do you know who my favorite prophet is? No.
Starting point is 00:44:24 Jeremy, do you know who my favorite profit is? Maybe the guy who wrote the Culture Three novels. My favorite prophet is George Carlin. and I think maybe my second favorite prophet is Bill Hicks. You speak about Fulte Towers and you interviewed John Cleese for this book and I laughed that loud when you describe the scene in Fulte Towers where he has to choose the right branch to hit the car.
Starting point is 00:44:51 I think Fulte Towers is a masterpiece. Comedy and prophecy. Why do you write about comedy? I wanted to go all out in this book. It's partly about being older and thinking about how short life is and being willing to say what you think ought to be said. And not thinking about whether it's going to be commercial or whether it's a bit more. Let's say this book is post-tenure. and in many ways
Starting point is 00:45:34 I allowed myself much more freedom than in the past. So that's part. And comedy is a big part of my life. I love comedy and it influences how I see the world. But it's also something that I've learned is very important for politics.
Starting point is 00:45:50 So one of the marks of a government turning authoritarian or a society turning authoritarian is the loss of a sense of humor. And this is not new. Milan Cunderra wrote a brilliant novel called The Joke, which is exactly about this, about how in a communist regime when somebody made a joke, it could cost you your career and potentially your life. There is something very irreverent about comedy. And it's also very much against prophecy, because part of what makes something funny is when it surprises us.
Starting point is 00:46:29 We sort of expect a sentence to be one way, and then when it ends in a completely different way, it often makes us laugh. And going back into the past, I think part of the lesson of the book is that we've seen this before. Don't be dazzled by the new technology. This has happened before and look at this pattern. And it's easier to see a pattern of the past because you don't believe in those technologies anymore and because you have distance. And when you went into the court of a king, you realized that if you wanted to know the truth, you wouldn't go to the prophet. You'd go to the jester. The jester was the only person with the freedom to say the truth because it was under the veil of comedy.
Starting point is 00:47:16 And one of the parts that I enjoyed most about in writing the book is the part about Seinfeld. Seinfeld was one of my favorite comedy shows. Did you speak to him for the book? Did he agree to interview? I tried. Did not work out. Jerry, come on, Jerry. Still on offer, Jerry.
Starting point is 00:47:45 But yeah, so Seinfeld is a fascinating example because people hated the show. It was a show about nothing. It's a very funny episode about it. And nobody had seen anything like it. And so at the beginning, it was decidedly not what people wanted to watch. So if we had to use predictive algorithms to choose TV shows, it would have never chosen that show at the time. And part of its magic is that it changed the sensitivity of the audience.
Starting point is 00:48:16 It's an acquired taste. You learn to see the world through Seinfeld lenses and take pleasure in the absurd. and it became one of the most popular TV shows in history. And part of the brilliance of it is how they did not follow the cookie-cutter approach to sitcoms. And every time NBC gave them guidelines, they would say, we're not going to do that. And how the more we rely on predictive algorithms, the more we're losing that innovation and that boldness and that creativity and that humor. Yep. There is nothing like comedy to short circuit your serious gene.
Starting point is 00:48:59 I love it. I have to, this is such a dumb little addition, but I'm going to say it anyway. My wife and I went to see Seinfeld stand up. I actually was laughing so hard and for so long I thought I had to leave to like for my health. I couldn't stop laughing. My stomach was, it was crazy. But no, comedy has such a, such a beautiful way. to disarm people and the most brilliant comedians access that button and then deliver such
Starting point is 00:49:32 profound messages to it. So I think it's like a like Mark said, I think it's a great piece of this book that couples your this, this almost a reverent. I'm just going to say whatever the hell I want because I believe in it so hard and the message needs to be heard. It was a great, it was a great balance. Thank you. And I find that comedy is good for creativity. It kind of makes you loose. It makes you pliant. It puts you in a kind of state of mind that I think makes for better decisions and more insight. Give me one piece, one little thread of this book. And it could be a small little nugget when you were writing it. And you're like, this is very personal. This is very locked into my mindset that you would wish. someone reading the book would grab and want to ask you about. There's so much of it because for the first time, I told personal stories and I'd never done that before in writing.
Starting point is 00:50:36 And it's scary, but it's also nice and interesting. And partly comes from me being very grateful to authors who are willing to do that, to tell me where they're coming from, to tell me what kind of person I'm talking with. because a book is a kind of conversation between the author and the reader. And I think when you guard yourself too well and you come up with a pretended view from nowhere, there's a kind of distancing that you impose on that relationship that I didn't want to impose this time round. So perhaps one story that is quite personal and that makes me think about how everything could have been.
Starting point is 00:51:19 indifferent is about how when my grandmother fled Spain, she was too pregnant to go the whole trip by boat and then and then train. And so my family bought her a plane ticket from New York City into Mexico, but she was so pregnant that a flight attendant asked her whether she had a doctor's note that giving her permission to fly and she didn't. And so she had to get off that flight and and goes to the doctor at the airport for a note and that flight crashed and my family thought that she she was dead and how those unpredictable acts and unpredictable events change the whole of somebody's history. The personal nature of your stories in this book make the connection between author and
Starting point is 00:52:18 reader more impactful, I think, as I was reading the book. I got a sense for that as well. I felt like I had to act more reading it. It felt like it was the batteries that I need to, I don't if fight back is the right way to describe it, but to be conscious of what is happening. And it gave me the fuel for that. In that sense, I think one of the stories that I find inspirational is the one about Caricot, the Nobel Prize winner who helped come up with the RNA vaccine and the obstacles that she had to face and how she persisted in the face of being ignored by academia and being harassed in academia and being demoted and she just stuck with it because she knew she was right and she was right and how much talent are we losing out on because people
Starting point is 00:53:18 might not be as persistent. And how can we make things easier for that kind of person to thrive? Because we need that talent. We benefit a lot from that vaccine. This book was a spark for me as it was for Mark. And I hope other readers as well to kind of grab it and feel empowered as an individual.
Starting point is 00:53:40 Because so often in society, we're just a piece of the puzzle. And we think it goes back to the business of busy. And, you know, oh, I can't affect this. I can't affect that. This power is too much and this is fuel. So let's talk about some of these takeaways that you have in the end that I really love. I'd love to pick out a few that resonated with me, Mark. And Mark, feel free to jump in on your side.
Starting point is 00:54:06 Let's talk about increased serendipity. That one just talk us through that. And let's unpack it a little bit. That's my favorite. I was thinking about that one exactly now. One of the downfalls of the tyranny of predictions is that you get told where to go, where to be, and who to meet. And if you just follow that, then somebody else is deciding what you were exposed to. And one of the ways to fight against that tyranny of prediction is to increase your serendipity.
Starting point is 00:54:41 And how do you do that? Well, essentially by being open to the world, by reading widely, So read things outside of your comfort zone and outside of your field of expertise. I've gotten so much out of books. You can only live once, but if you can read, you can live many lives in one life, and that is an incredible gift. But also meet people, talk with strangers. The more I read about difficult political times in the past,
Starting point is 00:55:10 I realize that one of the most effective weapons of bad governments, is to erode the relationship between citizens. And part of how we fight back against that is just being comfortable with strangers, that strangers around you, for the most part, are probably friends you just haven't met yet. And to have that comfort of striking up a conversation with a stranger and learning something new.
Starting point is 00:55:37 And being curious about the world, being curious about what it's like to be someone else, being willing to rethink things. chair have to look like that? What if it looked like something else to innovate and design from scratch? And in general, the metaphor that I like the most is just take strolls along the beach because you never know what the tide might bring. Beautiful. My favorite is to go home and watch Fulte Towers or watch Seinfeld or put on an old Bill Hicks video, put on some comedy, get on on that. I will choose one because I think there's a strange
Starting point is 00:56:18 link between this one and some circles in big tech. That stoicism has become the go-to philosophy for many in, not just in Silicon Valley, but on the internet, it's almost like the, this, the prophecy runs hand in hand with stoicism. Live in the present. Could you speak to that one? Yes. And even though I sympathize with a lot of what stoicism, has to say, in the book I make the case for Epicureanism because they share a lot of the good stuff.
Starting point is 00:56:55 So both of them think that part of what makes life good are the simple pleasures in life and tempering your desire and your fear. And one major difference between Epicureans and Stoics is that Epicureans were a lot more democratic. And they, Epicureus famously had a garden in which everyone was welcome, including slaves and women, and they thought against Stoics that there was no fate, that even though there are elements that constrain our freedom, there are some things that still allow us a lot of leeway to write our own life, whereas Stoicism is much more about accepting your fate and believing in divination. And that makes for a huge difference. Another thing that
Starting point is 00:57:48 I really like about Epicureanism is its emphasis on friendship. And I think in Western society, we value a lot of the family, and of course I value family as well, and we value professional relationships, but I don't think we give friendship, including civic friendship, friendship between citizens, the value that it should have. And the basic idea of Epicureans is that deep down, what people really fear is death. And if you can have the courage to look death in the eye, to accept that we're mortal beings, and to lose that very extreme fear, to know that when death comes, you won't be there anymore.
Starting point is 00:58:44 And so there is less to fear. you're much less vulnerable to profits and much more easily willing to live in the present because if you're not afraid of what's to come and you're not desiring things that you can never completely satisfy so Epicurious makes this distinction
Starting point is 00:59:08 between desires that you can satisfy if you're hungry eventually you will be satisfied or if you're thirsty whereas if you are lusting over fame or fortune, you will never be satisfied because you could always be more famous and you could always be richer. So once you temper the fear, which at the end of the day is fear of death and desire
Starting point is 00:59:29 by focusing on desires that can be fulfilled, then the prophet has nothing to offer you because you're living in the present and enjoying yourself. And when you're reading a good novel or in a comedy show or really enjoying a conversation with your friend, who cares what the prophets is prophesizing? Let them talk. They lose their power over you.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.