Sean Carroll's Mindscape: Science, Society, Philosophy, Culture, Arts, and Ideas - 207 | William MacAskill on Maximizing Good in the Present and Future

Episode Date: August 15, 2022

It's always a little humbling to think about what affects your words and actions might have on other people, not only right now but potentially well into the future. Now take that humble feeling and promote it to all of humanity, and arbitrarily far in time. How do our actions as a society affect all the potential generations to come? William MacAskill is best known as a founder of the Effective Altruism movement, and is now the author of What We Owe the Future. In this new book he makes the case for longtermism: the idea that we should put substantial effort into positively influencing the long-term future. We talk about the pros and cons of that view, including the underlying philosophical presuppositions. Mindscape listeners can get 50% off What We Owe the Future, thanks to a partnership between the Forethought Foundation and Bookshop.org. Just click here and use code MINDSCAPE50 at checkout. Support Mindscape on Patreon. William (Will) MacAskill received his D.Phil. in philosophy from the University of Oxford. He is currently an associate professor of philosophy at Oxford, as well as a research fellow at the Global Priorities Institute, director of the Forefront Foundation for Global Priorities Research, President of the Centre for Effective Altruism, and co-founder of 80,000 hours and Giving What We Can. Web site PhilPeople profile Google Scholar publications Wikipedia Twitter

Free is great, but only if it's useful. Free credit scores from some apps can differ by as much as 100 points from your actual FICO score that 90% of top lenders use when you apply for a credit card, personal loan, car loan, or mortgage. That can mean a higher interest rate, a bigger monthly payment, or worse. Denied. My FICO gives you your actual FICO score, the score lenders use straight from the company that created it. For the moments that matter, get the score that matters, your FICO score. Visit MyFICO.com and get started for free today.

If you're a QuickBooks customer looking to grow your business without the growing pains, you need the Intuit ERP. Upgrade to Intuit Enterprise Suite in a matter of hours. It's the AI native ERP from the makers of QuickBooks. Learn more at Intuit.com slash ERP. Hello, everyone. Welcome to the Mindscape podcast. I'm your host, Sean Carroll. I don't know about you, but sometimes it's enough to just get through the day, right?

There's a lot of things going on in our individual lives, in the wider world. just staying afloat is a bit of an effort. But we would like to do more than that. We would like to take a step back and really think about how should we behave as good, moral, rational, nice people here in the world. Of course, if you've been listening to the podcast at all, you know that even asking this question is not enough, because the possible answers about how we should behave, what it means to be a good person, to be moral, to be ethical, not clear. We have not only disagreements about meta-ethics, about how we should justify our ethical beliefs, but about what the actual ethical beliefs are. So today's guest, William McCaskill, who goes by Will,

is someone who is really leading the push for a very specific way of thinking about how we should act, how we should be a good person. It's within the tradition of utilitarianism, consequentialism, okay, the idea that whatever we do, we should judge its moral worth by what consequences it has, what affects it has on the world. And that's not obviously true. I mean, it's plausibly true. It's a very reasonable thing to do, but it's not the only way to go. There's alternatives about deontological views where it's a matter of following the right rules, whatever the consequences might be, virtue ethics positions, which say that it's about cultivating your virtues, not precisely about how you act and everything, but certainly utilitarianism is a big player in this game. And Will has been one of the

most influential in really turning utilitarianism into a philosophy of action and thinking about how it should apply to real world problems, especially sort of big picture problems. He's a leader of the effective altruism movement. We've had other interviews with people like Josh Green talking about the best ways we can be charitable, but then also we've had supporters of the Mindscape podcast, Give Well and the 80,000 Hours podcast have asked. I've advertised here on Mindscape about ways in which we could be better altruists, okay? How, when we do want to give some money away to make the world a better place, what is the best possible way to do that?

In his new book called What We O O the Future, Will is turning his attention to a specific question within how to be a good person, which is how much should we weigh the lives and statuses and situations of future people in our. calculations. So there's a standard trick in economic theory where you try to decide, you know, what good to do, which is discounting the future, which is to say that, roughly speaking, we don't count future situations as much as present situations. And part of that is just, we don't know what the future situations are going to be. Now, the counter argument to that is, okay, sure, we don't know, therefore there's some uncertainties, and we should take those

uncertainties into consideration. But there's an awful lot of people who could exist. in the future. So this is intellectually fascinating to me because it's a classic example of taking a big number and multiplying it by a small fraction, right? And then the big number is the possible number of people who might live in the future, and the small fraction is the probability that we're correctly getting right what our current actions are going to imply for their conditions. Sometimes we know, sometimes it's pretty obvious like that destroying the climate and the atmosphere and the environment of the earth is probably worse for the future than keeping it in good working order, right? Other times it's less clear, you know, is super intelligent artificial

intelligence a big worry? What are the risks from pandemics? They're certainly there, but how big are they? All these questions. So Will has thought very carefully about all these questions. In fact, he did allude to in this podcast something I wish we had more time to go into, which is his PhD thesis, which is not that long ago. He's a very young guy. was about basically, if you don't know the right moral theory, if you're not sure, whether it's utilitarianism or virtue ethics or deontology, you can kind of split the difference. He developed ways to sort of take the average of the recommended actions given by all these different points of view. And so that's, it's a crucially important question because it sounds all abstract and there's math involved and things like that, but it has potentially enormous consequences for what we do here on Earth right now. where we spend our money, our attention, our concern. And, you know, honestly, if you listen to the podcast, you know that I am not convinced by utilitarianism. I think that there are issues there that kind of get swept under a little bit by its advocates. And so we talk about that with Will,

and I'm not anti-utilitarianism either. I'm open-minded about these things. And so this was a conversation that definitely gave me lots of food for thought in thinking about these questions, which I repeat are super-duper important questions that we should all be thinking about. Whatever your views on long-termism itself are, it's an important question, one that matters to us going forward in the world. That's what we do here at Mindscape. So let's go. Will McCaskill, welcome to the Mindscape podcast. Thanks so much for having me on. I'm a big fan. Oh, thanks. And I'm a big fan too. I'm a big fan of, actually, I'm going to disagree with some of your conclusions, but in a very soft way. Like, this is stuff I don't have fixed.

opinions about. And I really am interested in learning more in hearing how you think about it. But I'm certainly a big fan of the idea of thinking deeply about morality and how we should think about it, because I suspect that in this world that we have of technology and dramatic change, our inborn moral intuitions aren't always up to the task of guiding us as a civilization. I completely agree with that. And so your idea, about how we should be thinking is this thing called long-termism. So let's just start with the basic thing. So what is long-termism?

And take as much time as you want. Tell us why you think that this is the way that we should be thinking about how to guide ourselves from moment to moment as we do live here in the world. Thank you. So long-termism is a view I defend in my upcoming book, What We O the Future. And it's about taking seriously the sheer scale of the future and how high the stakes are in potentially shaping it,

then thinking what are the events that could happen in our lifetime that really could be pivotal for the course of future human trajectory, and then ensuring that we navigate them well, those events, so that we can help think about a just and prosperous and long-lasting future, the sort of thing that our grandchildren's grandchildren will thank us for. I mean, our grandchildren's grandchildren, but also there's talk in the book about more cosmic perspectives, right? Millions, billions of years. Yeah. So it's taking, one thing that's distinctive about this view is that it's taking the whole course of the future seriously. So long-termism

really means long-term. Not just thinking, oh yeah, years, decades into the future, but really for how long we might live, which I think is thousands, millions, or even billions of years. So, yeah. I was just going to say, you know, in some sense, sorry, there's a delay between us, so I'm going to keep interrupting you. I don't mean to do so. But In some sense, sure, of course, we should care about the long-term future. But I think that you're trying to make a slightly stronger claim than just the thing that everyone would agree that the future does matter in some sense. You want to say that it matters a lot. It matters a lot.

And so why might we have that view? Well, a common view in model philosophy is that acting or thinking modeling is about taking a certain sort of stance, a certain perspective on the world, what the philosopher Henry Sidgwick called looking at the world from the point of view of the universe. So no longer just from your own particular point of view, but from this objective impartial standpoint. And so when you take this point of view of the universe, what do you see?

Well, the universe formed 13.8 billion years ago. Earth forms 4.5 billion years ago, first replicators, 3.8 billion. Ucuroats, 2.7 billion. First animals about 800 million years ago. Listeners of your podcast, I'm sure I'll be familiar. Human beings 300,000 years ago. The Cultural Revolution 12,000 years ago,

and scientific revolution really just a few hundred years ago. And there's a number of striking things here. One is just how early we are in the universe. Second is how there we seem to be. A third is how fast things are changing now compared to the billions of years that have come before. before this. And the final is that maybe if anything, things are kind of speeding up in terms of these great transitions. But we can also ask just not just so what's so distinctive about

the present, but what does the future of the universe look like? Well, here are kind of two ways in which things could go, like how might the story of the universe unfold. On one, humanity continues to advance technologically and modally, and someday we take to the stars. And at this point, it's no longer, the story of the universe is no longer just kind of governed by, you know, the really fundamental physical laws about how planets orbit stars and stars form. But you would start to see human agency is telling like a significant part of the story of what happens. You have these interstellar settlements. You have perhaps you have galactic megastructures, humans intervening to keep the sun smaller

so that it can burn longer, perhaps even for trillions of years. perhaps lowering suns into black holes as power sources, perhaps, you know, constituting solar systems into giant computers. That's one way the kind of future could go. And that would last for a very long time indeed. The last stars would burn out after hundreds of trillions of years. Actually, a small but steady stream of collisions of brown dwarfs would make that last even longer, I think a million trillion years.

But there's another possible future, which is that humanity does not, not get to that point of developing technologically. Instead, we cause our own extinction. And then the story of the universe would be much as the story of the past. It would be stars forming and stars dying. It would be galaxies colliding over time. But the universe would be empty. It would be lifeless. It would be dead. And so one of the ways into thinking about long-termism is taking that perspective of humanity as a whole, a prethieu. that, wow, actually the, what we call history, or even long history, 2,000 years since in the common era, is just this like tiny sliver of time. And there really are some things

that could, that we could do now that could have this course on humanity, this huge impact on the course of humanity's future. That's of enormous importance just because of the sheer time scales involved, because humanity's lifespan might be truly vast. And one of the ways in which we can impact that very long-term future is by ensuring we have a future at all by preventing human extinction. There are other ways as well. I mean, I think as well as ensuring

that we have this kind of long future that on earth would be hundreds of millions of years, if we could take to the stars, would be billions more, but also ensuring that it's good, ensuring that society is actually one of, you know, flourishing rather than some authoritarian perpetuation. actual nightmare. And so that's the kind of insight and background into this idea of long-termism, is taking that perspective seriously, thinking, wow, this is just huge stakes of this tiny

sliver of time that we have. The future is going to be wild, and we should really at least be morally thinking what are the things we can do that might be impacting the long-term, and how can we make them go better? I think that most of our conversation will not be very science-fiction-y, but let's lean in a little bit to the science fictiony aspects here. I mean, you're talking, I think in that explanation, talked about the future of humanity, but it might not be humanity 10 million years from now, right? We'll be a very different species,

not even taking into account the idea of technology and are blending with that. So is it really something else like consciousness or awareness that you're trying to help out? Yeah, like, I mean, I think there's nothing, you know, distinctively special about homo sapiens. I think that all animals with consciousness have moral worth. And so I also am a big proponent of animal welfare

and reducing like factory farming. But another thought here is, yeah, in the future, there could be very different beings. Perhaps they're artificial intelligences. Perhaps we've just, the path of human evolution has continued. You know, I would call them humanity in the sense that if at least they're representing values and pursuing goals that we think of as being good ones, where if there's a future society that includes pursuit of knowledge and love and friendship and happiness

and scientific accomplishment, even if that's being done by beings that are somewhat different from us, I would still think that's a morally worthwhile, something modally we should care about. And you gave us two options, one of which was basically going extinct and the other was spreading to the stars. Is there any third option where we more or less decide we're happy here on Earth and have some flourishing existence for the next couple billion years? Absolutely. I mean, I think, you know, I sketched this potential future of spreading to the stars, but if we do, if we played our card lights, the card light, the future would be one that, you know, we choose for ourselves. And on the basis of, you know, what we think is more

good and morally light. And perhaps in the future that we would decide, like, actually, you know, the universe is a big place, but we want to leave it, pristine. That's a good future we could have. And even there, even just staying on Earth, we have hundreds of millions of years before the Earth is no longer habitable. And billions of years before the sun,

before our son dies. So even on that much more, modest scale. We have this enormous feature ahead of us. Just to get as science fiction as we're going to get, please, yeah. You talk about love and the search of knowledge and things like that. And I agree, these are important things.

But once we start extrapolating a billion years into the future, how plausible is it that those are not going to be the important things anymore? Like I can at least vaguely imagine in my mind that if we did upload ourselves, into the matrix. All of those values that we have,

all of the goals and desires that we have as organic beings turn out to be intimately connected to our embodiment in a biology that wants to eat and reproduce and so forth. And we lose interest in those things that were so important to us when we were just biological organisms.

I mean, this is one plausible, at least conceivable, resolution to the Fermi paradox, right? You know, every technology, every civilization becomes so technologically advanced that they fulfill all their needs and become less ambitious. Do you give any credence to that possibility?

So I think as a solution to the Fermi paradox, I don't give it enough credence where for there to be some filter on the evolution of life, from something like an Earth to very technologically advanced, species kind of spread out across the galaxy. Because there are just so many planets and solar and solar systems, you need this really hard

filter. So it would have to be that 99.999% of civilizations decides to be unambitious. And it seems hard for me to believe that it could be that high. But you do raise a very important point, which is that the values that guide the future might be, in fact, probably will be quite different from the values that guide society now. And that could be very good or it could be very bad. So it could be very good if we've made moral progress where, you know, from the perspective of the Roman Empire, the values that guide society or today are very different. We don't own slaves.

Where has our honor gone? Where wimpy, women have political office. But this is all very good thing because it seems like this is kind of moral progress. And that could continue into the future, such that our distant descendants have made further model progress. And if they value different things and pursue different goals, but if we could have a conversation with them with sufficient time, they would convince us that those goals were actually admirable in the way that hopefully with sufficient time, perhaps I could convince at least, you know, maybe I couldn't convince Nero or Caligula, but I could convince Aristotle, I think, that earning. slaves is morally wrong. Okay. So that's one way it could go.

But then the second is that it could be just dystopian. Like perhaps we've ended moral progress too soon, perhaps some fascist ideology just took over and entrenched its values indefinitely. Or perhaps it's just some very alien set of goals because we made some mistake perhaps on the development of AI and the AI systems themselves took over and pursued some just fundamentally valueless goal. And that's a set of worries that I'm very concerned about, I think gets neglected. And I call value lock-in, this idea that certain bad values could take control and then just intents themselves, keep themselves there indefinitely.

Now I'd like to introduce you to Meaningful Beauty, the famed skincare brand created by iconic supermodel Cindy Crawford. It's her secret to absolutely gorgeous skin. Meaningful beauty makes powerful and effective skincare simple. And it's loved by millions of women. It's formulated for all ages and all skin tones and types, and it's designed to work as a complete skin care system, leaving your skin feeling soft, smooth, and nourished. I recommend starting with Cindy's full regimen, which contains all five of her best-selling products, including the amazing youth-activating melon serum. This next-generation serum has the power of melon-leaf stem cell technology. It's melon-leaf stem cells encapsulated for freshness and released onto the skin to support a

visible reduction in the appearance of wrinkles. With thousands of glowing five-stom, our reviews, why not give it a try? Subscribe today and you can get the amazing meaningful beauty system for just 49.95. That includes our introductory five-piece system, free gifts, free shipping, and a 60-day money-back guarantee. All of that

available at meaningfulbony.com. Hey, everyone, it's Cal Penn. I'm the host of Earsay, the Audible and I-Heart audiobook club. This week on the podcast, I am sitting down with Ray Porter, the narrator of Andy Weir's audiobook

Project Hail Mary, massive sci-fi adventure about survival and science. And what happens when you wake up alone very far from Earth? I really had to make a decision because I caught myself getting that frog in my throat and starting to get teary as I'm narrating some of these sections. And it's like, okay, yo, yeah, yo, is this indulgent? And I really thought about it. I was like, no, at this point, it would kind of be betraying the trust the author and the listener have in telling this story if I don't go through it.

But there's places in this book that deeply emotionally affected me, and I left it on the mic. That's great. Because it served the story. People will say like, oh, my God, I cried at the end. It's like, yeah, dude, me too. Listen to Earsay, the Audible and IHeart Audio Club on the IHeart Radio app or wherever you get your podcasts. Okay. Thank you for indulging my science-fiction-y questions because there's a lot of pretty down-to-earth moral and ethical and political questions involved with the philosophy.

of long-termism. So am I right to contrast long-termism with a more conventional economic strategy of temporal discounting? In economics, we know there will be people in the future, and we do count their value a little bit, but we count it less as a very explicit move. And it seems that long-termism is saying that's a mistake. Broadly, yes. So there are some good reasons for discounting the future. one is if you can invest money that you have and get a return on it, then money earlier is worth more because you get further years of investment returns. Or if you think that you as an individual or society are going to be a richer in the future, then you should discount just because money has diminishing the terms.

So, you know, people are poorer today than they are in 25 years. You might think in the same way that we today are richer than we were 50 years ago. But then there's this final reason that economists discount the future, which they call the rate of pure time preference, where you just say a given benefit or harm, and not just talking about money, but talking about anything, is intrinsically worth less year on year. And a typical value they might give is like 2%. So a happy day today is worth 2% more than a happy day in a year's time. But this gets really absurd once you start considering. I mean, I think it's absurd at any point in time. But I think it gets really absurd once you consider long time scales.

So suppose I can give you a model choice. You can prevent a single death in 10,000 years, or you can prevent the genocide of a million people in 11,000 years. And suppose there are no other kind of knock-on effects of these two things. Which do you think you ought to do? I'll just be the naive answer here and say I would prevent all the largest number of deaths. Well, I think you've got the correct answer there. Saving a million lives is more important than saving one life, even if the million lives are in 11,000 years and the one life in 10,000. But if you have this rate of pure time preference that economists sometimes use of 2% per year, then you think it's more important to save the one life in 10,000 years than the million lives in 11,000 years. And that is absurd. I think everyone would agree all the economists as well, who are normally discounting of a much smaller, timeframes, you know, a few years or few decades, we'd agree that's absurd as well. And in the very long term, we should not discount the future in that way. There's a, at least an analogy to

sort of spatial discounting. And this has been one of the emphases Peter Singer and other people have put on how we should think about morality, that we tend to value ourselves and our friends and people near to us more than people far away. And maybe we shouldn't. It's sort of spatial discounting. And is that a good analogy? Or is it basically perfect or is it very different considerations? I mean, I think it's a great analogy in particular because the distinction between space and time is not that clean cut. And if you disc – I mean, it gets a little technical, but if you want to discount across time but not across space, then you get into all sorts of paradoxes because of general relativity. But on a fundamental ethical level, look, it just seems obvious that the well-being of someone

doesn't diminish if they move further away from you in space. Someone on the other side of the world matters morally there. They have a life too. They have interest hopes, joys, fears, they suffer. And if you can prevent that suffering or avoid harming them, you ought kind of modally to do so. And similarly, just the fact that someone lives in a sense, centuries time or a thousand years time or even a million years time should not matter when we're considering kind of how important are their interests. Is it wrong to harm them? So this is where

my skepticism begins to arise. I keep calling it skepticism, but that's too strong. My lack of clarity begins to arise. I get the kind of philosophical appeal of the idea that lives matter equally, no matter where they are, no matter when they are. But as a practical matter, I mean, you know, Singer's original examples of saving someone who's drowning in a pond in front of you versus saving someone who is in a different continent, it seems to be perfectly okay

to save the person that you're looking at and worry about that, care about that more than someone who's far away, just because I know terrible things are happening far away, and I'm not in favor of it. I would like them to not happen, but the actions, going to take in response to them are just not going to be of the same order as the actions

that I see happening right in front of me. And maybe that's okay. What side do you come down upon that? Yeah, I want to say two things. So one is that I do think we have some extra reasons to care about people in the present generation compared to people in future generations. And in the same way, I have special reasons to care about people near and dear to me compared to distant strangers. So a couple of reasons. One is just special relationships. If you're my mom, that gives me an additional reason to care about you. Similarly with friendships, possibly with compatriots. That's, you know, lay me a little bit less clear. But I think everyone, you know, is a very intuitive view that if you can save one person from dying or two people from dying

and the one person is your spouse or a family member, then, yeah, it's fine. to save the one. A second reason is reciprocity as well. So by being embedded in society, people are given me enormous gifts. Education I've received, again, there's compounds for special obligations I have to my parents, also to like, you know, teachers that have helped me,

all these sorts of benefits, also just benefits have received in the state, all these sorts of benefits, and I have some kind of obligation to repay them. But people in distant nations, they still matter a lot. I still are not justified in harming them in order to even to benefit my family. And insofar as I have disposable income, money I would otherwise spend on luxuries,

like, you know, nicer clothes or nicer house or things I don't really need, then that's not conflicting with these obligations of their suplosity or partiality, and I can help them. And I think the same is true when we look at future people. So I don't have special relationships with them. in particular. Perhaps if I have children, I'll have great, great grandkids. But perhaps that gives me some extra reason, but that's not true for most future people.

And I certainly don't have a relationship of reciprocity with them. Perhaps I have some sort of passing the baton obligation where past generations have benefited me enormously through innovation, through model change that people thought for, through laws and political systems that have been developed. perhaps that gives me a reason to pay it forward. But I think the fundamental thing is that people, we owe what people, at least, we owe people in the future at least what we owe strangers in the present. And that might not be as much as we owe our near and dear,

but it's still an awful lot and it still really makes a big difference for how we act to modely, I think. Well, there is at least one disanalogy in that the people who are far away exist. And the people who are in the future may or may not exist. right? I mean, there's some sort of uncertainties involved in thinking about caring for future generations that are less of a worry when we're just caring about people who exist right now but are far away from us. Sure. So yeah, uncertainty is huge, of course. And in what we owe the future, essentially the whole book is just addressing the issue of uncertainty. I think a very reasonable

starting point of skepticism is, well, can we really affect any of this, like in a predictable way? And, you know, I spend five chapters, then argue, yes. But even then, it's still thought with uncertainty. But that, I think, doesn't undermine the fundamental model issues. So, you know, Singer has this thought experiment. You can walk into a shallow pond and save a child's life. It'll ruin your suit. Now, suppose you're not certain it's a child.

You think it's probably a child, but could just be a, you know, a boy or something, a buoy, as you would Americans would say. I think you should still go in and save the kids. Like uncertainty is often used as a justification for inaction, but I think it's a very bad justification. So in climate science or the discussion around climate change, people say, oh, well, we don't really know what the impact of climate change will be exactly. And that's true to an extent, but it doesn't justify inaction.

I think if anything, it justifies further action than we might otherwise take, because things could be much worse than we think. And similarly, when thinking about the future, it's like, okay, yeah, we don't know that people in the future are going to exist. That gives you something of less of a reason. But also, things could be very different from how the present day are. Things could be much, much better if we play our cards like, or much, much worse, in fact, if things go in a dystopian direction.

And so, if anything, uncertainty, I think, gives us more reason to act, more reason to take a cautious responsible approach to the future rather than just winging it. Yeah, no, I completely 100% agree that uncertainty by itself is not an excuse for inaction. I mean, we have to do our best, right, to weigh the uncertainties and balance them against rewards, et cetera. But that is hard to do, I guess, is definitely a lesson that we should keep in mind. But, okay, let's just be as dramatic as we can, you know. You've been very active in thinking about the possibility of existential risks. So you started by contrasting a glorious future

of life in the stars and exploration and love and discovery versus extinction. So how big of a worry is extinction? Let's grant that it would be bad. But lots of things would be bad. You know, I could quantum tunnel through the floor of the building I'm on right now. It's just so unlikely that I don't worry about it. How likely or unlikely are these existential risks? Are they, things that should affect our day-to-day planning or at least our social planning? I think they should certainly affect our social planning. So I'll focus just on pandemics as one source of existential risk where we as a community were really worried about pandemics for many years, you know, making donations in the area, encouraging people to go into career choices

from 2014 onwards. And people didn't pay all that. much attention. Unfortunately. But I think, and we got the COVID-19 pandemic, showed us that we are not immune in the modern world to a pandemic. But in many ways, COVID-19 was much less bad than it could have been. So it's an enormous colossal phagody, but the virus could have been more deadly, it could have been more infectious. And when we look to developments over the coming decades, I think we have reason to worry to expect things to get potentially worse because we don't just have to conflict with wrestle with natural pandemics, but we'll increasingly be able to create engineered pandemics, namely by creating new viruses, new pathogens, or upgrading

existing ones. And that could really wreak destruction on an enormous scale that I think it's unlikely. I think it's, you know, one percent or less that it could reach this within our, within the coming century, that it could reach the scale of killing everybody. But one percent, that's, you're more likely to die of that than in a fire or from drowning. If you got on a plane and it's like, oh, only one in a thousand chance of dying on this plane. I don't think you get on it.

And that means that we as a solidarity issue. And then, so it's a real, you know, not. negligible risk and the stakes would be extraordinarily great. The work that one does to prevent pandemics, which I'm happy to talk more about, has enormous other benefits in the near term too, in such that often this work is just justified without even having to think about the long-term future. And it does, you know, it has particular policy prescriptions. It means we should perhaps be more cautious about certain areas of biotechnological advancement. I think it should be means we should be even more concerned about the potential outbreak of a

third world war or some other war between great powers where I think the risk of using bio-weapons and engineer pathogens goes up a lot in wartime when people start doing very dumb things. Is there something about our current moment historically and moment broadly construed, you know, plus or minus a few hundred years, that is especially perilous? You know, we've come across new technologies in the last hundred years that we didn't have before. And our threat to ourselves is greater than ever before. Is that just the permanent future condition and we've had a phase transition? Or is it a temporary case where we're going to become safe again down the road? I think there's a good argument for thinking it's temporary, or at least that the rate of

technological change we've seen can't persist indefinitely, and that we're living through can't persist indefinitely, where, you know, we've only had really rapid rates of tech progress for a few hundred years. But again, the, you know, even on earth, the human race could survive for hundreds of millions of years. Well, if we have this continued rate of technological advancement, even for thousands of years, what happens? Well, how do you measure tech advancement is hard, but let's just use the kind of economic measure, like contribution to economic growth, of again, about 2% per year globally.

Well, we have an economy of a certain size now. continuation of 2% per year growth for 10,000 years means that the economy would be, I think, 10 to the power, 89 times as big as it is today. There are 10 to the power, 67 atoms within 10,000 light years. So it would mean that we would have this economy of, you know, a trillion civilizations for every atom we could access. And it's just like, it's just not plausible that we're going to have that much

technological advancement. So things have to slow. And so what this means is that we're living through a period of unusually fast technological progress, unusually fast compared to history or compared to the future. And if every technology is just you're reaching into an urn and you're drawing a ball, and most of the time it's just like, it's great. Technology in general has just been enormously important in terms of improving our lives. But you get dual-use technologies.

So nuclear fission, well, could have and could still give us abundant clean energy, but it also gives us the power to develop nuclear weapons. Similarly, advances in biotechnology, I think, could do a huge amount to eradicating disease, but could also help us create new pathogens. I think similarly with artificial intelligence could bring about a new era of prosperity, could also enable dictatorial lock-in or result in a future where artificial systems themselves control things and human values are kind of out of the loop.

So that's why I think things are particularly important now. And then will we ever get to a state where we can just reduce existential risk to zero? Basically, I don't know. If we can't, then we've got no future. We're kind of doomed anyway. But maybe we can. Maybe we can get to a position of what my colleague Toby Ord calls

existential security, where we've actually just matured as a speech. We've figured stuff out. We've managed to get these risks, all the relevant risks down to zero, and then we can continue with civilizations trajectory. And if you think that's 50-50, well, I think those are odds worth betting on at least. Hey, everyone. It's Cal Penn. I'm the host of Earsay, the Audible and I-Heart Audiobook Club. This week on the podcast, I am sitting down with Ray Porter, the narrator of Andy Weir's audiobook Project Hail Mary, massive sci-fi adventure about survival and science, and what happens

when you wake up alone very far from Earth? I really had to make a decision because I caught myself getting that frog in my throat and starting to get teary as I'm narrating some of these sections. And it's like, okay, yo, yeah, yeah, yo, is this indulgent? And I really thought about it. I was like, no, at this point, it would kind of be betraying the trust the author and the listener have in telling this story if I don't. go through it. But there's places in this book that deeply emotionally affected me, and I left it on the mic. That's great. Because it served the story. People will say like, oh my God, I cried at the end. It's like, yeah, dude, me too. Listen to Earsay, the Audible and IHeart Audio Book Club on the IHeart Radio app or

wherever you get your podcasts. Well, okay, good. So now we're getting into the slightly more technical questions here because a lot of this seems to, and maybe it doesn't, maybe I have a misimpression, but it seems to depend on these probabilities. I mean, you said, okay, 1% chance of a pandemic that would wipe out all of humanity. And I completely agree that 1% for that particular kind of number is enormously big and we should worry about it a lot. But if it's a 10 to the minus 10 chance or a 10 to minus 100 chance, then maybe things are different. I don't know.

So, like, how do we know what these crazy small probabilities are? There's a bigger question here that I always worry about, even though I'm a very good Bayesian, I think that's how you should think about things. Multiplying really small probabilities times really big effects is just always really dangerous to me. I mean, have we learned about a better way of doing that, or do we just do our best and cross our fingers? So on the issue of tiny probabilities of very large amounts of value, I just completely agree.

If the arguments I was presenting were about 100 trillion, trillion, trillion chance of such enormous stakes, then I'm like, I would not have written this book. I'd be doing something else. I'd be floating something else. And actually, many of my colleagues at this Institute, Global Priorities Institute, I helped to set up, have worked on this very topic of how do you handle tiny probabilities of enormous amounts of value. And I'm sorry to say that the big results have been impossibility results. So just showing that no plausible.

view, in fact, proving that no plausible view, sorry, no view you can have on this topic has implications that are just generally plausible. There's just, you've got some, like, very unintuitive implications for all such views. However, thankfully, we're not in that world. So it's kind of philosophically interesting. But the probabilities of the risks we're facing are really pretty sizable, you know, of just the same sort of magnitude as risks from, you know, dying in a car crash, higher than the risks of dying in a plane crash, exactly the sort of, you know, threats that we typically worry

about. So my, again, a colleague Toby Ord puts the overall chance about one in six of existential risk in our lifetime. And I don't significantly disagree with him on that. I have like some differences in how we might lose most future value, but not enormous differences in the fact that, yeah, the size of these risks really are very great. And then when you consider the fact that most of the work you're doing to reduce these extinction risks are also helping to reduce just catastrophe. So what's the chance that we get a pandemic far worse than COVID-19 in our lifetimes? I'd say considerably worse than COVID-19.

I'd say one in three, perhaps. What's the chance of like a World War free? You know, war between great powers in our lifetime. Again, I'd say at least 25%. Could even argue to go higher. You ask the question of how do we even assess these probabilities? And it's super hard. And I think there's just enormous amounts of work we can do

to try and get these more and more precise. But one thing we can do to start making estimates a little more reliable is the kind of what's called the art and science of forecasting, where you get lots of individuals who each come up with their kind of own best guess judgment on the probability of something or some sort of forecast. And then you use certain algorithms to aggregate those forecasts into kind of an all things considered prediction. And so there's a community prediction platform called Metaculous that does this, but like in a, among kind of aggregating predictions among many people. Back in 2016, I think it made a prediction, what's the chance of a pandemic that kills at least 10 million people occurring between 2016 and 2026? And it gave one in three was the answer. And if you thought it was one in three, we would have taken.

taken a lot more precautions that would have done pretty well. And so then when I said the risk of a biological catastrophe was something like 1% maybe a bit lower, that was again appealing to Metaculous, where its estimate of the chance of an engineered pathogen that kills at least 95% of the world's population is about 0.9%. And then like what fraction of that goes to extinction? It's hard to say maybe it's 0.5% in total. of extinction. So, you know, I'm trying to use the aggregate of many, many people each having remote takes. But I think this seems a little bit different than pure long-termism. I mean,

sure, I do want to avoid the extinction of the human race. And it is absolutely true, as you say, that many of the things we can do to avoid the extinction of the human race are good for other reasons. Like, you also want to avoid the extinction of half the human race. But how do we connect that, or let me put it this way, why do we need to care about a billion years in the future to talk like that? I mean, I don't want to, especially if the worst risks are in the next thousand years, isn't this compatible with a much more near-termism? Like, don't do things that could wipe us out in the next couple centuries? I mean, I completely agree, basically. And it was, this kind of

into the way I was giving was me experimenting with different flamings where often I just want to say, look, at least consider centuries or thousands of years because I don't think it makes an enormous difference whether we're on a timescale of thinking

thousands of years into the future or billions of years into the future. But having said that, I'm a model philosopher. I really want people to just have the underlying model views and the big picture morality correct, where look, there are these enormously pressing reasons in the near term for reducing risk of catastrophe for ending, stopping the human race from ending.

But, like, as, you know, I'm hoping that the ideas we promote in the way that ideas in the past, like liberalism or the scientific method, continue to have relevance and impact over decades or even long, maybe centuries, maybe even longer. And for that, I just really want people not just, just to be acting on the right conclusions, but also have to the right underlying worldview. And even if that's weird and has this initial, oh, wow, this sounds like sci-fi.

I'm like, okay, yeah, but it really isn't. Just really start, or if you think it is, like, you know, tell me, tell me why. Like, I'm really not someone who came in from a kind of sci-fi background. I don't have much interest in that. I just care about people, and I care about them on the other side of the world. I care about them in the future, too. And I think it's really just important that people in general have the right moral views such that, I don't know, 20 years time, let's say we have got extinction risk just down to close to zero because of, you know, pioneering work by people who are inspired by some of these ideas.

Then I want us to keep acting to try and benefit the long-term future, even if that might start looking at all different. I guess this does resonate with something I worry about a lot, which is that the human brain or human values anyway don't seem very good at dealing with risks that are less than 50% chance over a century, right? We just don't care about that, whether it's solar flares or pandemics and so forth. So I don't want to put words in your mouth, but maybe, again, forgetting about millions or billions of years in the future, maybe like one, kind of obvious moral upgrade that we could do is just to take more, pay more attention to things that are unlikely and but disastrous over a century time scale. Absolutely. So the way people learn and society learns tends to be kind of trial and error process,

learning from experience. And so look at planes. Flanes are very, very safe. One of the safe modes of transport per mile. And why are they so safe? Because there have been a lot of plane crashes in the past. And so over time, we've been able to build better than safer planes, reduce the risk of accidents. Pandemics kill many, many more people per year on average than plane crashes.

I think we could just take the last, you know, I doubt 10 million people have died in plane crashes over the last 100 years. But are we prepared for them? No. And it's because on natural, in terms of natural pandemic, something like COVID-19 or the Spanish flu, that's like a one in the century event. We don't get to learn from feedback. And the same is true with these like somewhat lower probability, but high consequence events. If something happens just every few decades, then we don't learn from experience.

And that means what we need are people who are firstly just clear thinking and numerate. So really understand and, yeah, understand the risks that we face going forward. And secondly, kind of morally motivated, concerned to overcome the political hurdles such that we take action on them now. Because the situation we face is actually even harder than for natural pandemics. Because at least there we have a history of pandemics to use as evidence. But when we're talking about truly new technology, you don't have. have that history. So think about the situation just before development of the atomic bomb. Those who are agitating for US political leadership to take such a possibility seriously, we're saying, you know,

you should take really seriously this technology, this event that we have never seen before. And that's pretty tough. And we face, I think, a similar situation where we're talking about the ability to engineer new pathogens. We've never seen that before, but we should be worried. We should be thinking this through or the development of artificially intelligent systems that are at human level or greater than human level, we should be taking that really seriously and we should be taking it seriously before it arises because we don't get nearly as much feedback as an ability to learn from trial and error as we do from these everyday sources of tragedy like car crashes, playing catac clashes and so on.

And it reminds me of an issue that seems to come up in democratic governance. You know, I've talked a lot on the podcast about democracy and the threats to it and so forth. And recently I had a good question in the Ask Me Anything episode where they said, okay, so what is the downside of democracy? What is the argument against it? Just so we can sort of steal man that. This is probably one, right? I mean, democracies don't seem to be super good at long-term.

planning because the people you vote for very naturally worry about the next 10 years at most. Am I too cynical about that? And is there something we can do about that? Yeah, I want to say two things on democracy. One is the question of what exactly you're democratizing, where with nuclear weapons, you know, they're terrifying. One way in which we got lucky is that fissile material is very easy to control. So it's not the case that anyone can manufacture a nuclear weapon in their gas.

marriage. With future dangerous technology, ability to engineer new pathogens, if depending on the regulatory environment, could go a different way. And so there are some sorts of power we don't want to democratize precisely because they're very dangerous. So that's one thing. And then you're right that there's this, once you take seriously the interest of future generations, it seems like democracy has this major flaw, which is of the people that will be impacted by the policies we choose today, where let's conservatively say that future generations outnumber us 1,000 to 1, that's just if we live as long as a typical mammal species.

That's how many people there will be. Well, then it's only 1.1% representation of those who are going to be impacted. So what can we do about it, though? That's a tougher question. And again, in what we are the future, I was going to have a whole chapter just on institutions dedicated to taking the long term into consideration and how we can augment democracy to have a more long-term view. And I ended up not having that chapter just because I think the situation's a little bit

dismal. There are some things we can do. So you can have an ombuds person. for future generations. So Wales has the future generations commissioner. And their point in government is just to go around advocating for future generations issues. Wow. I really like citizens' assemblies. So this is where you take a randomly selected sample of the population. So let's say it's a thousand people, but that statistically represent the United States elect of it. And they would get together, you would present them with various different issues

that might impact future generations. You tell them to take on the role of their great, great grandkids and imagining, like, okay, what should we be doing in order to make sure that their lives, our descendants' lives, go well. And you get them at least in an advisory capacity to say, okay, this is what we think we should do. And these are things that I think are good.

I think they would make the world better. they would help inform government. I just think there are a far cry from, you know, seriously taking the long-term impacts of elections and put, you know, of democratic decisions and having that in practice. And so we can make these incremental changes, but I think something that's like trying to represent future generations in any serious way is just my guess is that it's impossible

because future generations aren't here. yet they can't represent themselves. And so any system you set up to try and represent them will have the force of special interests aiming to co-opt it. And that's just a tough situation within, I think. Now I'd like to introduce you to Meaningful Beauty, the famed skincare brand created by iconic supermodel Cindy Crawford. It's her secret to absolutely gorgeous skin. Meaningful beauty makes powerful and effective skincare simple. And it's loved by millions of women. It's formulated for all ages and all skin tones and types. And it's designed to work. as a complete skin care system, leaving your skin feeling soft, smooth, and nourished.

I recommend starting with Cindy's full regimen, which contains all five of her best-selling products, including the amazing youth-activating melon serum. This next-generation serum has the power of melon-leave stem cell technology. It's melon-leave stem cells encapsulated for freshness and released onto the skin to support a visible reduction in the appearance of wrinkles. With thousands of glowing five-star reviews, why not give it a try? Subscribe today and you can get the amazing, meaningful beauty. system for just 4995. That includes our introductory five-piece system, free gifts, free shipping,

and a 60-day money-back guarantee. All that available at meaningful beauty.com. Hey, everyone. It's Cal Penn. I'm the host of Earsay, the Audible and I Heart audiobook Club. This week on the podcast, I am sitting down with Ray Porter, the narrator of Andy Weir's audiobook Project Hail Mary, massive sci-fi adventure about survival. and science, and what happens when you wake up alone very far from Earth? I really had to make a decision because I caught myself getting that frog in my throat and starting to get teary as I'm narrating some of these sections. And it's like, okay, yo, yeah, yo, is this indulgent? And I really thought about it.

I was like, no, at this point, it would kind of be betraying the trust the author and the listener have in telling this story if I don't go through it. But there's places in this book that deeply emotionally affected me, and I left it on the mic. That's great. Because it served the story. People will say like, oh, my God, I cried at the end. It's like, yeah, dude, me too. Listen to Earsay, the Audible and IHeart Audio Club on the IHeart Radio app or wherever you get your podcasts. Okay.

All right. That was about as cynical as I am also, so I can't really argue with you. It is a flaw. We should think about... Hypologies. Trying to do better. Yeah, you know, reality. never promised us a rose garden.

So let's back up, and you are a moral philosopher, and we've sketched out some of the ways in which we do long-termism thinking, but now I want to really dig into why we should be thinking this way in the first place. And I take it that your starting point anyway is some kind of consequentialist utilitarian point of view. So tell us what that is, and then maybe a bit about the meta-ethical, why you think that's the right perspective to take. Terrific. So I'll say that the things I argue for are utilitarian flavored or consequentialist flavoured insofar as they are really focused on what good outcomes can we promote.

But I'm pretty careful to just in public only defend things that I think are justified in a wide variety of moral views. And my second book, it was an academic book, is precisely, and my PhD topic was precisely on this question of, on this. this issue where I think you should have some degree of belief in a variety of different moral views, consequentialism and non-consequentialism. And when you're taking action, you should try and act in a way that's the best synthesis or compromise between these different moral views. And I think that concern for the long term is something that would pop out of that.

So briefly, what is consequentialism? Consequentialism says you should do what's best, where what's best is what brings about the best outcomes. and it says that that's all that matters. And that can be unintuitive in one of two ways. One is it means that it's never permissible for me to just spend some resources on myself or do my own thing if I could do something else that would be, you know, think about the best outcome.

So I could, you know, spend the money on a nicer house or a nicer car. but that money could be used to improve the lives very significantly of people in very poor countries or could be used to help steer the future into a more positive trajectory. And let's say that, which I think is very likely to be the case, that will do more good overall from this kind of impartial point of view. Then you ought to do it on consequentialism. The fact you shouldn't give yourself any greater weight than any others. And that I think is very practically relevant because we in Western countries just do have a lot of the sources that could be used to do a lot of good.

There's a second aspect which can be unintuitive, which is that promoting good outcomes is always the right thing to do, even if that involves a kind of means that is not as well that we have like a common sense objection to. So if you can kill one person to save 100 other people, then on consequentialism, you ought to do that. Now, I think that's much less practically relevant for a couple of reasons. One is that when you do this kind of compromise between different moral perspectives, on at least some plausible moral views, people have rights that protects them against being used as a means, even for the greater good. And that's really serious on those moral views. and so the best compromise, I think, means we do respect people's rights

while trying to promote the good in cases where you're not violating anyone's rights or doing harm. And then secondly, it's just like, how often is the best thing to do, kill one person to save 100? It's just like, it hasn't come up in my life. It hasn't come up in the lives of anyone I know. It's the sort of thing that tends to be more, I mean, it would come up in wartime, but it's the sort of thing that comes up more in the philosophy seminar room than in practice

where in practice I think if you're the good consequentialist you actually want to be even more scrupulous you want to go around like being very very honest being very respectful to other people being very cooperative and the effect of altruism community in general has really tried to promote those virtues and so but yeah but the key thing is that makes the things I promote, like effect altruism, long-termism, kind of sound consequentialist. Is this ridiculous focus on bringing about good consequences, making the world better? But that's something that other model viewpoints really care about to, non-consequentialism,

virtue ethics. They also think that doing good things matters. It's just that they think it's not the only thing that matters. And I would agree. Can I ask? I'm actually very intrigued. I'm kind of sad we didn't do a whole thing. episode on this probability distribution over moral systems and using that as a guide to action.

So I'll just ask one very simple question. Would that strategy also be applicable to religious views? Should we take the expectation value of what we think God wants us to do if and if not God exists? I mean, I think you should give some, you know, am I certain, 100% certain that I'm not going to wake up after I'm dead? and God will tell me that, you know, I got it all wrong. No, I'm not 100% certain. And then there's this very thorny argument going all the way back to Pascal of saying, well, if you've got some credence in God and God can produce infinite amounts of value,

then you should really be waging on God. This, again, I'm having to be like a bit of an apologist for philosophers. This again, I think is something where we just don't have a very good. answer. Again, because, you know, we mentioned before this tiny probabilities of enormous amounts of value. Well, this is that issue at large. Where it's not tiny probabilities of enormous amount of value. It's tiny probabilities of infinite amounts of value. And I'm like, I mean, my practical attitude is like, we haven't figured this stuff out. If I try and take action on it, I feel about as equally likely that I'll do harm as I'll do good. Maybe it's like slightly more likely that I'll

produce infinite positive value and infinite negative value. But I can't think of a much better strategy than like trying to ensure that future generations exist and they can figure this stuff out and have more of a clue than I do. But like, I think if someone was like this really hard-nosed bullet-biting decision theorists, I don't think I have like a good philosophically justified response to them, to be honest. Well, I think, okay, you were careful in the discussion of consequentialism. You didn't emphasize the utilitarian version of you, consequentialism, very much. So, I mean, maybe explain what makes utilitarianism special within consequentialism,

and then are you a utilitarian and should we all be? Sure. So utilitarianism is a form of consequentialism. So consequentialism says just do what produces the best consequences. utilitarianism has one particular understanding of what those consequences are, which is namely the sum total of people's well-being. So you can take your action, look at everyone who's impacted, look at all those who are benefited, add up all those benefits,

look at everyone who's harmed, subtract away those harms. That's the amount of good you've done. And that can be contrasted with views, primarily with views on which things other than well-being matter. So preservation of the natural environment, knowledge for its own sake, perhaps, art for its own sake, just complexity, perhaps, life itself. And again, I think you should have a view where, you know, take lots of different moral perspectives into account, I think, and acts on compromise. That does mean that in practice I would give some weight to those other things. Having said that, at least in the seminar room, I find the arguments in favor of utilitarianism,

as compared to other sorts of consequentialist views, really quite compelling, where I find it much more plausible that there could be moral properties, properties that are good and bad, that pertain to conscious experiences in particular, than there's this thing, good and bad, that is the property of a natural landscape or of a piece of human created art, where it seems to me, I have direct awareness of this, of goodness when I have a good experience. I have direct awareness of badness when I suffer. I don't have direct awareness of the goodness or badness of a painting. Instead, that feels just like some judgment I have. And that's this crucial distinction, it seems to me, between well-being, in particular as it consists in conscious experiences and other

things that are purportedly good about. And that does raise the issue of the metaethical justification for all this. I mean, why should we be consequentialists? Are you someone who thinks that, well, we have moral intuitions and we're trying to systematize and formalize them? Or is there some kind of objective, transcendent argument that this is simply the right thing? Great question. So within the effect of autism community and people who work on this, there's a big diversity of views. Many people have this view which is kind of called sophisticated subjectivism where what you're doing morally is just trying to act on the preferences of what an idealized version of yourself would want yourself to want. So if you could just reflect forever, that's ultimately, but this is what you would

want to do, but ultimately it's just about your preferences. I actually personally don't find that view that plausible. I think it's a kind of awkward kind of how halfway house between one of two views. One view is just nihilism or one. Model philosophy is called Eretheory on which every time I say some model claim, murder is wrong, giving to charity is good, it's all false. It's like I'm talking about witches or flogiston. It's just a false kind of view of the world. And honestly, I find that be quite compelling. I'm not sure exactly what, you know, probability I give it, depending on my mood, somewhere between 10 and 90%, I think. But then there's this other view, which is that, no, that actually is, like, as part of the fabric of the universe, they're the model truths, or at least evaluative truths.

Yeah. Where, yeah, on my kind of preferred view, those truths are grounded in conscious experience, where there are these famous arguments for error theory. So J.L. Mackey, writing in, I think it's the 70s, calls them the arguments from queerness that moral properties are queer. And it's funny how the term has evolved because it's like, oh, yeah, it sounds like, oh, these properties are too gay to exist. He's just saying, well, if there are moral properties, things that in virtue of just understanding them, you think you should promote them, that would make them radically unlike other sorts of. properties, like natural properties in the world. And I'm like, yeah, they are kind of different, but consciousness is already just radically different from other sorts of things that we find in nature. So at least I've got a bit of a partner's in-gilt argument there. And then he has this

second argument from queerness, which is that, well, how would we know if there's this moral property, something we ought to do? What's the, what's the like way in which we get to this truth? And again, my answer is, by direct awareness, like how do I know truths about consciousness? It's because I directly perceive them. It's not because of anything I know in the kind of scientific realm. And so that I think at least, you know, there's plenty more to be said, but gives us at least some grounding for thinking that, oh, no, it really could be the case that there are these fundamental facts about what is good and bad and what we ought to do and not what to do. And then if you have got some degree of belief in both nihilism and

the model realist view. Well, there's no chance of making a mistake by doing the model realist thing. Because if nihilism is through, nothing matters. And I've spent all my time doing these good things. Well, the nihilus is like as good as anything else. Because crucially, the nihilism appeals to reasons I have to benefit myself as well. It's not saying there's no morality. I should just benefit me.

Saying I've got no reason to do anything. Yeah. It'd be including helping myself. So in that situation, you just may as well do the model realist thing. You may as well act as if there's a meaning and purpose to the world. So even if Pascal's wager doesn't work for God, maybe it works for moral realism. You might as well act that way. So if you might as well act that way.

To be clear, if I thought, if I had a 0.000,0001% degree of belief in model realism, I'd be a bit sketched out by the argument. but really I'm kind of more like 50-50 or something. Or at least, you know, I think it's really pretty plausible that this could be true. And so that makes me think this isn't this kind of crazy Pascalian low probability argument. Okay. I mean, again, we could talk about this for a long time, but I really want to start applying it to long-termism. And in the book, you have a wonderful chapter about population ethics where you sort of give

the epilogia for being utilitarian about all the well-being of the future generations and so forth. And correct me if I'm wrong, but you seem to be careful in that chapter to really use utility, the total well-being of all the people in the world, in a kind of ranking sense without actually attaching real numbers to it. So I mean, the skepticism I've had about utilitarianism has always been, what number are we attaching to the well-being of a person and how in the world do you think you can just add them up for all these people? And I mean, maybe there's a weaker version where you don't really need it to be an actual number as long as you can reach conclusions just from saying, but you have to think that this is better than that.

Terrific. So let's, we'll get onto you to maybe population ethics next, because this first question of how you even measuring well-being is kind of more fundamental, really. And so the way that I would measure well-being is by appealing to people's carefully considered preferences and the trade-offs that they would make. My particular favorite method is time trade-offs. So if I'm – well, you can do both time trade-offs or problems. ability trade-offs. But if I'm indifferent between, let's say, you know, I'll definitely die tomorrow, or I can have two days skiing or one day at the beach. And I'm indifferent

between those two things. One day at the beach and then I die, two days skiing and then I die. Then plug in up as long as my preferences fulfill certain other axioms. Then we can create what you know is called a ratio scale, such that my preferences, where the fundamental thing is just preference for X over Y. It's not like there are numbers like buried in deep inside me. I just have these preferences for, yeah, okay, indifference between two days skiing and one day at the beach,

perhaps I prefer to either of those things, four days in Paris. And I just have preferences over all of these things. The preferences satisfy certain conditions. Then we can use numbers to represent my preferences. And that's all that's going on, that numbers are just like, yeah, being used to represent those things. Then there's a second question of like, okay, that's my own preferences. How do we compare my preferences with yours and so on, or at least my well-being?

with yours the preferences are just a means in to what my well-being is and that's really hard there's like various philosophical accounts of it but the thing i want to say is look if you're saying that it's you know me pinching you in the shoulder and her or and someone else getting horrifically beaten up there's no comparison between those things can't say for whom is that worse i'm just like come on um we We obviously know, and so perhaps there's debate about like what's the correct theoretical account of how we're making comparisons of well-being differences across different people. But clearly we know they are. And in general, like, you know, you can just look at various sorts of experiences.

Like headaches are probably like, you know, on a similar range for most people, like other things like similar range for most people. you can at least kind of roughly start to make these comparisons. And that once you've got that, then at least approximately you're able to engage in, you know, utilitarian reasoning where you can meaningfully talk about some total of people's well-being. Well, maybe this is fair, but I'm not completely convinced you're not cheating a little bit here. I mean, I see the move that you're making, which I'm sympathetic to, which is, look, if you spend all your time worrying about the weird edge cases, you're going to miss some basic truths in the simple cases that actually arise. Okay, good. I'm on board with that. I get a very similar dialogue when I discuss the many world's interpretation of quantum mechanics.

But we're going to be extrapolating in some sense, right? I mean, this is what long-termism is all about. I mean, couldn't a skeptic say, like, no, no, no, until you have a rigorous theory that works even in the edge cases, I'm not going to believe your extrapolation a million years into the future. I'm curious on what you see is that edge cases that we're worrying about. Like when it comes to... So I guess the fundamental, all objections to utilitarianism, I think,

boil down to the idea that do you really believe that there is, if I have a person with a certain amount of well-being and another person with much, much more well-being, that there is some difference between their well-being that is so great, that I am indifferent to the person with a little bit of well-being existing versus a 50% chance that the person with a lot of well-being exists versus a 50% chance that nobody exists. You know, could I take that expectation value?

Like, those seem a little bit incommensurable in some sense to me, but it kind of lies at the heart of what utilitarianism asks me to do. So I think two things to say. One is at least on the, you know, utilitarianism doesn't yet make any claims about population ethics. You have to have a view of population ethics that you kind of attach on. So in particular, total utilitarianism says maximize total well-being. So if you can add someone with a thousand well-being to the population, that's making the world a, you know, plus, 1,000 well-being better. But there's other sorts of forms of utilitarianism. Average total

tailism says increase average well-being. That's the same if you've got a fixed population of people, but it changes if you're adding additional people. But now if you're looking at, but let's just let's even just assume total utilitarianism for now. And we've got a choice between adding one person for sure or adding 50-50 chance of another person with a better life. So it seemed to me that, okay, we've got that we've got that choice. Let's say person one has this life that's positive but just barely worth living. It's really devab. There's a lot of suffering in it.

The happiness just outweighs it. Person two has most wonderful life that's ever been lived and lives like 10 times as long. and it just has peaks of joy and creative insight. Just imagine your best day, but 10 times better every single day. Then I think, like, it's perfectly obvious that you should take the 50% chance of the one person with a very happy life. And then, okay, make the person's life, the really happy person, like, less happy, the person with the life barely worth living, a bit better. They will become some range where we just don't know, because these comparisons are really hard.

and perhaps there's even some range of incommensual ability. So not just that we don't know, but actually there's fundamental differences. So perhaps one person's life has great accomplishments and involves pursuit of knowledge. The other person is just this kind of beach bum and just like surfs all day and has a great time. And both of those lives are very good. But perhaps it's just very hard to weigh off those, weigh those two sort of lives together. the first view wouldn't deviate from total utilitarianism at all. It's just limits to our understanding of what we can know about the world.

The second one would move away from totally utilitarianism as it standardly defined, but not in a really fundamental way. Instead, it would just be sometimes you can say that one outcome is better than another one, sometimes you can say that equally is good, but sometimes you can't. sometimes one outcome is neither better nor worse nor equally as good as a second outcome. They're incommensurable. And as long as at least sometimes one outcome really is better than another, then utilitarianism is still action guiding.

It's still saying, at least in some circumstances, it's clear what you ought to do. Even though in other circumstances, it just says there's no fact of the matter, because these are both good lives, but they're good in different ways. But the conclusion you said was perfectly obvious. I worry if we try to extend it from one extra life to the entirety of the human race. Like if we say there's going to be a trillion people that exist in the future, and either they will be, yeah, meh, they'll have moderately happy lives, or there's a 50-50 chance they'll have amazing lives,

and a 50-50 chance they will just be extinction and no one will exist. I think that there, it's not so obvious, and maybe even most people would vote for 100% guaranteed existence, even though I just scaled up the numbers, right? Great. And this actually, so love how you're placing me and how deep we're getting into these issues. So there's this, and it actually relates to these issues before about tiny probabilities of large amounts of value. You start to get into the same kind of formal tools. So the question is, is value unbounded? So if I have two really good lives, happy lives, is that twice as good? Or once you've already got one, maybe a second one, still really good.

It's still making the world better than virtue of their being another happy person. Or does it make it just a little bit less good than the first one did? Are they diminishing returns to value itself? And the intuitive case for this is very strong, I think. Imagine a future where you have populated the solar system or even the galaxy or just Earth, but it's just you've created this utopia. There's a large population, 100 trillion people all living the best possible lives. And then I say, yeah, 50-50 chance of that going to zero.

or you can have three such utopias. And people are like, I don't know, the gamble doesn't seem worth it. I have those intuitions very strongly. And that suggests that value is bounded. So just as, you know, in an individual life, it's plausible that you can get happier than happier and happier, but there's some kind of upper limit, at least on a day-to-day. it's not that just your day, you know, you couldn't have a day that's like a trillion times as good as your best days. That might be true just for value as a whole.

And that would mean that, you know, take this flourishing utopian civilization, have it three times over. Maybe that's 10% better or something because you're very close to this upper bound of value. However, there are really bad problems for that view, too. because if you've got, if you think there's an upper bound to value, then it really matters how, when you make decisions now, it really matters how many people have lived in the past or how many people are alive in different places in the universe and how good or bad those lives are. Because you need to know, am I very far away from the upper bound? In which case, I don't really need to think about them.

I can just say, okay, if I'm far away from the bound, then two happy lives is approximately twice as good as one happy life. But if I'm close to the bound, then it really makes a big difference. And that just seems absurd to me. The fact that we ought, morally, to be, like, figuring out, like, what were the lives of the ancient Egyptians like and were they good or bad? How many animals were the lives in the past and were they good or bad? Are there other alien species and are they good or bad? that just seems absurd to me too. And so again, we have a paradox, I think.

We've got this issue where any view we have has these kind of devastating objections. And honestly, actually, one of the things I think model philosophy has contributed most over the last five years, especially as we've started to use some more formal tools, is demonstrating a whole bunch of these areas where you can prove, I mean, I call it a paradox. because that's only in the informal use of the term. But you prove that impossibility results is the technical term. You have four conditions, four principles. Each one is just, yeah, absolutely this.

You've just got to accept this principle. Couldn't not be true. And then you prove that they're inconsistent with each other. And this is one of those cases. So I'm not saying, yeah, I'm going to. But then the thing. I should say is that's compatible with utilitarianism. That's just a different.

The view on which we have this like bound and like additional happy lives get less and less, have less and less additional value. That's utilitarianism, but with a different view of population ethics kind of slapped on. And that's actually, that category of views is one of the categories I didn't have time to get into in what we owe the future. But that's a shame because they're, yeah, it's a, Yeah, they're an interesting and important category of views you might have. Well, I guess personally I'm in that what you call the wishy-washy middle ground of not being a nihilist, but not being a more realist either, being a constructivist who thinks that we have these intuitions, we try to formalize them.

And as you say, it very often is the case that you think you're formalizing your intuitions and you use your formal system to prove a conclusion you think is morally repugnant. So clearly you've made some mistake there or you're just incoherent. And neither one of those is a happy place to land. But let me just push it one more bit on utilitarianism because it does seem, even if you sort of have diminishing returns to value or an upper bound on the total amount of value, the fundamental idea is you're adding up value or well-being or utility or whatever in all these different lives. And that leads to a feeling that all else being equal, more people is better, if all else is equal. And, you know, Derek Parfit famously reached what he called the repugning conclusion. And maybe you could tell us what the repugning conclusion is. And if I'm not mistaken, your attitude is, yes, let's accept this repugting conclusion.

He was cheating to call it that. So I agree that it was cheating to call it that. and I'll tell you the reason why, which is that any, again, within this area where you have these very strong intuitions and they're all inconsistent with each other. So, and I actually do think that the repugnant conclusion is maybe the least repugnant of the principles we'd have to reject. But let me explain. So again, bearing in mind that utilitarianism is compatible with many views of population ethics,

and every moral of view has to engage with population ethics. We all face the question of how should we weigh up bringing new people into existence, is that good or bad at all? So the pudulent conclusion is the following. So imagine you have a population of 10 trillion people, some large number of people, and they have amazing, amazing lives. Let's say it's plus 1,000, let's use that number, just to mean really, really good lives, bliss. and now consider you have some second population where you can imagine by population just future

of the world let's say and they have lives that are barely worth living let's say plus 0.1 you know it's good they think yeah I am happy to have lived but I'm close to indifferent perhaps my life has just been very boring not much has happened or perhaps it's had like both happiness and suffering in it but there are a very, very large number of them. Let's say there's a trillion, trillion of them, or a trillion, trillion, trillion, we can make the number very large indeed. Now, if you have this total utilitarian view, which is saying even in cases where you're adding people to the population, what you do is just add up the well-being across the whole universe, across the whole future, well, then you would get this conclusion, this conclusion that the very, the very well, the

very, very large population with trillion, trillion, trillion people with lives that are just barely worth living. That has total greater well-being than the much smaller but still very large population of people with total bliss lives. And that seems the pungnant, says Derek Parfit. And he really stuck, I mean, he deserves to name the conclusions because he really inaugurated this field. did believe it was repugnant. I mean, before he died, he told me that he would give up on the idea of their being moral truth at all. And he was a pre-committed moral realist. He would give up in that idea before accepting the repugnant conclusion. However, you can prove that if you,

you can prove that there are a small number of principles within population ethics that are inconsistent with each other. And I'll kind of give you the argument just very quickly. So start off with this 10 trillion people who live extremely good lives plus 1,000. Call that the A world. And now we're going to change that A world in two ways. We're going to double it in size. So it's now 20 trillion people. With the original 10 trillion people, we're going to add just plus one well-being. So that's 1,001 now. The other trillion people, 10 trillion people, are going to have lives that just a little bit less good, so plus 998.

Now, has that move, and we'll call that world A-plus, twice as big, you've made the original people a little bit well-off, the new people have lives that are extremely, extremely good, but not quite as good as the well-being of the lives in the A world. Does that make the world better? So maybe I'll just ask you, you can tell me what you think. Well, actually, no, I mean, it's cheating because I've read your book and I have had time to think about it. And actually, this is the step that I would deny. I think that just adding more people of equal levels of happiness is not necessarily better. And again, I started off the whole thing by saying I don't have the once and for all final answer in my own mind.

So I'm happy to change. But that's the weakest link to me. So I agree it's the weakest link, actually. And that if you're going to reject the appellant conclusion, it's this step of the argument. that I'm giving you that you should reject. But I mean, let's linger on it. So let's say you're one of these 10 trillion people originally, original people. By making this change, you're going to make your life a little bit better.

And the life of everyone else you know, you're going to move from 1,000 well-being to 1,001 well-being. And the cost of that is adding more people who will also have supremely good lives. In fact, lives close to this bliss world that I stated, plus 998. what's your argument for saying that that's long? Sure. And the intuitive pull of the argument is perfectly clear to me. So I don't have an argument that it's wrong. I have the idea that maybe the function that goes from the set of individual people and their happiness

to the total consequentialismness of the whole thing is highly nonlinear and not just additive over the number of people? So, I mean, that might well be true. That doesn't, okay, I'm going to stop pressing you on this at the moment. I think you're right at the stage to get off the boat. Yeah. And I think actually what you can say is that, yes, in this case, it would be better. But once well-being levels start to get lower, then it's no longer better.

And in fact, in that all things considered view that I kind of present or endorse in what we owe the future, it actually rejects that step. And so the view of present in what we are the future does not endorse the repugnant conclusion. Even though I'm sympathetic, I don't think it's crazy to endorse it. But okay, let's say that you think, yeah, this is better. You've made the existing people better. 10 trillion people have moved from 1,000 to 1,000 to 1,000. you've added another 10 trillion people at 998. Seems pretty good.

We'll call that the A-plus world. And the move there was by a principle of like, for saying that's better, that's called dominance addition. Edition because you're just adding good people, dominance because it's like also making existing people better. Now we're going to move from A plus world to world B. And to do that, we're not adding anyone at all. And in fact, what we're going to do is increase total well-being and we're going to make it more equal. Because we've got this world of 10 trillion people at 1001, 10 trillion people at 998. And we're going to move that to a world of everyone at 99.9.5.

That has increased average well-being. It's increased total well-being. And now everyone is equal. There was a little bit of inequality in that former world. we've made that more equal. Again, it seems like that world is better. Like, what is the argument why you would reject that move or say that that is not making it better? Right. I mean, you have now lowered the average well-being

compared to the original world. So, that's the rub. Because, okay, I've now comparing A and B. Yeah, I've created a large, population and I have a larger population of lower average well-being. Now you can iterate that. So I had A to A plus to B, we can do B plus. Now it's 20 trillion to 40 trillion people moving from 99.5 to, let's say, 1,000,996.

And so on and so on and so on. You keep iterating going from A to B, then B to B plus to C and so on. And what you're doing is increasing the size of the population and lowering the average. average well-being. Keep doing that. You end up moving from the A world, 10,000 people of 1,000 well-being to the Z world of everyone with barely lives that are barely worth living. Yeah. And yet, but a very large number, you end up with that world being better. And so what are the, you know, what the principles you have to reject? Well, you either have to say that at some point, the move them something like the A-world to the A-plus world is bad,

or that move the A plus world to the B world is bad. Or you have to deny transitivity, which is to say that if B is better than A plus, and A plus is better than A, then B is better than A. That seems pretty compelling. Or you have to accept a pugnant conclusion. Yeah. And in light of those, yeah, I think it's hard.

my best guess is that the best thing to do is to accept of a permanent conclusion. But as I've been saying, what I think we ultimately ought to do is take a compromise between different moral worldviews, including different views of population ethics. And what that ends up giving you, if you really reason it through, is what's called a critical level view, where there's some difference between the zero level is the point at which I is an individual, I'm like indifferent between having ever been born and never having been born. And it's just this higher level.

And it's only above that higher level that it's good to bring someone into existence. So bringing someone to existence with a life that's barely worth living is not a good thing, might even be bad. But if they have a sufficiently good life, then it's a good thing. And that avoids the apprugnant conclusion. And it does that by rejecting this dominant sedition principle. Right. once you start to get to lives that are not fairly good.

Okay. Thank you for indulging me in my poking about utilitarianism, because I've talked about a lot of the podcast, but never quite directly in the way I wanted to, and your answers are extremely illuminating. And so I don't dogmatic as well, which is great. So let's wrap up by going back to the real world and being practical and saying that, okay,

if we have this idea that future generations matter and we can influence them, and therefore that should color our actions in the present. What are the practical things we should do? And actually, let me not be quite as fair yet. Let me start with the really hard questions. Like, is there some implication that we should, it's more important to have kids than anything else

because, you know, that would bring some extra positive utility into the world? I'll say two things on that. The first is that if you think that, Yes, bringing a sufficiently good life into the world is good. Then the overwhelming focus, or the overwhelming upshot of that, is that extinction of the human race is much worse than we would otherwise think, if you didn't have that view.

Because the loss of life would be huge, astronomical, trillions upon billions of future lives. And in fact, like the question of whether you should have hit kids today would actually just depend on, like, how does it impact the long-term future, including a reduction of extinction risk. The second thought, though, is like, okay, let's just take that question at face value. Like, is it good to have kids? There's obviously a very influential view that says no because of the climate impact. I think that argument is really quite weak and perhaps even harmful.

And I think because it only looks at one side of the ledger. It looks at the harms that a child can have, you know, imposes on the world, but not the benefits, too. Where if you have children and are a good parent and bring the kids up well, your kids will go on to do lots of good stuff too. They will, you know, contribute to society. They will pay taxes. If you bring them up well, they will be model change makers. But then also people innovate. So they help create new technology, new technology.

ideas that help make the world a better place. And if you have the view that, well, it'd be bad for me to have kids, do you think it'd be good if there was just like no people at all? Well, probably not. I hope not. But if so, then it's like, okay, well, where's the line? What's the difference? Or at least in the past, supposing there'd just been way fewer people in the past.

Well, let's say they'd been, yeah, you know, half as many people. well we'd be still living as farmers we would you know the amount of technological progress and intellectual progress we've made is because we've had a lot of minds tackling these problems so we would still be living as farmers we wouldn't have anesthetic we wouldn't have you know modern prosperity we wouldn't be able to travel and the same I think is going to happen like the same considerations motivate larger population sizes in the present as well because in fact like the fact that population is going to peak at 2050 and then decline after that, I think poses real challenges for us as a civilization.

The standard economic models would say that the technological progress that we're used to would actually stagnate. And I think that could be a very bad thing. Could increase extinction risk, could turn the world into a kind of zero-sum affair. And then the final consideration is just the benefit to the kids themselves. Where, yeah, if you can be a good parent, bring up your children, bring up your children to have sufficiently good lives, that is a good thing from the perspective of the kids,

who I mean, I'm happy I was born and got a chance to live, and a good thing from the perspective of the world. So I think that decision whether to have kids or not, deeply personal matter, certainly not something that the government should be involved to try to legislate or dictate. But is this one way of being a good citizen and contributing to the world as having children bringing up well, I think, yeah, absolutely.

Well, is it, I mean, I understood and got the logic behind everything you just said without necessarily agreeing with it, but are you sort of wimping out there at the end? I mean, shouldn't we, if you buy everything you say, just conclude that it is a moral imperative to have kids? Well, I think we have a moral imperative to try and do good things, to promote the good. And there are many ways of doing that. And for some people, perhaps the best thing is to have a family and bring up your kids well. There are other ways to contribute as well, which will segue me onto the other action

that I talk about in what we have a feature. So one is donations. So you can do enormous amounts of goods by spending your money well. I have set up an organization called Giving What We Can that's encouraging people to give at least 10% of their income to the charities that you believe do the most good. That's just a way that I think almost anyone can contribute to making the world better.

You can pursue what career you're excited about, but donate 10%. You're having a really big impact. And there are organizations that you can fund that are really doing important. important work to reduce the chance of the next pandemic or safely guide artificial intelligence or, you know, reduce the risk of war. And so that effect out to some long-term future fund is one place where one can give. And then a second thing is career choice. So you spend 80,000

hours working in the course of your life, maybe even more. So I set up an organization called 80,000 hours that encourages people to pursue those careers that do good and give advice. on how you can have the biggest impact you can because you can really have an enormous amount of impact if you choose to use your careers in ways that will benefit the common good. And so I would encourage people to see this as like a menu of options, like the many ways to contribute to the world. I think that like having children should be one. And if you want to have kids, I'm like, great, go ahead. Don't feel bad about that, at least if you're going to bring them up well. If you don't, okay, great, there are other ways to contribute as well.

You know, donate to really effective places. I also talk about voting is something that's just like extremely important, especially politically and, you know, well-informed voting. But most of all, if you can pivot your career onto issues that are to help and work on issues that are going to be pivotal for the future of human race, enormous amount of good that you can do. I don't know if you know, but 80,000 hours, organization is a sponsor of the Mindscape podcast. So I've been doing ads for them. Oh, I actually

didn't know that. But it makes me very glad to hear because, yeah, I'm sure in your audience, many scientifically minded and altruistic people who I think could take up the mantle and, yeah, really try and put some of these ideas into practice. I do want to give you the chance to respond to, I think is maybe the most common worry about long-termism, which is that. And you sort of, came close to talking about it there. If I really thought that, let's say, there was a 1% chance of a pandemic, and therefore that would wipe out all the,

and maybe I do think that, I'm not sure what I have credences, but that would wipe out all of humanity, then wouldn't I be tempted to give all of my disposable income to fighting against that and not give any of it to helping starving children around the world? Is there a danger that being long, term in our thinking gets in the way of just doing something immediate and tangible and good right here and right now? So this is the trade-off we face. And I call this the horror of effective

altruism, the absolute horror of the world we are in at the moment, where you now, sitting in your office and me now sitting in my office, live in the mother of philosophical thought experiments where this same question applies, even if you're just focused on the present generation, I fund bed nets to protect children against malaria. That means I've not funded deworming tablets. I funded malaria bed nets in one country. It means I've not funded them in another. Some people are dead if you're donating even just thousands of dollars because of your decision.

And so anything we do has this unreal opportunity cost. And so we need to just grapple with these hard tradeoffs. The next thought is going back actually all the way to the beginning, of the podcast is to think in terms of humanity as a whole, the point of view of the universe, but humanity as a whole, where should it be all of my disposable income or should I split it? Well, from the point of view of the world, that really is not a large decision. You are just ever so slightly changing the allocation of the sources in the world. And how much money at the moment is being spent on these issues that protect future generations? My guess is that something like 0.001% of global GDP,

And now what percentage should it be? Like, I don't know. Should it be 1%, should it be 10%? Like, I honestly don't know. I also just honestly don't know at what point long termism just stops being interesting as a flaming at all. And instead, the thing that's best for the long term is just exactly the same as the thing that's best for the short term, which is building a flourishing society. At the moment, there are kind of more targeted things that are still very good for. from the near term, even if they're not necessarily the very best for the near term, like

preventing the next pandemic, like avoiding dangerous biotech. But what I am saying is that for the world as a whole, yes, I would love there to be radically more spending to improve the lives of the very poorest people in the world. But I would also love there to be an awful lot more attention paid to issues that impact the entire future of our civilization from this incredibly low baseline that we currently have. You know, that's a very, very good place to end, I think. There's no reason to think that we can't do both of those at once, helping us ourselves now and the future.

So Will McCaskill, thanks very much for being on the Mindscape Podcast. Thank you so much for having me on.