On with Kara Swisher - The Vatican Takes On AI

Episode Date: July 2, 2026

Kara talks to Father Paolo Benanti, a Franciscan friar, theologian and AI ethics adviser to the Vatican, about why AI has become a central concern for the Catholic Church. They discuss Pope Leo XIV’...s encyclical on AI, the Church’s role in shaping a more human-centered debate and why Benanti sees today’s AI race as a question of power, democracy, belief and human dignity. Kara presses him on Silicon Valley’s influence, Peter Thiel's views on the Antichrist and whether the Vatican can serve as a public square for global AI debate. Plus: can the Church help unite countries around shared global safeguards? Questions? Comments? Email us at on@voxmedia.com or find us on YouTube, Instagram, TikTok, Threads, and Bluesky @onwithkaraswisher. Learn more about your ad choices. Visit podcastchoices.com/adchoices

Transcript
Discussion (0)
Starting point is 00:00:00 We can imagine church in a lot of ways. One could be a pyramid with a big boss on the top and he says something and everything happened. That's not the church that I know, especially because I belong to Franciscan. And we are used to say as a joke, we more than an order, we are a disorder. That means that, you know, we have a lot of different processes. Little Catholic humor for you. Yeah, yeah.
Starting point is 00:00:23 It's Catholic. It's for everyone. There is space also for someone like me. It's all. Hi, everyone from New York Magazine and the Vox Media Podcast Network. This is on with Kara Swisher, and I'm Kara Swisher. My guest today is Paolo Bonanti, a Franciscan friar, Catholic priest, and one of the world's leading experts on the ethics of artificial intelligence. Father Benanti has advised the Italian government, the UN, and two popes on technology, and some of his insights
Starting point is 00:00:58 are reflected in Pope Leo's encyclical on AI, magnificent humanity. In that formal letter to Catholics around the world, Leo called for an artificial intelligence that serves humanity, not one that would take the place of human intelligence and creativity. Benanti has a background in engineering and teaches moral theology as a university professor. He also has a healthy skepticism of tech oligarchs, as the Pope does, who stand to profit from AI. I'd love to talk to the Pope obviously about this topic, but I've known about Benonte for a while, and I know he's had a real impact on the thinking of the church, which has been bubbling upward since before this Pope with Pope Francis. and the idea of what technology does to society.
Starting point is 00:01:39 I think it's critical that the church and other leaders have a role in this. And in fact, it's turned out that the Pope has more of an impact on Silicon Valley and their thinking than I thought possible. And I'm glad to see it. I am a lapsed Catholic, but I really do respect what the Pope is doing now. And it almost gets me back into the church. We'll see. Our expert question today comes from computer scientists and AI ethicist, Joshua Benjio. So stick around.
Starting point is 00:02:17 Support for this show comes from Odu. Running a business shouldn't feel like surviving a software group project. One app for accounting, another for inventory, another for sales, and somehow none of them ever talk to each other. That's where Odu comes in, and an all-in-one business management software that brings every part of your business together, from sales and accounting to inventory and marketing,
Starting point is 00:02:38 all in one powerful platform. No messy integrations, no bouncing between tabs, and best of all, no spreadsheet. Stop managing software and start managing your business with one unified system. Try it today for free at odoo.com slash Kara. That's ODOO dot com slash Kara. This episode is brought to you by Accenture. When your advertising operations fall out of sync, everything else follows.
Starting point is 00:03:07 Spotify and Accenture are working together to reinvent the rhythm of ad sales, using automation, analytics, and smarter workflows to simplify campaign delivery and access better data across the business. The result? Less time spent on operations, more time connecting brands with the moments and fandoms that matter most. Learn more at Accenture.com slash Spotify.
Starting point is 00:03:30 Hey, quick thing before we get into it, I'm doing a live taping in Washington, D.C., on July 16th at the Johns Hopkins University, Bloomberg Center. My guest is Gina Ramando, the 40th Secretary of Commerce and the 75th Governor of Rhode Island and someone who has a lot to say about AI and what it's actually doing to the American economy.
Starting point is 00:03:50 She may also run for president, so we should listen to what she has to say. It should be a good one. You can be in the room. Head to voxmedia.com, Kara Swisher, live to register. Spots are limited. We always sell out.
Starting point is 00:04:02 The link is in the show notes. Father Benanti, thanks for coming on on. Thank you for having me. Just where people don't know, you're a Franciscan friar, a moral theologian, and you also advise Pope Leo,
Starting point is 00:04:19 the 14th on AI. I believe you advised previous Pope also, correct, on technology issues. Yeah, much more Pope Francis. Pope Leo is now shaping the courier in a way that is desired. So we are in the meantime's area. Right, right, right, exactly. So talk about why is the rise of AI a faith and religion issue. And you had been talking about this before. Well, you know, there is something that happened before in the church, 100,000 years ago. with an encyclical that the name was Rerun Novarum, pushed out by another Pope and the name was Leo 13. At that time, I think he named himself for that reason. I have a feeling. Yes, he said day one. He said day one. Like Leo 13 faced this idea of a technology pinging on the human. I'm seeing
Starting point is 00:05:08 that AI has done the same things. Right. Explain Pope Leo the 13th, what he was discussing at the time. Yeah, you know, that's the point. It's interesting things because it's not steam power. The problem. The problem is that steam power was producing a shift of power in society and a leverage of things that was pushing a certain number of human beings in a really uncomfortable situation in a way in which we can define a sort of modern slavery. And today it's not a matter again of digital or artificial intelligence. It's not this, the hot topic of the church.
Starting point is 00:05:46 But once again, is power coming back. among us as a species in a domesticated way, like is it in democracy or is coming wild? And this is a little bit the way in which Pope Leo 14 is looking at this new wave of technology in society. So why Pickett now? You and I have talked about the internet before, and the internet's been a rising power. Why does it become critical when it comes to AI? With the invention of Chad GPT, that is probably for everyone, the equivalent.
Starting point is 00:06:19 of AI is not true, but it's for the pop culture is the same things. My smartphone is 1,500 times 500 times most powerful that the computer that bring guns to the moon, but is not able to compute something like that. And so we need a new centralized form of computation that this time has the shape of data center and cloud. What is the problem? That this new centralization of data center is, is in some way collecting all the part of our life that we digitize in the last 30 years.
Starting point is 00:06:56 And actually, a data center is someone else computing that can simple compute and control what's going on in our life. And we see a lot of countries that are doing this really, really effective, try to think to China or rather non-democratic country around the world. And so who is controlling the hardware now is controlling also a huge part of our life. that sound to me really political. Right, and so therefore the church needs to weigh in here. Now, you have a background in engineering.
Starting point is 00:07:26 You did make a note that AI has been here before. It's just taken off in this kind of Cambrian explosion, is a word I tend to use. I mean, obviously, everyone remembers Galileo, but the church has been very deeply involved in science in not necessarily a negative way for many, many different times. Explain the juxtaposition between faith and science. Well, you know, from a core point of view, we are talking of an institution that is 2,000-year-old.
Starting point is 00:07:52 So it makes sense that the perspective was not always the same. When the first Christian people arise, they have to confront with a really well-shaped form of power that was the Roman Empire and the other country. It's interesting that Christian, in a different way from other religion, don't try to found their own regulation on God. but they found reason. Why? Because if we believe as a Christian that there is a God that make us, everything that he made in us is not against his will. And so everything that is reasonable is according in some way with God will. And that produced during this 20th century of history a lot of different reflection. Don't forget, for example, that one of my friar, 13th century, the name was Raymondolulu, produce a book, the name was the Arts Magna, in which he started to think, well, can we say
Starting point is 00:08:48 all the fruit on God with that simple permutation of words? And you have this double will things with a lot of words that are turning on. And a couple of centuries later, Leibniz look at that and said, wow, that is fantastic. Let's move to logic. And when you are typing something on a computer today, you are using logic that is driven by Leibniz that is founded on a Franciscan reflection. Or you can have things like the genetic studies, Mandel, it was a monk. So this idea that if the nature is a gift of God, you can investigate, you can acquire it without fear, is something that belong at least to some tradition inside the church. And when you think about the history of the church, people don't realize there's a lot of
Starting point is 00:09:36 scientists who were various Jesuits or Franciscans throughout history. I was taught a lot. I went to Georgetown. I was taught by Jesuits. Me too. I know that. Yeah, many of whom were scientists. One of the things that's important is that the church can weigh in on these things. And there's been pushback from politicians, stay in your lane church essentially, or Pope. Stay in your lane, Pope, I think is the message they're trying to deliver. That was something that probably find the most powerful way with secularism, you know, in which day someone start to say, well, you belong to a specific place that is the sacristy of the church. So stay there. Don't go around. Like, you know, like a wild beast in the zoo, if I can use these images. But now, let me push a little bit back
Starting point is 00:10:25 on what does it mean to be a member of the church. If you do the priest, so only the priest, you should preach the gospel during the mass. And so, if it is, it. If the gospel say, you have to look at everyone in this place like a brother and a sister, isn't it a political message? Isn't it something that go beyond a serious of displacement of power that usually has the tendency to frame, divide, and in some way shape the order in society? So, of course, that is not always clear in the history. When some Paul brought to the slave, you know, behaving.
Starting point is 00:11:04 like a brother of your owners and to the owners, behave like a brother's to the slave, at that time, 20 century ago, was not thinkable that slavery was something that is unjust. Today, if you would like to be Catholic and you think that slavery is acceptable, you are not Catholic. So it's an ongoing process, but it's political too. Right, right. So let's talk about the encyclical specifically. Scott and I discussed it on Pivot, and we were very impressed by it.
Starting point is 00:11:32 In it, the Pope calls for government regulation protecting children. He warns against AI warfare. Talk about the conversations as he was writing this and the process. I'd love to understand where your role was in shaping it and where his mind was. Because putting these things out is a very big signal by the Pope that this is an important topic. So talk about the creation of it. Well, you know, we can imagine church in a lot of way. One could be pyramids with a big boss on the top.
Starting point is 00:12:02 That's simple as a lot of worker done. And he says something and everything happened. That's not the church that I know, especially because I belong to Franciscan. And we are used to say as a joke, we more than an order, we are a disorder. That means that, you know, we have a lot of different processes. Little Catholic humor for you. Yeah, yeah. It's Catholic.
Starting point is 00:12:21 It's for everyone. There is space also for someone like me. Well, so what does it mean? That means that we can imagine church like a cake with yeast. So the yeast is something that is used to produce an augmentation of the shared thinking. So it's not important to be in the Catholic Church that the Pope has a clear view and understanding of an issue. But it's much, much more important that this belongs to every faithful. And this is a process that need time, need reflection.
Starting point is 00:12:54 So when something surfaced on the top level of the church, so as a Pope document, that means that there was a huge digestion of a lot of different things. Let's be back to Leo 13. Leo 13 was a nuncio, that means a diplomat of the church, to the northern Europe, and he saw what the steam power and machine are doing to the worker and do the transformation. He reflected on that with a lot of people inside the church in Europe. When he was elected pope, he was able to connect the dot and produce this rerum-novarum, that say to the church, look, we are not in the sacristy,
Starting point is 00:13:33 we are with the step in the world and with the brain in the heaven, but we have to walk on this world with other people. And that was really interesting, because the same things probably is happening here with Pope Leo 14. So there are a lot of people that are reflecting on that, lay people, professor in the university, strange Franciscan like myself, doing strange things like this discussion with you.
Starting point is 00:13:56 And this is producing a sort of a global, awareness. It starts with Pope Francis, really clear. Yes, it did. 2018, then 2019, 2020. In 2020, it's not just a matter what's happening inside the church, but it's also something that happened in everyone's life with COVID pandemic and COVID push every generation on digital means. Right. It did. And so digital became top one of the topics. So you can go back and see that the topic of AI is surfacing a lot of time in discussion on autonomous weapon, in discussion of child protection, in discussion of ethical approach to the algorithm. Then it becomes a little bit more consistent in a note from the congregation of the doctrine
Starting point is 00:14:47 of the faith and education decastery, because education is touched by AI, and the name was Antiqua Itnova, old and new things. and then Pope Francis Passaway, and Pope Leo is the first document that he simple proposed out. This document was signed and published. Meanwhile, a couple of office in the OLECE that we are used to call the castery, the castery of fate, the castery of integral human development, and the castery of education was involved in producing the blueprint of the document,
Starting point is 00:15:26 And the Pope touched it with his personal style. So if you find that Santa Augustine quotation or something like that, you can see how he's finishing this collective work. Because the Pope take the duty to offer to the church a reflection that he said, for me is what I'm understanding as a correct reflection. But it's not his own reflection. It's a reflection of the whole church. And this is typical.
Starting point is 00:15:53 The whole church had been discussing it, starting with Francis. And then it rises up to him as it's been worked on. And he presumably was involved in the part where Francis was focused on a. Yeah, yeah, yeah. Also because he was in charge of the new bishop. And he invited me as a cardinal to talk to the new bishop about the challenge of the new media, the digital media and AI. And he has also a scientific background.
Starting point is 00:16:19 So he's not far from the topic. Now he takes the duty of the office to tell to the church, Look, it's not a secondary element in everyday life. It's something that they shape our personal opinion, the way in which we interact with either, with other people. We are seeing something like a companion chatbot that can shape forever what we understand of intimacy. So let's have a reflection on it. This is not the end of the reflection. No, the beginning.
Starting point is 00:16:49 The beginning of it. But he could have picked anything at all to start with. But this to him was central, was what. was happening with our relationship with AI and the tech executives that he does call out in some ways negatively and sometimes positively because when he presented the letter, Anthropic co-founder Chris Olaugh was seated right there with him on the dais. Anthropics currently suing the Trump administration after refused to let the Pentagon use Claude for autonomous weapons and mass surveillance. Talk about was that a pointed rebuke to the Trump administration's handling of
Starting point is 00:17:22 AI, or in general, the kind of reflection that he was trying to establish was a more safety-focused, human-focused? Well, we have to recognize that in this moment, the leading innovation on AI happened into country, that is, the United States and China. But we have do not forget that the majority of faithful probably belong to Africa and Latin America. And so when you see a document like Magnifico Manitas, you have to be a document. to see a more broader approach. It's not focused on Silicon Valley. It's focused on humanity. And if I have to express the percentage of preoccupation of the Pope probably will be on the
Starting point is 00:18:05 effect of the most fragile in the world, that will be the Global South. And the idea that the big tech was present when the press release of the document was there, it's not to be understood as a picking one because he is faithful to the document. Because the popsy really clearly, it's not enough, an ethical approach that is decided by someone in the bottom room without a big sharing with other people. And this means that, you know,
Starting point is 00:18:37 maybe Anthropic is the most active on AI, but it's not the way that the churching is looking too. Every human beings, in the same fact that is a human as the right to participate to the debate. So we are simple finding a dialogue, also with people that are building such kind of things. We do not pretend that they are in the same page where we are. I think that this is a huge expression that Pope Francis lived to the church, this ability to be open to the difference without fear and without the urgency to say, we have to think everyone the same things now. Right, right, right. So it goes up through committees and then he adds to it, right? Is that
Starting point is 00:19:17 correct, and you just keeps getting worked on. And this is something you had been interested in and many others. Did he write it himself or he works with a group, as with any president or political figure, they work with a lot of people in formulating it, correct? Well, we can understand probably the best metaphorical image to understand that is like when a plant grow. You take water that come from outside, but is filtered from the roots. So there is not a scientific concept. that advise him directly. But scientific and technological people was involved in a lot of satellite reflection that happened in the policy.
Starting point is 00:19:59 Last September, there was one really big in which people like Joshua Benjo, Hinton, and other scientists, Max Tegmark, was talking about the dangerous superintelligence. Or we have the wrong call for AI ethics that's much more directly addressed to people like the tech guys, the people that make business on this, or to the international organization. So all these different sources become something that is a nutrient for the developing of the process. But you cannot have this paragraph is connected to this, this connected to. It's a digested things. We'll be back in a minute. Support for this show comes from Teleport. In the AI era,
Starting point is 00:21:01 one of the biggest questions is how to contain agent behavior in your production infrastructure. In Teleports survey of more than 200 infrastructure leaders, the company found that those confident in their AI deployments have more than twice the incident rate of those that aren't, 72% versus 33%. Let's talk security basics. The most frequent causes of data breaches are human error and compromised credentials. But in the AI era, the challenge is that agents with broad privileges can find these credentials and gain access to sensitive data. Teleport establishes a unified identity layer for humans, machines, and agents that is cryptographically backed, which enables agents to be controlled and contained with the same rigor that you apply to other actors in your infrastructure. Security is complicated, but with teleport, it doesn't have to be.
Starting point is 00:21:47 Download the free report at go-teleport.com slash on with Kara. Support for this show comes from Rippling. As a business owner, you know how in-demand top talent can be, and that means retaining your best employees is something that you should always keep at the top of your mind. That's where Rippling comes in. Rippling is built on your live global workforce data, giving you full visibility into your business and the ability to stay ahead of the curve. Say you want to retain top talent. Just ask Rippling AI, who are my top performers this year? And instantly receive a workforce report highlighting your top employees with supporting data like recent performance reviews
Starting point is 00:22:29 and engagement metrics. But it doesn't stop there. Rippling can turn these insights into a proposed retention strategy, including a recommended 10% spot bonus for top performers. All you have to do is tap confirm and the spot bonus is added to next month's pay run. So don't settle for AI. That's all talk. Head to rippling.a-i-slash-K-R-I-S-K-A-R-A-S-R-A-S-C-A-R-A. Sign up for exclusive access today. I'm Seth Matlins. My new show, Creator Destroy, Reimagining Marketing, explores how every decision a company makes,
Starting point is 00:23:11 not just the marketing ones, but the HR, IR, pricing, org design, and planning ones, the ones most don't consider marketing at all, contribute to either creating value or destroying it. Each week I sit down with CMO, CEOs, founders, cultural thinkers, the people building, breaking, and reimagining how businesses grow or don't, for conversations about what creates value and what destroys it. It's a business show, it's a marketing show. Creator destroys the show that argues. They've always been the same thing from the Vox Media Podcast Network and the Wisdomus Company.
Starting point is 00:23:42 New episodes drop weekly on YouTube and your favorite podcast app. Pope Leo sees work as a, quote, fundamental dimension of the human experience, and AI is threatening that. That's why it was called Magnificent Humanity, essentially. Last week, the tech companies are trying, Anthropic, Open AIS Foundation and several other companies now, It's a $500 million effort, which seems small, to help states and employers prepare workers to the effects of AI on jobs and the economy. Talk about what you're all hoping. You know, I consider it reputation laundering. Maybe it's both that they really mean it or something.
Starting point is 00:24:20 But let's talk a little bit about what the Pope hoped to have happened here, just that the dialogue begins or what impact was the church hoping to have here as a whole. Well, you know, clear political statement that Pope Leo made on day one when he's, explained to the word why he picked that name, was that what the church has to offer to the world is the wisdom of the social doctrine of the church. Now, for people that are not like myself a theologian that work with this word every day, what does it mean? Well, first of all, we are not in a Galileo moment when the telescope was asked to be blessed or to be damned. Social doctrine of the church, simple take the things as it is. The machine is here. The genius is out of the battle, they will remain with us. And now the problem is how much is too much on how
Starting point is 00:25:11 this is the question. And this is a different perspective. But the social doctrine of the church in this shape is a reflection that is 150 year old. So in 150 years, we are really clear state in a multiple occasion that first, we are not the owner of solution. We are part of the debate. We are part of the dialogue. And probably the church is also able to pour to bring voices that otherwise will be excluded. Second things, has to be plural. There is a really interesting debate happening during the 60 and the 70 about unions. Should Christian look just for Christian unions? Well, the answer was really clear. No, because unions is something that is made to make social justice and negotiation of the paycheck is not important that you belong to
Starting point is 00:26:01 the church. It's much more important that you are looking for justice. And so also the solution that probably the church is expecting is not a confessional one, but it's a plural one, something that can happen in a multilateral debate. And this is also politically interesting now, you know, in a moment in which multilateralism is seen like, you know, smoking the eye, it's a stressing things that no one should be excluded. More than one, there is another element. when you talk about regulation, it's really cultural understood.
Starting point is 00:26:38 In China means, yeah, the leading party give the rules that everyone will follow. In Europe is much more understand as an hard law thing. In the United States, we have a totally different understanding. In the social doctrine of the church, regulation is also the name that you give to standard. When we produce a car, we decide that we can drive it on the right, side of the curb or on the left side of the curb, and having such kind of standard is something that is avoiding accident that helped to save life. So the first step is a standardization of AI in which it's really clear and transparent the definition of some of the criteria and that's
Starting point is 00:27:20 bringing this technology to us. Everyone knows that the quality of data, for example, is something that produced the quality of answer. And if there is a demography that is underrepresented, probably the judgment of AI will be not so just on such kind of things. And this is a standard. Should we declare the number of people of the demographic or the quality of data that are behind such kind of system? Or should they can remain secret like the recipe of Coca-Cola or Pepsi? Right, right.
Starting point is 00:27:50 And so you're trying to broaden debate. Now, Pope Leo met with representatives from tech companies, including meta, Google, and Amazon. But do you know about the concerns he shared with them? Or what was the reason? Well, let me say that this is happening in a sort of event that we are calling audience. An audience is a wild animal in the sense that can happen in a one-to-one private room in which you are really free to discuss about everything and nothing happened.
Starting point is 00:28:21 Or it could be a public audience. And can you imagine when you are the CEO of a company that is at the value of $1 trillion? if you say something that can produce the lowering of value. So in that case, they are really, you know, hard-guard related, and it's not easy that personal talking up. Or it can happen in a plural way that is much more official, diplomatic, with a lot of, you know, dancing style. And so the speech that you are talking about happen in a really different condition.
Starting point is 00:28:55 So we can imagine that in one-to-one private-torn. audience. I was in a private audience between Microsoft guys and Pope Francis. The debate was really free. Because you can say what you want and nothing surfaced. In a public moment with pictures, shaking of hands, every declaration could be a nuclear weapon. And so it's much less effective. But Poplar has been trying to reach out to talk to different tech people to also get their insight, presumably. Yes. It's a two-way discussion. But at the same time, there's been some that have been very critical. Billionaire tech investor Peter Thiel.
Starting point is 00:29:32 He was also in Rome ahead of the encyclicals release, where he gave a series of talks, including one where reportedly said the Antichrist would be, quote, a Luddite who wants to stop all science. What is he doing from your perspective? He's called people like me, antichrist, things like that. Probably you think that I'm a communist. Yes.
Starting point is 00:29:50 What do you make of what he's up to? Well, you know, I think that this, like everyone has his own idea, and blueprint, an ideological blueprint. My sensibility tells me that he's not believing in democracy, is not believing in society, and he's believing that it's time for a society
Starting point is 00:30:11 made by individual. But this is good. It's something of the society is remained. So you are looking to trench your own space and one country's passing. But if all the global society collapse,
Starting point is 00:30:27 well, you will have not security. Try to think that. Every time in the morning, I wake up and I would like to wash my teeth. I don't go and make any analysis on the water that is come out from the pipe because I'm believing that the water company, other human beings, other member of my species, are make what is enough to bring me the best water that they can. Welcome to the social contract. So, in a really wild situation in which there is not more any social contract and you are an individual with a series of individual, this is simple something that cannot work. And this is where the ideology has to face with reality.
Starting point is 00:31:08 And so in my perspective, the problem with Peter Thiel is this, is pushing on an idea that is good if you have to capture some values from a still remaining society. If it will produce the collapsing of society, it will be the law of the jungle. Which I think he prefers. I think monarchy is what he's going for. Vice President J.D. Vance is the highest-ranking Catholic in the U.S. government, but I'm sure you remember when he told the Pope to, quote, be careful when talking about theology. He's been a little more muted on the encyclical. And at the same time, he's sort of asked you all not to involve yourself in this thing. Were you surprised by that? And what would you like to see from him as a fellow Catholic? Well, you know, I don't want to be involved in the United States politics. I'm a foreign citizen. I'm a poor Franciscan. let me talk about theology that is easier.
Starting point is 00:32:00 And probably what I see as an Italian with the Italian politics, that what you express in public sometimes as an answer that is a politic ends and is not really connected to the word that you are expressing. So my feeling as someone that belonged to a country with a really complex politics and that probably we have to divide the content of the world by the political ends of the man that was pushing this. And so he is free as a politicians and probably as a brilliant one if he was elected as a vice president to use the political term that he want to make his own politics. Well, if we have to meet in a church talking about the gospel, what the gospel could be a different topic, you know.
Starting point is 00:32:44 But you feel that as opposed to what these guys are saying that the church has every right to be weighing in on this and actually making the debate wider? Well, my really deep faith as a Catholic is that the church cannot stop to talk about the gospel. And if the gospel bring us on some friction is not the first time in the history, we have with a lot of different situations in the past, and we will face also this. We'll be back in a minute. Now, you've said this is just the beginning of the discussion about tech. And I'm going to step back and talk about the idea of the church's moral authority. So every episode we get a question from an outside expert. Here's yours.
Starting point is 00:33:41 I think you'll like who it is. My name is Joshua Benjio. And the big question I would ask Father Paolo Benanti is the following. International collaboration on AI safety is currently being outpaced by a competitive race to develop increasingly powerful models. How does the Vatican intend? to continue leveraging its unique position to unite countries around shared,
Starting point is 00:34:09 human-centered safeguards for AI. Which are the main stakeholders that you believe also need to be involved? Well, you know, Joshua is always brilliant and he put the nail in the cube. The problem is, is it a competition and at always competition, something that is in some way good for everyone?
Starting point is 00:34:33 Well, you know, competition brings a mathematical model that say win or take goals. So if you win, I lose. When we face the most important thing in the world, like, for example, achieving a cure for the cancer, if you and me sit at the same table and you have an idea and I have an idea, both of us have two ideas. So the difference is a zero-sum game and a non-zero-sum games. So if we are looking at the best outcome for humanity, we have to play a non-zero-sum games. That do not mean to not be naive because there are a section like, for example, security, the protection of the most fragile, in which there are to put some guardrail in action. But if the global landscape, we become at zero-sum games, we are not producing any kind of wealth, we are not producing any kind of good results for humanity and probably also for the tech company.
Starting point is 00:35:30 Because, you know, we can imagine a gold rush model, but with a gold rush model, well, you know, I don't know if all the money that are putting in inside this model will bring back some kind of return of investment. And so here it's a matter of a win-win situation, which is not just a matter of being faithful to the gospel, but also a matter to being really rational on what does it mean to look to a real return to everyone and return to someone. to everybody. So what can the Vatican do to continue to leverage this? What is next from your perspective? To keep these human-centered safeguards that was talked about in the encyclical in place. And who are the main stakeholders here? Who do you have to continue to be in dialogue with?
Starting point is 00:36:17 Well, it's easy this kind of question because it belonged to the social doctrine of the church. So I simple have to repeat like in a catechism, some kind of point. Well, first of all, we saw really clearly we have not the way. but we could be the square, you know? It's not easy to have a dialogue if I know that you have billions and I have billions of interest on both sides. But because church is not investing, church has not money that are going on,
Starting point is 00:36:46 it could be a square in which different player come to discuss. Which player? Well, the social doctrine of the church stayed really clearly, and we are talking of the late 18th century where we are not so open like we are now today, that is not for the elected, but it's for every woman and men of goodwill. So everyone that is looking for the best good,
Starting point is 00:37:11 best interest of humanity is welcome here. So the only thing is, are you open to change your mind if you see something that sound really convincing on the topic that we are discussing? This is the entrance gate. And I think that could be workable as a long-time institution that is not the first time that is crossing a changing of time. So we are not the United Nations.
Starting point is 00:37:38 We are not a sovereign state that globally has nuclear power or things like that. But we know something about what does it mean to be human and that there is a lot of voices that are asking to not be simple suppress for the convenience of someone. Right. So you would be a square, an actual public square. I know the tech companies think they're a public square, but they're not. It's a private square owned by them, controlled by them. Probably this is the mistake that we make in the first decade of the center. I've said that.
Starting point is 00:38:07 You know, this has been my thing. I said, Mark Zuckerberg-ons. I give this part of your speech to my student, and also they know very well. It's a private city in which he doesn't help you. You don't get water. You don't get power. And you pay a rent. It's just, it's not a square.
Starting point is 00:38:23 Yeah. And the funny thing here is that we use a word that has a really, high value, that is democratization. Because when, you know, the fighting for civil right produced at a black woman sit on a chair that was reserved to white people, we was democratizing a right that we wrote in a paper but was not active in society.
Starting point is 00:38:46 If now we say that democratizing something is the ability that you have to have a login and password to a social network, well, I feel that we are betraying what does it mean to democratize things. That's correct. Yeah, they love to use a lot of words that aren't true. Anyway, so you write a lot about belief. On one hand, all of Catholicism is based on believing and an all-seeing all-powerful God. In the other hand, we have AI, a technology that, if you said, a, quote, voice that has no face, no history, and you've asked, how do you resist the seduction of a system that sounds like it knows everything, end quote. Talk about that idea, and what can the church teach us about what deserves to be believed. I always say to people, there's nobody there. There's nothing. There's nothing. nothing there. You know, it's the most performative reflection of ourselves in a way that isn't controlled by us. So talk about what you meant by that and how do you resist the seduction of
Starting point is 00:39:39 such a system with church teachings? You know, I'm also a philosopher, so let me twist the two concepts together. I grew up and during my elementary school, my teacher, to make me be practical with a pencil, allowed me to draw mushroom, red mushroom with white dots. Then some other teacher persuade me to not eat because they are toxic. So here is how we become human, not by a set of instinct, by the transmission of the lived experience of the former generation to us. This transmission needs that, I'm sorry, we believe to such kind of things.
Starting point is 00:40:27 You have to believe that the mushroom is toxic without tasting it. Otherwise, you have no chance to get by. And sometimes we believe, sometimes we have not to believe. Otherwise, there is no such things as a progress. And so, human nature that require to believe to someone else's experience is also required to make problem on someone else's experience. Welcome to the human journey that bring us to the moon, but bring also us to the ability to kill someone because it's different by us.
Starting point is 00:40:59 And this is the, if I can express in that way, the tragedy in the sense of the playing that we are doing as a humanity. Now, AI is a new act in such tragedy, in which we shape the way that we give for granted to acquire the quality of someone else's experience. The way that we get was a really architectural way. When we enter in the library of Churchtown, you know that it's huge, it's fantastic, and you have the scaffold with medicine, and you have the scaffold with social science. If during COVID pandemic, I tell to you, you know, if you make a mat wash with vinaigret, you don't get the virus. Do you don't pick this piece of information and bring to the medicine scaffold,
Starting point is 00:41:46 but you bring to probably fancy stories, collect, in a night bar from a drunk at Franciscan. Well, this way work and allow us to grow in science, allow us to criticize some kind of bias that was injected in society. Now, AI is changing. There is not anymore any scaffolding, but the machine is throwing out things. And we don't have a brain helmet, a pair of lenses,
Starting point is 00:42:14 an instruction that allow us to judge what is desirable to be truth and what is not. And this is producing a lot of tension that go between the innovation of science, like, for example, what alpha-fold can produce in drug discovery, and the spreading of things like, you know, conspiracy theories and other fancy stories. Right. And so that is the problem. These are synthetic beings who are just giving us information, not facts, information.
Starting point is 00:42:45 So one of the things that's important is to understand that it's being controlled by, as you know, a very small group of people. in this private square, and Pope Leo warned about the risks of consolidating power in the hands of a few. Talk a little bit about that because he is the Pope, too, and that's the power is consolidated in the hands of the Pope. How could he show being more transparent to make this point? Now, the church is not, as you said, an investor, church is not making money from it. It's just trying to collect souls, presumably, over time. How could he be even more transparent about making a public square more public here? Well, Kara, you know me and you know that I don't resist when I can make a joke.
Starting point is 00:43:27 As a Roman, as an Italian. So let's start from the starting point, the power in the hands of the Pope. You know, the Pope is someone that also confess. And you know very well that between saying something and that the people do what you say, there is a huge, huge difference. So you can be a voice and the voice can achieve the bottom of your heart. heart, but it's a voice. The second thing is really interesting. How can we be sure that the participation is really effective? Well, first of all, because the church is not the authority
Starting point is 00:44:01 that has to put the seal on it. So could be the square, but it's not the means. You cannot enforce these things in the force of the church. And so you need something else. And that means that the dialogue remain as a dialogue. You need something that is an institution. An institution, according to the law or law, according to John Rawls, they are founded on the rule of law and on this idea that the participation and the transparency and the publicity of the ruling mechanism belong to everyone. So the church can offer a space, but the enforcement has to be made through the rule of law. That means there has to be also diverse. And it's parting of the rule of law that there are countries that can feel much more at easy with one way and country that will feel much more at easy in another way. But they democratically and according to the rule of law produce a discussion on this topic.
Starting point is 00:45:00 And now here is a clear window where we see that the way in which we do think is much more important that the result of the things in itself. So one of the things that I've noticed, the moral authority of the church would have come to question as a result of child sex abuse scandals, etc. Had caused a lot of people to lose faith. It seems like this is gaining moral authority for the church again, like focused in on what affects average citizens and humanity because most people in the United States right now tend to agree with what the church is saying here, even if they're not Catholics, you know, in terms of AI.
Starting point is 00:45:37 I know just a personal thing, my son is, I'm Catholic, but I never took my children to church. My second son is going to mass every week now, and I never brought him, which is kind of interesting. But one of the things that attracts him, he's a technologist, is this moral authority of the church on not just AI, but in terms of what it means in a bigger sense. Does this give the church an ability to recover that authority, moral authority? Because it seems to with him, and that's just one case. You know, there is a danger in such question, because it's like if the church would like to have it, like a form of power on caution. Right, right. I think it's a secondary effect of pursuing what does it mean to be faithful to the gospel.
Starting point is 00:46:21 And the moment in the history in which we stress such kind of a concept is also the moment in which we lose it. And so with my feeling as a Franciscan, as someone that belonged to a saint that simple say, respect everyone and don't grasp power on everyone, is if we have them really happy, because means that the gospel is resoning in society, the seeking for true, and the moral reflection that we are doing are shared with people, but this is not the end of the journey. We are not doing that to achieve this. We have to serve the human dignity, Magnifica humanitas is saying.
Starting point is 00:47:00 And if this is resonating with people, well, I'm much more scared that probably the damage that are produced by this transfer of power to few privates end is already produced at deep wound inside the society. Then, if you ask me as a theologians, is it this thinkable 20 years ago? I don't think. But this is interesting because, you know, history can surprise us. Well, one of the things when I think about the idea of what, you know, the church does have a moral authority and they do seem to have their finger on the pulse of what average people think, right, about this thing. And one of the arguments I keep having with tech people is that why are they so reasonable
Starting point is 00:47:42 and you are not. And this is a church. It's a hierarchical church. And so it's a really interesting juxtaposition to have a church talk about democracy in the correct way and the wealthiest people on the planet insisting on their way or no way at all. And so when you think about where it's going, what is your best case scenario here for the impact of this encyclical? And what is your biggest worry? Well, my best scenario is that we open up a way. And so because technology is running, probably thinks we're running much and much and much. And in this scenario, the square is open and we continue the dialogue. And the dialogue will also produce a changing in some view of the church. Because of course, having, for example, what does it mean to have the responsibility as a state
Starting point is 00:48:36 on a lot of people is not so well experienced in the church. And this is something where we can have some kind of enrichment in the dialogue. What will happen? I don't know. I see that the most endangered animal here is not anymore the panda. Could be the democracy, as we know. I think that it's an ideology that is really dangerous. Fukuyama told us that transhumanism,
Starting point is 00:49:06 be the most dangerous things. I think that the end of democracy could be the most dangerous things. Because, you know, in a really technological way, you can understand democracy as an algorithm that produce a result. And I see a lot of these people say, well, it's not enough efficient, simple change algorithm here. But this algorithm costs a lot of blood. It was a bloody century, the century that we shimpole left, if we forgot it, I'm scary that we are betting on someone else's blood to bring back what is important for us. Yeah, that is absolutely. I said to one of them many years ago when I was thinking about income inequality a lot, like in terms of them getting this, this is when they were just a little bit rich. Now they're really rich. Like it's really,
Starting point is 00:49:55 they were super rich, but not like what's happened now. And I said to the person, I've sold this story before that you're either going to have to do something about income inequality and what's going to happen here in terms of people being a permanent underclass, or you're going to have to armor plate your Tesla. That is where we're going with this. If you don't, if you're not careful, and you're all making all the decisions and you don't care, and you are less and less caring about all of humanity and more and more caring about yourself and your group of people. And I swear to God, I looked at this guy and I thought he wants to armor plate as Tesla. That's what he wants. That's the world he does want.
Starting point is 00:50:31 And that's terrified. May I remember thinking? Yeah, you know, the history will judge it. What we are remembering from middle age, brilliant things, or armored things that remain in a museum? You know, I think that the history will judge this. That does not mean that we don't have to act because it will cost a lot of blood, especially of the most fragile people. But this is where ideology came in. If you say it's better an armor, Tesla, that have public square.
Starting point is 00:51:00 in which you can walk and enjoy what is there? I don't know if you are thinking good on this. Yeah, I would agree. Anyway, Father Benanti, I really appreciate it. Thank you so much for talking to me. It's always a pleasure, and I will see you in Rome. I promise. I hope so.
Starting point is 00:51:16 I absolutely will. Thank you for all the job that you are doing. Thank you. Today's show was produced by Michelle Aloy, Catherine Millsop, Tracy Hunt, Madeline LaPlante, Duby, and Kalyn Lynch. Nishat Kerwa is Vox Media's executive producer of podcast. Podcasts. Special thanks to Lisa Soap, Amen Whalen, and Julia Sharp Levine. Our engineers are Fernando Aruta and Rick Kwan, and our theme music is by Tracademics. If you're already following the
Starting point is 00:51:47 show, you fend it off the Antichrist for another day. If not, that's a heresy and you'll need to seek forgiveness. Go wherever you listen to podcast, search for On with Caras Swisher, and hit follow. Thanks for listening to On With Caras Swisher from Podium Media, New York Magazine, the Vox Media Podcast Network and us. We'll be back on Monday with more.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.