Technology, Connected - Your Chatbot Knows Too Much

Episode Date: March 5, 2026

There are now more non-democratic countries in the world than democratic ones. Only a third of Americans under 35 say it's vital to live in a democracy. The share who would welcome military government rose from 7 percent in 1995 to 18 percent in 2017. On this episode of Thinking on Paper, we talk with Carissa Véliz, associate professor at Oxford's Institute for Ethics in AI and author of Privacy Is Power, the Economist Book of the Year, about why privacy is not just a personal preference but the load-bearing wall under liberal democracy. Véliz walks us through what privacy actually is (a right, a duty, and a piece of social infrastructure all at once), why your music taste reveals your politics and your location reveals your religion, and how the East India Company is the historical model for what big tech could become if we keep mistaking convenience for a fair trade. Along the way: why corporate and government surveillance have quietly merged into a single system, how the Nazis' use of personal data shaped the Universal Declaration of Human Rights, why Signal beats WhatsApp on every metric that matters, the difference between behaving like a user and behaving like a citizen, and the line that lands hardest near the end, that democracy is a conversation, and if we leave it to the chatbots we lose our place at the table.Please enjoy the show. --Follow Carissa on XBuy Privacy is Power----⁠Listen to every podcast⁠Follow us on ⁠Instagram⁠Follow us on ⁠X⁠Follow Mark on ⁠LinkedIn⁠Follow Jeremy on ⁠LinkedIn⁠Read our ⁠Substack⁠Email: [e-mail]--TIMESTAMPS(00:00) Trailer(02:26) What Is Privacy(05:31) Is Democracy At Risk?(08:34) Government & Big Tech(10:39) How To Decouple Big Tech & Government(12:33) Privacy & The Common Human Experience(16:02) Tools To Protect Your Privacy(17:18) Cookie Clutter(19:30) ChatGPT Writes Policy(20:05) Radical Open Mindedness(21:52) AI Alignment(22:56) AI Ethics(28:09) How To Erase Your Data(29:27) What Should Humanity Be?

Christopher Gilles teaches AI ethics at Oxford. She's a philosopher, a fellow at Hartford College and the author of Privacy's Power, the ethics of privacy and surveillance, and the soon-to-be-released prophecy. She's here to help us understand the importance of privacy and why guarding it is a communal responsibility. And if we don't work together to protect our privacy, then what's at stake is democracy. And if you like books, stick around to the end.

Chris gives the most human defence of reading we've ever heard. If you want to read more books or you're worried about someone in your life who doesn't read, play them this. It gets emotional. Please enjoy the show. And remember, everything that really matters is analog. Wannabe tech gods would like to profile every single person in a society in order to run a simulation of that community. It used to be the case that Europeans were very careful with corporate surveillance and Americans were very careful with government surveillance. But today, the surveillance is really done in collaboration.

and pretty much all data that big tech collects could end up in the hands of a government and the other way around. Do I want my policymaker to be asking Chad GPT about how to regulate itself? We get people to realize what they're giving up when they trade their sovereignty for convenience. We should remember that life is analog.

Everything that matters is analog. Your body, your personal relationships, where you live, what you eat. None of that is digital and it's never going to be digital. digital. We're acting too much like users and too little like citizens. Technology is not God-given and we could design it in a different way. What is privacy first and foremost? Because I think a lot of us have forgotten what that is. It is three things. It's the ability to keep certain kinds of information to yourself or to share it with just a group of people that you decide to share it with. It's also a right

in the sense of it's a human right and it's a human right for good reason. Not only to protect individuals, but also to protect society and to protect democracy. But it's also a duty. And sometimes we forget this. Because when you don't forget, sorry, when you don't protect your privacy, you're also exposing other people to potential harm and you're eroding democracy. How do we get people to realize what they're giving up when they trade their sovereignty for convenience? Well, my best try was writing privacy is power. But essentially, we talk about this and we share previous experiences and we remember the experiences of the people who have come before us.

So it's not a coincidence that privacy is part of the universal declaration of human rights. And that's because how the Nazis used personal data in the context of the Second World War. It's exactly how they found people and then assassinated them. So it was a very fresh lesson when the Universal Declaration of Human Rights was written. And we revisit those lessons. We also kind of make more apparent the kinds of information that are being collected about people. So I think a lot of us are aware that very abstract information is collected about us, like what we're purchasing or things like our music taste and maybe our location.

But that doesn't really show how vulnerable it makes us. because when you explain to people what location means in the sense of what can be inferred, it means that they know who you live with, where you live, where you work, whether you're going to a psychologist, whether you're going to a church, whether you're buying drugs, all kinds of things can be inferred from location. And when you explain to people that it's not that companies are interested only in their music taste, they're interested in inferring things from their music taste, such as political tendencies and sexual orientation.

So it's partly about knowing a lot more about how privacy works, how it interacts with democracy, how exactly your data is being collected, what data, how is it being inferences, use for inferences, who is it being shared with? And it's also about thinking more deeply about the pillars of democracy. I think it's easy when you live in a liberal democracy and you've been born in a liberal democracy to take it for granted and to forget what it's like to live under an authoritarian and regime and to not understand exactly what makes a democracy a democracy and how we might be weakening it. And one of the points I make is if you think about many of the elements that are

essential for a liberal democracy, such as being able to have a lawyer and being able to have a confidential conversation with your lawyer or being able to be a source for a journalist and for investigative journalism to be able to do research and to publish. All of that depends on privacy. If you don't have privacy, you can't have a confidential conversation with a lawyer. And if journalists don't have privacy, they can't do their jobs. I want to read two quotes from the books. The first quote is an incredible quote from George Orwell.

Political language is designed to make lies sound truthful and murder respectable and given an appearance of solidity to pure wind? The second was a stat that you wrote, quote, only a third of Americans under 35 say that it is vital to live in a democracy. And the share who would welcome military government increased from 7% in 1995 to 18% in 2017. I can assume that that number has increased since then. That's a terrifying statistic.

Is democracy at risk? Definitely. And there are many signs that that's the case. One of them is that every year the economist's intelligence unit makes a study about how democracy is doing around the world. And every year it gets a little bit scarier because we're losing democracy on two fronts. The number of countries that are democratic are fewer and fewer to the point at which now there are more countries. are non-democratic than democratic countries around the world. And the second is that the quality of democracy within democratic countries is weakening.

And so you see authoritarian tendencies and very worrisome tendencies within democracies. And Georgia will also make this point that it's easy to become dissatisfied with democracy. Democracy is horrible. It's very inefficient. It's slow. in every country you can find cases of corruption, cases of injustice, and sometimes really egregious injustice. And Georgia Will makes the point that it's not that in democracies,

those things doesn't happen, they do. But it's how often they happen. And that makes a huge difference. So even if in a liberal democracy in a very high-functioning liberal democracy, you can find instances of injustice, it pales in comparison to the number and the gravity of injustice in a non-democratic country. And I think people forget this.

And when they get dissolution with democracy, they think that an authoritarian regime might be better. And that is terrifying because power corrupts and absolute power corrupts absolutely. And even if we can find instances of dictatorships that in the beginning might seem benevolent. It's a really dark path to tread on. And especially once you take into account digital technology.

You know, I'm partly Spanish and I've talked to a lot of people in Spain who were alive during the Francoist era and who were part of the resistance. And I've been talking about what methods they used to fight against this very, dictatorial regime. And none of the methods that they used would work today because we have so much surveillance in digital tech with everybody with a phone that it wouldn't work. So we are eroding democracy at the most dangerous of times.

And I don't think it's a coincidence. I don't think it's a coincidence that we're seeing the weakening of democracies at the same time as the rise of digital tech. Government is in bed with big tech. And there isn't increasingly less and less of a division between government and technology. Yeah, it's true that it's very concerning how big tech is interacting with governments.

And when we talk about surveillance, you know, it used to be the case that Europeans were very careful with corporate surveillance and Americans were very careful with government surveillance. But today, the surveillance is really done in collaboration. And pretty much all data that big tech collects could end up in the hands of a government and the other way around. I've made this point in writing before about how for people who think that only government surveillance is dangerous, in addition to this point about companies and governments sharing data, there's a point about, again, learning from history about how companies can become quite dangerous to political structures. So if you think about India and the colonization of India, I used to very naively think about Great Britain government colonizing India.

But that's not what it looked like. It was actually the East India Company who had more soldiers than Great Britain. And so we have had the experience of companies that become so powerful that they become as dictatorial as the worst. political regime we've ever seen. And so we should be very careful in making sure that these companies are supporting liberal democracy and not eroding it. You speak about this, the loss of the common experience and the power of data and why this data is so valuable to corporations and go and let's stay on corporations for a minute. And it's this idea of just hyper personalizing everything. Everything is completely personalised to you because they have so much data and there is no common experience. If anybody goes on any news channel, any social media app, everybody has a different story.

They've been told a different story. Is it possible even to have a cohesive common experience anymore? Like, do we have to change the rules of engagement? Do we have to change our idea of what privacy can be and define it as something new that takes? this all into account? No, I think this is partly, has always been there and it's nothing new. I mean, it has new expressions and it has new manifestations of it. But like this is, this is old news in a way. You know, even just like if you go back to the movie that gave the name to the act of gaslighting, part of what the gaslighter does is they isolate the victim and they bend their reality. partly by, through isolation.

And so I think part of the challenge ahead is to remain strong in our social ties. You know, at the end of the day, democracy depends on citizens trusting each other. And you can't trust complete strangers. And so it's incredibly important to have a very strong network of people, people who are your friends, who are your colleagues, who you trust, and then you trust who you trust who they trust and it's a bit of a chain of citizens that holds things together. And we should remember that life is analog. Everything that matters is analog. Your body, your personal relationships, where you live, what you eat. None of that is digital and it's never going to be digital and

everything digital depends on the analog. So to keep the analog world strong, to protect the coffee shop and the pub and the bar and the restaurant and the school and the courtyard and the playground and the places where we meet because at the end of the day that's all that matters and with regards to this extreme personalization

we're seeing something a manifestation of that that is really concerning so Miles Clee at the Rolling Stone has written a series of articles about how chatbots are making people crazy, essentially. They're extremely sycophantic, this large language models like chat GPT. They tend to tell people that they're brilliant, that they're, you know, writing everything, and that has a tendency to spiral into really bizarre and scary conversations. And sometimes

it has ended in tragedy with people thinking, you know, there was this one guy who thought there was a personality in chat GPT called Juliet's and that the CEO of OpenAI wanted to kill Juliette and so he wanted to kill the CEO of Open Eye and ended up with an encounter with the police in which he ended up being killed by the police. And I think that's an extreme version of the same thing. It's being isolated. We partly keep our sanity by interacting with others who challenge our views. And it is very annoying to encounter people who think differently.

But it's partly that they think differently because they have different experiences and you are confronted with them. And if you're just talking to a chatbot, it might seem like the same thing. You're just kind of asking the chatbot for its opinion or whatever. But that's not a person. It doesn't have any experience. It doesn't confront you with reality. And so how we keep our check on reality and how we keep democracy together is talking to one another.

Like we're talking to each other now. Democracy is a conversation. And if we leave it to the chatbots, we're just, you know, losing our place at the table of democracy. I want to circle back to your suggestions. So you said, WhatsApp, here's the option, Signal. Can we go through a few more? Because Mark and I are a guilty party. We literally have been texting on WhatsApp in show prep, and we do that quite a bit.

So there's Signal. What are some other things that we could make better choices with to protect ourselves and our data? So instead of using Google Search, use duck, duck go or start page. There are a few options. instead of using Dropbox, use proton drive, use proton VPN, instead of using Gmail, use proton mail. And, you know, full disclosure, I'm part of the board of the proton foundation. But that's because I believe in them.

What else? Just in general, I don't know. It used to be the case that it took a lot to get off your phone. It was like a sacrifice. You knew that your phone was bad for you, so you made it fast. The sacrifice are staying off of it. But more and more, I'm finding my phone so incredibly annoying. And it's becoming easier and easier to ditch it.

I'm just sick of it, literally sick of it. And I think that's a good thing. In general, like, if you want to have a good party, ask your friends not to upload any photos. If you want to have a good conversation with your students, turn off the cameras, turn off the microphones. If you want to have real time with your children, with your family, leave the phone. One of the biggest frustrations I have now is cookies that demand GDPR has created even more friction where a cookie will pop up and you can't just deselect all. You have to, I just, okay, I won't go on that page in the end. Sometimes the guardrails, the protection, cause more friction and more frustration.

Well, no, that's just a bad law. And it's not bad because people were stupid. It's just at the time, we didn't have enough clouts to do better. You know, if the default was no data collection and you had to opt in for data collection, then you would only have to do it once, so it would be a lot less burdensome and a lot more rationales. We shouldn't assume that guardrails are going to be a headache. No, not the right kind of guardrails. But yeah, you're right about how frustrating it is. And I think, I would find it very depressing if somebody were able to add up the hours I've lost of my life saying no to cookies. When so many people are outsourcing their decision making to, what's that eroding? What's that removing from us, this outsourcing of decision making, this outsourcing of creativity, the outsourcing of work? A lot of things. And I think we could spend a whole hour just on this question. But perhaps the thing that worries me the most is that we, we're becoming dependent, and that is the first worry.

Because when you get used to something doing things for you and you forget how to do it yourself, then you are a lot less autonomous. And people being less autonomous means that's bad for democracy because we're depending on something. But furthermore, it's not just any company that we're depending on. We're depending on companies that have proven themselves to be notoriously untrustworthy.

And thirdly, it's not any kind of product. we're depending on. We're depending on a product that is, again, notorious for being unreliable, for being glitchy, for coming up with confabulations and fabricating information. So that makes us a lot more brittle. We're using this in very sensitive context like justice and like health and education. So one of the main worries is that we are losing critical thinking skills. You know, one of the funniest, and in a funny, not funny kind of way, story was Chris Stokel writing about how the minister in the UK that deals with digital affairs was using ChachyPT to ask it about policymaking with regards to technology. And you're thinking,

do I want my policymaker to be asking Chad GBT about how to regulate itself? No. I think one of the biggest pieces to activate critical thinking is having open-mindedness. How do you land on open-mindedness as an activator of critical thinking? That's a good question. I don't know. And it depends how we cash out being open-minded. I think that there is a lot of value in trying to understand another person,

what it's like to be them, where are they coming from, where are they feeling the way they are, what is their argument that they're making, what are the premises, what's the conclusion? That is incredibly important. But I think sometimes we think of ourselves as being open-minded when we accept uncritically certain kinds of views And so I think somebody who criticizes tech, for instance, can be accused of not being open-minded enough about how, you know, oh, yeah, it's just like you're not used to it. And if you were more open-minded, you would just be enthusiastic about all of these products that these companies are selling us.

So I wouldn't want open-mindedness to be confused with lack of good judgment. I think to have good judgment, you also have to be discriminatory in the sense of knowing what is a good judgment. argument and what isn't and what is good evidence and what isn't. And so I think I would be, yeah, I'm not sure it's my favorite term because I think it can be confused with other things that are not as constructive. But maybe... Are you saying being open-minded, if you're too open-minded can be, you're at risk of... Too malleable, yeah, maybe. Believing something which just shouldn't be on your radar. Yeah, I think I'm just saying, don't be uncritical. Be open-minded, but never at the cost of being uncritical.

I've been thinking about the internet and if all of the data on the internet, all of the information on the internet, all of the stories on the internet, is enough to align AI with these analog values that you've outlined. And let's say it isn't. How do we, and very quickly,

because time is of the essence, How do we align AI more precisely or more accurately with our values? So the way you phrased the question is really interesting because I think it includes a lot of assumptions that I would like to push back against. Please do. So the first assumption is like in order to align AI with our values, we need a lot of data.

And is all of the data in the internet enough? You said enough as if, oh, if only we have, more data, then it will be enough. And that is, of course, what Open AI, you know, really wants us to believe, partly because they're asking for all the data available and then some. I think that's the wrong way about it. Ethics is not about the amount of data we have. The second assumption is that we can, yeah, this term alignment has been associated. And this is a cultural thing, and it it needn't be like this, but of late, it has been associated very closely with tech company's own vision of how to consider ethics in the context of AI. And of course, tech companies

are not the most ethical companies have encountered. And so here's an unpopular view. Questions of justice and principles and the rule of law are incompatible with machine learning. So machine learning is a statistical analysis of data and it outputs likely responses or actually responses that human beings are likely to find attractive. Not true, not ethical, attractive for human beings.

And that's not a good way to design ethics. It's just not. So I'll give you an example because this is sounding a bit abstract, okay? You go to a bank, you ask for a loan. If we use AI, machine learning, then you get a response, yes or no. Why did the algorithm say yes or no? We don't know.

Does the data scientist know? No. Does the banker know? No. It analyzed data. Your data, it compared it to a lot of other data, are people who have paid their don'ts or not, and then came up with a response. And sometimes when you look into it, it turns out that it was tracking your race,

that it was tracking your name, that it was tracking things that it shouldn't track. Now, how do we make... an ethical decision based on our values. We put transparent criteria. We say you have to have X amount of dollars in your bank account. And if you don't, then you know how to fix it. But that's not machine learning. And it's not the kind of AI that we're using now.

And so how do we align AI with our values? We don't. We don't. We either program it, program our values. We set a set of transparent criteria. that people can know and can appeal, and then we just make it obey it, but we don't use it to come up with those rules.

And again, in a rule of law, in a liberal democracy, people need to know what the rules are. Rules need to be public. Laws are public. Right now, we are ruled by rules that we don't know and we don't understand and we can't appeal. And that is the definition of Kafkaesque. codifying kind of human values to integrate into these models.

What would be three of those values be to show a machine what the important things about being human are and what the important things for that machine to recognize about humanity to make it safer for us? I don't think it works out like that. You know, there aren't three things that you can codify. easily. But a good start would be human rights, you know, the list of universal human rights. And one of those is privacy. And just with that, I mean, it would mean a huge difference,

right? Because at the moment, all of this technology is based on surveillance. And that is kind of mind-blowing. We've become used to it. But if you told someone in the 1950s, you know, we're going to have this system. And it's got to rely on surveillance the way we think it's okay to surveil a criminal, but not anyone else, you know, in the 1950s, people would be shocked for good reason. How do you forget in a system that never forgets? Like, is there any way that I can erase my footprint and get back some of the privacy I gave up? Yes, and no. There is a lot you can do.

You can ask companies to delete your data, and it makes a difference. People are too black and white about privacy. They tend to think like, well, if my data is out there, then there's nothing I can do, and I might as well share more data. And it doesn't work that way. It's a bit like eating sugar. It's like saying, well, I've already eaten one cupcake in my life, so I might as well, you know, have breakfast, lunch and dinner, just like buckets of sugar. Well, it doesn't work that way, right? The more sugar you have, the more you expose your health to risks. And in the same way, you can delete your data, you can also obfuscate, as I explained in the book. So let's say that your date of birth is out there. And let's say that you've tried to delete it, but you've been unsuccessful. Then you can

give other dates of birth. And suddenly it's not as easy to figure out which one is the right one. So there's a lot you can do. And everything you do makes a difference because you never know which is a data point that might actually risk something important, like your bank account or or whatever it might be. Kevin Kelly left a question we asked for all I guess at the end. His question was, what should humanity be and how does technology help us get there? I think humanity should be kind.

And technology can be a tool in helping us. So every time we use technology to communicate in kind ways, it can help. but fundamentally it's not what's going to make the difference. In a way, the technology that I find most incredible and most conducive

to becoming a better human being is books. Books are incredible. They allow you to have many lives in one to learn about what it's like to be a human being in so many different kind of circumstances that you might never find yourself in.

It allows you to communicate with people who have lived across the world and thousands of years away from you. And it allows you to communicate with people who are not here yet to tell them what it's like to be a human being now, what you've learned and what you might have learned that might save them time and pain. And that is part of being kind.