TRIGGERnometry - Privacy is Power - Carissa Véliz

Episode Date: September 16, 2021

Carissa Véliz is an Associate Professor of Philosophy at the University of Oxford and the author of Privacy is Power https://www.penguin.co.uk/books/112/1120394/privacy-is-power/9780552177719.html Get TICKETS to TRIGGERnometry Live with Peter Hitchens here: https://leicestersquaretheatre.ticketsolve.com/shows/873620658 Join our exclusive TRIGGERnometry community on Locals! https://triggernometry.locals.com/ OR Support TRIGGERnometry Here: https://www.subscribestar.com/triggernometry https://www.patreon.com/triggerpod​​​ Bitcoin: bc1qm6vvhduc6s3rvy8u76sllmrfpynfv94qw8p8d5 Buy Merch Here: https://www.triggerpod.co.uk/shop/​​​ Advertise on TRIGGERnometry: [e-mail] Join the Mailing List: https://www.triggerpod.co.uk/sign-up/​​​ Find TRIGGERnometry on Social Media: https://twitter.com/triggerpod​​​ https://www.facebook.com/triggerpod​​​ https://www.instagram.com/triggerpod​​​ About TRIGGERnometry: Stand-up comedians Konstantin Kisin (@konstantinkisin) and Francis Foster (@francisjfoster) make sense of politics, economics, free speech, AI, drug policy and WW3 with the help of presidential advisors, renowned economists, award-winning journalists, controversial writers, leading scientists and notorious comedians. Learn more about your ad choices. Visit megaphone.fm/adchoices

We are creating an architecture of surveillance that is so good that if it gets taken over about a bad government, we are in serious trouble because it will be impossible to resist. Hello and welcome to Trigonometry. I'm Francis Foster. I'm Constantine Kisson. And this is a show for you if you want honest conversations with fascinating people. A brilliant guest we have for you today. She's an associate professor at Oxford University and the author of Privacy is Power, a brilliant book. Carissa Velis, welcome to Trigonometry. Thank you so much for having me.

It is great to have you on the show. Listen, before we get into talking about the subject you covering your brilliant book, just tell everybody a little bit about who are you, how are you where you are, what has been the journey that leads you here to be sitting here talking to us. I'm a philosopher because I was never good enough at jobs to be a comedian, although I would love to be a comedian. But, you know, second best is philosopher. and I studied philosophy as a BA, MA, and then PhD.

I was writing my dissertation on something related to ethics, but very different. And then I started researching the history of my family. My family were Spanish refugees in Mexico from the Spanish Civil War. And they had never talked about the war. It was a very sensitive topic. And I went into the archives and uncovered so much about them that they hadn't told us. And it made me wonder whether I had a right to know these things that they hadn't told us, whether I had a right to tell my family, or maybe even publish about it, because it was so interesting.

And being a philosopher, I looked into the philosophy of privacy and realized there was a huge gap in the literature. There was very little written about it. The literature that there was was kind of outdated and we didn't really address the questions that I was asking myself. And then that same summer, Snowden came up with his revelations that we were being surveilled at a mass scale. And I thought, I just, I need to change the topic of my dissertation. So I did, and I started researching about privacy, which eventually led me to write this book, Privacy is Power. And there's so many fascinating insights in your book. I'm actually going to quote you in my book because you talk about some stuff that's really not get, doesn't get talked about much at all.

And we'll get into some of that. What were some of the things that shocked you or surprised you the most when you were doing this research and talking about privacy? It was shock after shock after shock because I think one of the interesting things about studying data is that it's so abstract that we're just not made to understand that kind of thing. Our psychology just doesn't work that way. So even now when I read things that I have written myself, you know, that I have researched and written and I know them by heart, I still get shocked because it just does, it's so invisible. So I'll give you a few examples. Actually, the most shocking example and a most concerning example, I decided not to publish it because I think it's so dangerous that anyone could misuse it, and I wouldn't want to facilitate that.

But some examples of just corporate surveillance, your phone at night is sending information that it has collected throughout the day, and it sends us at night so that you don't notice that your battery is draining, because, you know, people typically connect it at night and charge their phones at night. And companies are tracking very sensitive things like whether, you sleep okay at night or not, at what time do you wake up, with whom do you sleep, which can tell people, you know, who is your partner, but also things like whether you're having an affair. If you have a smart car, it's tracking not only where you go and how fast you drive and how well

you drive, but things like the music you listen to and what that tells about your mood, but also the seats in your car is measuring your weight and, you know, potentially selling that information to insurance companies who might want to know, you know, if you're getting a bit too slim or a to fat. They know things about your health records, your educational records, your purchasing power, your browsing history, and people search the things that they care most about,

the things that worry them, their diseases, whether they can pay their loan. So it's very, very sensitive. It's just shock after shock after shock. And Garisa, why is this a problem? Well, you know, because people go, well, so what? All they're doing is collecting data.

What does it matter? It matters because privacy is power, because whoever has the data in the digital age will have the power because data gives not only the possibility of selling that data, which gives them money, which already makes them very powerful, but it gives them the possibility to try to predict what you're going to do next and try to influence that and change that. And that is incredibly attractive for companies, but also for government.

And it's something that we haven't been focusing on, enough. I was reading this book by Bertrand Russell, the philosopher, called power, and he argues that we should think about power as something like energy, in that it can transform itself from one thing into another. So if you have enough economic power, then you can buy votes or buy politicians. If you have enough political power, you can buy military or get military power and so on. And there's this really important kind of power in the digital age having to do with forecasting and prediction that has always been there in a way. We've always known that the more knowledge you have on somebody, the more power

you have over them. But we have never had this capability of amassing so much data and of analyzing it. So it matters because the more others know about you, the more you're vulnerable to them in all sorts of ways that are invisible to you. But you might get discriminated against for a job or a loan application or an apartment. You might get extorted. Your identity can be stolen and your democracy can be stolen as well because it influences how we relate to one another as citizens. And in your book, you use many, many examples. The one that I found particularly powerful was the example of the man in Virginia. Oh, yeah. So I have a friend who was training at the time to be a data analyst and he was telling me like what it's like to go through that training. And one of the

exercises that they had to do is just pick a random person anywhere in the world and just research anything and everything you can about them. So this random data analyst picked this random guy in Virginia and he learned everything about him. So if I remember correctly, this was a man who had diabetes, who was having an affair, he knew what kind of car he drove, what job he had, who were their friends, who were their family. And this person had no idea that somebody was completely taking off his clothes metaphorically online. And you know, I think one of the concerns that people would have had in the past is that the government is watching you, right? The government is surveilling you.

And as we now know, as you say, from Edward Snowden, they were right to have that concern. But I think, am I right in thinking that now actually the biggest threats in terms of our privacy, in terms of being watched, in terms of being controlled, in terms of people kind of untracking us and trying to influence and shape our behavior? It's not necessarily the government itself. It's actually corporations, big tech. people who want to sell you shit, basically. Yeah, that's partly right, because most of data is actually collected by companies,

and only then do governments make a copy of the data, but it's really the companies that are best and have more resources to collect data. And they have many reasons to use that data against you. Like I mentioned, you can be discriminated against in all kinds of settings. So essentially, they're undermining equality and equality of opportunity. You are not being treated as an equal citizen. and you're being treated on the basis of your data. But at the same time, we shouldn't forget government

because it almost makes no sense to separate the corporate surveillance from the government surveillance right now because they share data all the time. So every time a company collects data, that data can potentially go to the government, and the government very often just makes a copy of the data immediately. But also, every time the government collects data, that data also ends up in the hands of corporations.

So we've seen this in the coronavirus pandemic, in the UK, the NHS has given data to Palantir, this very shady data company that was partly funded by the CIA. And they gave data not only that might be more kind of, I don't know, understandable about people's health to fight the coronavirus pandemic, but they also got data, for example, about people's criminal records. And there was no explanation as to why exactly this company needed that data and what's going to happen to that data. So the flow of information goes both ways to such a large extent that it almost makes no sense to to differentiate between them. And the other thing that people often say when I've talked to

people about this is it's always the same thing, which is, well, look, I'm not doing anything wrong. I don't care. What should I worry about? What do you say to people who think about these things that way? Well, there are at least two responses. One is you actually do care because nobody wants to have their identity stolen. That takes, you know, that's a hassle. It can actually get to jail without having done anything wrong whatsoever because somebody else uses your name to commit crimes in your name. But also, you are vulnerable in all sorts of ways, even if you do nothing wrong. So, for instance, maybe you have a disease or maybe you have a disease that you don't know about and that a company wants to pick up on before even you do and then discriminate against you

next time you ask for a loan or a job or something like that. So there are all kinds of reasons why our privacy is important even if we do nothing wrong. But furthermore, even if you didn't care about yourself, you just said, you know, I'm a masochist. I want to be stolen and I want to be exposed and extorted. I'm fine with that. It's just an interesting experience. You should protect your privacy because privacy is actually a collective thing.

This narrative that the tech companies have sold us that privacy is just a personal preference, is something individual, and if you're not shy and you're not a criminal, then you have no reason to protect your privacy. It's totally misguided. because when you expose yourself, you expose others. So every time you share your location, you're sharing data about your neighbors and your coworkers.

Every time you share data about your genetics, you're sharing data not only about your siblings and parents and kids and cousins, but very distant kin that can suffer really bad consequences, like being deported or being denied health insurance or life insurance, even if they didn't do the test themselves and you don't know that you're actually, kin and you've never met that person. And in the same way, society benefits from people protecting their data.

So one example is Cambridge Analytica. Only 270,000 people gave their data to the firm. And with that data, the political firm managed to get their hands on the data of 87 million people who were the friends of these original 270,000 people, but who didn't consent to anything. And then with that data, the company made a tool that was, supposed to profile voters around the world. So that's a very clear case of how those persons didn't have the moral authority to share

their data because that had consequences for everyone else. And I think people often massively underestimate the predictive ability of individual bits of information. So I remember during the Cambridge Analytica thing, there was a website where you could see what their website would essentially predict about you. And when I did it, my Facebook profile literally just had like my music and movie preferences on it at the time. And when I put that in, it was like terrifyingly accurate about my political views, about all sorts of other things, just based on the movies and the music that I had happened to enter. And this is one of the things I think that's massively underappreciated, just how much you can predict about a person with often things that aren't even necessarily.

relevant to the thing that you're predicting. Like, you know, we had Dr. Pippa Malmgram on the show talking about this, how people who eat blueberries are more likely to be conservative or whatever. Like stuff that you would never think is actually predictive can be used for that effect, right? Exactly. That's super important. When people think that they're sharing their data, they think about the data that they think they're sharing. And then they say, you know, what do I care if Facebook knows my music preference?

But what we don't imagine is that being used to calculate our sex, sexual orientation or IQ, and that's being used by companies when we ask for a job or something. So one example was, it turns out that people who like a Facebook page of curly fries have a very high IQ. And of course, when you like a Facebook page for curly fries, it's never going to cross your mind that that's going to be used to calculate your IQ. The hypothesis is that this Facebook page was probably created by somebody very smart, very smart people tend to have friends who are also very smart and that's probably the explanation. But that's one of the problems with algorithms, that they always work on correlations, not causation. And there's no way of knowing

what they're going to correlate. So the concept of informed consent doesn't even make sense with data because you don't know what you're consenting to because you don't know what kinds of inferences are going to be made with that data in the future. I bet you like Curley Fry's page, mate. Yeah, I do actually. I always thought I was very intelligent as a result. Carissa, why is it? So we've seen what's happening. We've seen that there's been a massive data grab by these companies. Why haven't laws been put in place to stop this kind of behavior? It's a good question and it's complicated.

The first reason was because governments, after 9-11, were very scared about what happened. They wanted to prevent it at all costs. And it was intuitive to think that the more data they had on people, the more they could do something about it and keep people safe. Now, it just turns out that big data is not the kind of an analysis. that is good for preventing terrorism. Big data is fantastic at knowing what you're going to buy tomorrow because we have data from billions of people who buy things every single day. But terrorism will always be a very unusual event and that makes it very hard to understand for big data.

So the first reason was because governments thought they had an interest in keeping the data. Now, you know, 20 years have passed since then and I think governments are slowly learning that having all that data stashed away is a national security. danger. So I think that's going to motivate them to change the law. But another reason why laws haven't been put in place is because data is very hard to police. So actually, the GDPR is quite good. It's not perfect. It has a lot of flaws. But it was a very important step in the right direction. But it's just impossible to police because everybody's breaking the law all the time. Many times citizens can't even denounce it because we don't even know what's going on. It's

not like we can see our data being stolen and it hurts, right? Or you can't breathe or something like that. It's not tangible. So how are you going to complain if you don't even know that it's happening? So the implementation of the GDPR is facing a lot of difficulties. They're underfunded as well, the data protection agencies and we're dealing with giants. And the third reason why we don't have more strict loss is, you know, there's a lot of money involved and there are a lot of interest and these companies lobby really hard and they pay more than any other company in the world to pressure politicians. So it's a big challenge, but I am cautiously optimistic that eventually we're going to regulate it because this is just unsustainable. It's a ticking bomb. And you say it's a

ticking bomb. Have things got worse under COVID? Yes. Because, yeah, I knew what the answer was. Can you explain why, though, Carissa? Yeah. So one reason is that we've just been forced to use digital stuff more and more and more. And, you know, one advantage is that the narrative that, you know, we can always opt out and it's up to us kind of has totally disintegrated, right? It's obvious to everyone that you can't opt out. If you want to be a participant in society, there's no way to opt out of interacting with the digital. But we have been forced to use it a lot more and therefore a lot more data is being collected. Also, times of crisis are notoriously dangerous for civil liberties. These are times in which governments very often pass laws and measures that wouldn't be accepted in other

circumstances. And so in this case, I think there was a well-intentioned idea that the more data we have, the more we can have tools to stop COVID. It turns out that that actually hasn't been the case once again, that AI hasn't been helpful to stop COVID. And, you know, the tracking applications haven't been the most important aspect of fighting COVID. But the measures are, there and they don't have a sunset clause. So we don't know when they end and what happens to all that data that has been collected. And one thing the government

has done, and nobody's talking about this, but I think that this is absolutely major. They've got rid of cash. Can you explain why that is such a disastrous thing for our society? Especially for me, because I don't want to pay tax, but that's beside the point.

Which cost you're from again? Anyway. People are not talking about it. that enough. But cash is very important because it's the only way in which citizens can buy things without being tracked. And of course, you know, if we don't have a society in which we are interacting with each other, we're just interacting through online. We're all using credit cards. We're all using PayPal. And that gets tracked really easily. And so there's no way to buy things or services in a way that hasn't been tracked. And you can say, well, but that's great because

that fights crime, right? And yes, it fights crime. It can be helpful to fight crime, but it also does away with certain kinds of privacy that are really important. So here are a few examples. Paying a lawyer. That's very kind of revealing about what you're interested in or what you might be worried about. Paying for a psychiatrist or a psychologist or other kinds of support. Buying books. So books are very revealing, again, of what you're interested in and what you're worried about.

and a lot of people, I think, are very complacent because they think, well, we live in a democracy. Our government is not out to get me, you know, for what I read. And fair enough. But right now, the Taliban have gotten a hold of U.S. biometric systems. And the point is that we are creating an architecture of surveillance that is so good that if it gets taken over about a bad government, we are in serious trouble because it will be impossible to resist. And even if, I mean, I think it's very complacent to think that our government will always be benevolent and democratic because the best predictor that something will happen in the future is if it's happening in the past. And it's happening in the past that we don't have the best government possible.

But even if you thought, okay, no, this country is amazing and it's always going to have the right government, can you be sure that in 10 years time we're not going to be invaded by another government, by China, by Russia, I don't know, by somebody else? I think that you have to be really confident in your like soothsaying abilities to not be scared about having this architecture of surveillance. And the thing that I've been worried about, Carissa, throughout this entire pandemic, is the government has tried to introduce mass surveillance in the form of the vaccine passports. And people have been so complacent.

They've been like, they've just been while, well, what's the problem? Yeah, I think it's really hard to be critical in times of crisis because the kind of opposition you face is, oh, well, we need to save lives. Aren't you in favor of that? Of course, everybody wants to save lives, and so we don't want to be a problem. But I think we have to question not only whether we need vaccine passports, but even if we accepted them, in what format do we need them? So one question is, why do they have to be digital?

We've worked very well in the past with paper passports and paper certificates of different kinds. The problem with digital is that it can be hacked. It collects a lot of data. And there's just no justification. People just assume today that it has to be digital. You know, coming back just a little bit to what you were talking about in terms of the possibility of the data being used by foreign actors. I mean, you don't even necessarily have to be invaded by China.

The Chinese could just hack the database and then influence elections, cause civil unrest, you know, undermine democracy in one way. where another. And, you know, we saw in 2016 the allegations about Russia, you know, where I'm from, you know, causing Brexit, getting Trump elected. It doesn't look like that. That was what happened based on some of the things we've seen. But it's certainly not impossible, right? And vice versa, you could see powerful Western actors, the United States and Britain influencing elections in other countries using exactly the same technology. So, but before, you know what? Actually, actually,

before we talk about that, let me ask you the counter argument, which is increasingly my job on this show. A lot of people might say, well, I hear what you're saying, but on the other hand, doesn't big data have huge potential for identifying disease and people who don't even know they have it and therefore getting treatment ahead of time? You know, if it turns out that, you know, buying more of something means you've got cancer, don't we want to know that so we can stop people from dying of cancer. Don't we want to prevent certain things from escalating that we could prevent? Surely those are all good things, Carissa? Absolutely. And I have a whole section in the book about medicine because it's such an important point. And medicine is such a good example

of that. The first thing to say is that it has a lot of potential, but we shouldn't like give up all our dates and our civil liberties for something that has potential. We have to have some evidence that it's actually going to work, right? Having potential is like really dreamy and I'm fantastic and promising, but that's not enough. And it's been very, very disappointing how millions and millions and millions of pounds have gone into AI tools during the COVID pandemic. And the two major studies that have researched this claim that out of the hundreds of tools that have been developed, one of them is clinically viable. And that should make us think twice. Like, are we making the right decisions here? It's not only about privacy, it's also about

resources. Are we putting our resources in the right place? But let's say that, you know, it can work or like we should give it a shot or we should research. The devil is in the detail. It's not that we shouldn't use personal data at all. We can use it and especially for medicine. If you go to your doctor and you don't want to tell them what's wrong with you, they're not going to be able to help you. It's as simple as that. But we don't need to buy and sell personal data for that. We don't need to allow data brokers to know everything about us.

We don't need to allow the government to know everything about us. So there's a huge difference between using personal data for things that are important and then having a market of personal data in which anyone can buy it, the highest bidder gets it. That's very different, and that's what we should avoid. I mean, people might give another example of, let's say, the lending, which you've referred to a number of times already. You could say, well, they're able to make better decisions. I mean, 2008 would suggest otherwise. But in the past, if you wanted to get a mortgage, you'd have to go to the bank,

and the guy in the bank would have to make a personal decision based on. how you were dressed and whatever else. Now we have all this data, which is very good at predicting whether, you know, you're going to be able to repay your mortgage. Surely that's a good thing because, you know, this is the argument. We're not giving people debt. They can't handle, et cetera, et cetera. Well, it depends.

It has to be shown. It's not enough to just say, this works. I want to see how it works. And it hasn't been shown that way. So, for instance, you know, one argument is your banker might have all kinds of prejudice. They might be a racist. They might be a sexist. That's actually quite plausible.

and even likely. But it turns out that algorithms are also... You've clearly met a lot of bankers. No, I actually haven't. But it turns out that algorithms are just as biased or worse than people. So one thing that I have proposed, and again, you know, it depends on the kinds of proxies

that these algorithms are using. So in many cases, it's unfair that we are being treated as a category and not as an individual. So say you live in a certain kind of postcode in which people tend to not pay their loans, but you might be different, right? You might be like super serious and super good at paying back your loan,

and you're not going to get that loan because your neighbors don't pay their loans or your friends on Facebook don't pay their loans, and that seemed quite unfair. So one of the things that I have proposed recently in an article for the Harvard Business Review is that we should only allow algorithms that have really important decisions to make,

like giving somebody a loan, to go out into the world if they have passed a randomized control trial. So just like we do with medicines, we don't allow any medicine to just go in the market without having been tested, not even in a crisis like the coronavirus pandemic. We had to make sure that the vaccines were safe and they went randomized control trials. Well, in the same way, an algorithm could go through a randomized control trial. We could have an agency like in the United States, the FDA, the Food and Drugs Administration, to make sure that an algorithm is safe and to prove that it's actually making better decisions than the banker. Carissa, isn't part of the problem here that we have this new technology, it's brand new,

we don't actually understand the full ramifications of this technology. Yeah, so it's really reckless to just let it lose into the world. Essentially, we're treating people as guinea pigs, and that's totally wrong. And we used to do it with medicine. So when you went to the doctor in the 1950s, you could get signed up for an experiment without knowing. And after the Nuremberg Code, which is like one of the most important medical ethics codes, we decided that we don't experiment on people without their consent. So if you want to carry out clinical experiments, you get, you inform people about what you're doing,

you get their consent, and then you give them some kind of compensation. And we should do the same thing. And right now we're not doing it. We are being guinea pigs all the time of algorithms without even knowing it. And one of the things I found really fascinating about your book, and this is the bit that I mentioned I want to reference in mind, is the fact that people often don't understand that intentionality and malice and the deliberate evil is actually completely unnecessary for harm to be caused.

So the collection of data today completely innocently or indeed for benefit. purposes will often end up being useful to people who do want to use it for evil. Can you talk a little bit about that? Yeah, that's really important to have in mind because I don't think that, you know, I think that most people in tech and most people in finance and most people just have good intentions, or at least not bad intentions. They just want to do well in life. They want to innovate and so on.

But Hannah Arendt had this amazing term called the banality of evil. And the idea is when we think of evil, when we think of the Nazis, we think of the paradigm of somebody like Hitler who, you know, you imagine somebody who hates Jews and wants to hurt people and wants to kill people. And that's our paradigm of evil. But in fact, most of the time evil gets perpetrated by perfectly normal people who are just bureaucrats. And that was her conclusion with the Eichmann trial that this guy was just a bureaucrat. He wasn't a monster. He was just following orders. He wasn't critical. he was just a cog in the system. And it's really important that we don't become cogs in a system that creates injustice and that erodes democracy and even creates evil. And it's not enough for us to think of ourselves as good people and have good intentions. We need to be a lot smarter than that and a lot more critical to avoid evil. And one of the examples you give on that very issue, speaking of the Nazis,

is the fact that survival rates in different European countries for Jews, were different because they had different practices of collecting data. And those that collected data by ethnicity, which may have been perfectly reasonable under a legitimate, democratic, sensible, non-discriminatory government, then fell into the hands of people who wanted to use it to discriminate, to kill, to murder, to imprison. And that innocent collection of data led to more people being killed because of it, right? Yeah, I think it's a perfect example to show that personal data is a ticking bomb. So the Dutch had a very good system of statistics.

They had a guy called Lenz who was one of the pioneers of statistics, and he wanted to build a system that followed people from cradle to grave. And in his census, there were a lot of questions, and there was data collection about your religious affiliation, but also your ancestry and things like where your grandparents lived. Now, in contrast, in France, they had made a decision since 1872 not to collect. that kind of data for privacy reasons. And so when the Nazis arrived to France and asked, you know, where are the Jews? They said, you know, we have no idea how many Jews. We have,

let alone where they live, so good luck with that. And the Nazis had to depend on either Jewish people turning themselves in or having neighbors, neighbors turned them in, which was very inefficient. And the result is that in the Netherlands, the Nazis found and killed 73% of the Jewish population. And in France, 25% of the Jewish population. And the difference is hundreds of thousands of people. And if this has happened in Europe already, we really have to make sure that we don't make the same mistake again. There's a reason why privacy is in the Declaration of Human Rights. And we have forgotten that lesson, and we have to relearn it pretty quickly before something really bad happens again. And there's one story in particular in the Second World War that I think

is very illustrative of what we need to avoid. And that's because the Nazis, one of the first things they did when they invaded cities was go to the registry because that's where the data was held. There was a resistance cell in Amsterdam that wanted to destroy the registry in 1943. So they went into the building, they sedated the guards to spare their lives, they set fire to the records, and they had a deal with the fire department that they were going to arrive late and that they were going to use more water than necessary to destroy as many records as possible. And unfortunately, they were very unsuccessful. They only managed to destroy about 15% of the records.

they got caught and killed, and the Nazis found 70,000 Jews in Amsterdam. And the Dutch had made two mistakes. One, they had collected too much data that wasn't necessary to have a functional society. And the second one is that they didn't have an easy way to delete that data in the event of an emergency. And we are making both of those mistakes at a grand scale. And that should make us think twice. Do you have a website or do you plan to have a website? Well, if you do, then EasyDNS are the company for you.

EasyDNS is the perfect domain name, registrar provider and web host for you. They have a track record of standing up for their clients, whether it be council culture, de-platform attacks, or overzealous government agencies. Eno's a bit about that. So will you in a second. EasyDNS have rock solid network infrastructure and incredible customer support. They're in your corner, no matter what the world. throws at you unless it's your ex-girlfriend, in which case you're on your own.

You'd know about that. Move your domains and websites over to EasyDNS right now. All you've got to do is head over to EasyDNS.com forward slash triggered and use our promo code, which is of course triggered as well, and you will get 50% off the initial purchase. Sign up for their newsletter, access of Easy that tells you everything you need to know about technology, privacy and censorship. Carissa, what would you say to those people go, look, the cat's already out of the bag, there's nothing that we can do. These companies are so huge, they're so powerful, they're so wealthy,

and we're just nothing but a collection of individuals. I would say that's a lack of historical perspective. We have had very powerful companies in the past. We've regulated from, you know, railroads to cars, airplanes, food, drugs. There's no reason why we shouldn't be able to regulate tech. In fact, tech is less complicated in many ways than finance. and we regulated finance. Furthermore, the United States, for instance, had to face Rockefeller on their own. France, the UK, they didn't care about Rockefeller.

But now there's so many countries that want to regulate big tech, so yes, they're very powerful. But they're not more powerful, first, than all the collective users on which they depend, because they depend on our data. And if we rebel and don't give them our data or obfuscate, they're in trouble. And second, they're not more powerful

than the collection of the UK, the US, Canada, South America, Australia, New Zealand, Europe, Japan, we can gang up. And don't you think part of the problem is as well is that people aren't getting angry about this because a lot of the time they don't really understand it. They don't understand that it's happening to them first of all. And also they don't understand the long-term implications of this data harvest. Yeah, that is true. And that's a challenge we have.

on the upside, more and more people, I mean, upside, downside, more and more people are having bad experiences online. So in a survey I did with a colleague, Sheandbrook, we found out that about 92% of people have had some kind of bad experience related to privacy online. Sometimes it's about getting your credit card number stolen, sometimes it's about being exposed on Twitter, sometimes it's about, you know, having an ex-boyfriend track you or something like that. And the more we have bad experiences, the more we learn that actually privacy was important after all. And the more we get angry that these companies are making us vulnerable. But you're right that we need a lot more consciousness. And in particular, we need to be much more aware of the political implications of privacy.

This is not an individual thing. Or it is an individual thing, but it's much more than that. Do you think, like, for instance, when you look at something like China, that is a warning for us all, particularly in this country? It's such a huge warning. And it's such an interesting case for so many reasons. But one reason is that, you know, they claim that they are more ethical than we are because we're doing the same thing. We're scoring people, but we're just not telling them. And there's actually some truth to that. Of course, the flip side is that we have a lot more freedom. And even if you're being scored, say, as a consumer, that's not going to impact whether you get, I don't know, a place at a university or a job. So we're much more compartmentalized. And that's

part of what makes a liberal society liberal instead of being a totalitarian. regime. We have to make sure that we don't walk into that system. We have to walk away from it. And I am very worried that we're walking towards it. But now a really interesting lesson from China is coming through right now as we speak. And that's that one argument from the West for not regulating data was that, you know, we have to have as much data as possible because look at China, they're collecting so much data. And if we don't collect enough data, we will be at a disadvantage. And now it turns out that China is passing one of the strictest privacy laws in the world right now.

And that's very interesting. There's a lot of speculation about why exactly are they doing it? They're hurting their stock. And so why exactly are they doing it? What reason I think, it may not be the only reason, but one really important reason is that they are realizing how dangerous it is to have so much personal data stored because it's a national security danger and the West is going to hack it sooner or later and make a use of it.

So even if even China is regulating privacy, we really have to get our, act together quickly. And how close do you think we are to that kind of dystopia, Carissa, where, you know, you go and get a loan. And for reasons, unbeknownst to yourself, you fail it. Your credit rating is excellent. You know, you've never been in debt. You've never defaulted on a payment.

Yet because you don't know that you have a, you know, a genetic condition, but a company does, you're not going to get the money. It's really hard to say. part of me thinks that that probably is already happening, but it's just very hard to tell because everything's on underground and we can't see it. So it wouldn't surprise me that those things are already happening, but they just haven't come out to the light.

And then depending on, you know, it's hard to predict because if we get, you know, one prime minister or another in a few years, it can really make a difference. And if the U.S. gets a different president, things can change so quickly for better and for worse. as we've seen in recent days. Chris, moving on a little bit, I know that you are an associate professor

at the Institute for Ethics and AI. Do you mind if we talk a little bit about AI more broadly? Yeah, sure. Because I remember I was listening to Elon Musk a few weeks ago talking about, he was asked a bunch of broad questions, but one of them was like, what terrifies you the most about the future? And he was saying it's AI.

And I remember growing up as a kid, I would read all the sci-fi, the Asimov and the other stories, a lot of which were really exploring the implications of having artificial intelligence. And again, from a point of view, not of some evil plan to control the world or whatever, but actually from a misplaced desire to make things better, where people would give AI particular targets like, let's make human beings happier. And before you know it, everyone is suddenly hooked up. to like a heroin drip because that's the way you make people happy. Do you know what I mean?

Like, should we be worried about the increasing influence of artificial intelligence in our lives? Yeah, we should definitely be worried. So research in AI, in the ethics of AI, I mean, one way to categorize it, there are many ways, but one way is it broadly devise into two. The people who are really worried about what is often called superintelligence, so the point at which AI becomes more intelligent than human beings, and then, you know, what are they going to do?

Are they going to get us hooked in heroin or, you know, are they going to take over the world and so on? And the people who are worried about more short-term problems, like, you know, the bias that algorithms are instituting and so on. I think that those concerns about AI taking over the world and what are they going to do are legitimate and we should be thinking about it. But my own take is that the short-term risks are much more real, tangible,

and they're just like, they're here, they're here, and they can lead to a dystopia as bad as, you know, the AI that gets us hooked on heroin. So I think it would be as bad to have this complacent attitude towards AI and data and get into a totalitarian regime that we can't resist because we're being surveilled at the time than, you know, the worry that robots might be advanced enough to take over the world. I think we're pretty far from that still. Give us some examples of the smaller, more short-term stuff. that you're concerned about with artificial intelligence? So one concern is, you know, how is AI impacting things like equality? So one thing, you know, data and AI are very closely related because the most successful kind of AI at the moment uses a lot of data

and much of that is personal data. So it's hard to differentiate those two. So one question is, you know, is it okay for us not to be treated as equals anymore, but to be treated on the basis of our data? And what does that say about our society and what implications does that happen? Another very important issue is bias. It's really hard to have an AI that is not biased, and it's really difficult to identify the bias because when we're not aware of it.

And there's a lot of evidence that shows that AI is intensifying inequalities that we're already there. So we've been, as a society, we've been sexist for a long, long time, we're pretty good at it. But AI can make us even more sexist and without us realizing, and the same with racism. Another issue that I'm worried about. Sorry, Curis. How can it do that? How can, are we going to design a robot that is, you know, just goes around, wolf whistling.

So there are many reasons why I can do that. But to give you a couple of examples. So a few years ago, Amazon used an algorithm to, they tested an algorithm to try to hire people and to filter candidates. Because, you know, these big companies get thousands and thousands of applications. And they realized that the algorithm was being second. And the reason it was being sexist is because in the past, say, in the past 10 years, Amazon has tended to favor men. And so the kind of ideal candidate for the algorithm is, you know, the white guy.

And if, you know, a CV had things like somebody played in the women's soccer team in high school, say, as an example, the algorithm goes through the successful candidates that have applied to Amazon in the past. And nobody has been in a women's soccer game. and so it discriminates against that person for no good reason. And we have to be really vigilant to pick those up. But another example is there's a lot of sexism in medicine. And one of the reasons is that most of the data we have comes from men. So the kind of the paradigm of medicine is a white male.

And it turns out that women in many cases are very different for different things. So for instance, we are neural pathways for processing people. pain are different. So it turns out that painkillers are much more effective for men. And so it's very easy to be sexist without the intention of being sexist. It's not like, you know, we're trying to design a sexist algorithm. I think we wouldn't be able to do it as well if we're trying it. It's more that sexism is really baked into how we have seen and experienced the world, how we have hired and treated people. And because algorithms work on historical data, they tend to reproduce that reality. That's so interesting.

That really is very interesting. So it's actually reproducing inequality that may not actually exist in our minds today. I think very few people would choose deliberately to discriminate against women now in employment. But the algorithm is actually more sexist than human beings potentially because it's replicating data from 30 years ago. Wow, that really is interesting, isn't it? And it's fascinating because it shows us that it can actually start. progress, social progress and political progress. It's kind of regress in many ways. And one of the important functions of forgetting is to progress, because when you forget, you kind of let go of the

past and you're able to see the future with fresh eyes. And, you know, in the past, we used to forget all the time because our memories are not perfect and because recording was very effortful and very expensive. And because even when we recorded, say, you put something on paper, when the paper didn't have acid, it just fell apart after a few years or it got, or there was a fire or a flood or something. But now the whole economy of remembering and forgetting has gone upside down. And now we're remembering everything and just by default collecting all the data we can and storing it indefinitely. And that's really unwise. And one reason it's unwise is because it creates societies that are very harsh when they never forget, but also because it kind of stops progress.

And what about the other argument? Sorry, Francis, just to finish on this very point. You know, we talk about equality, but you might say from a kind of ruthless Russian mentality like mine, you could argue, couldn't you, that, well, look, these algorithms actually, in many cases, I take the point about the disparity when it comes to sex, but in many cases they're making better decisions in order. So, for example, if you are about to get a mortgage and you do actually have a condition, which means you're highly likely to die in the next five years, shouldn't the less. and know that before they give you a loan? Isn't this just a way of making better decisions and being fairer? Because we never really had equality. Like the fact that if you don't pay back your loans,

you shouldn't be treated as equal by your bank to someone who does. Do you see what I'm saying? And the algorithm is just amplifying the ability to make accurate decisions. Some people might argue. Yeah, so I'm reading an interesting book at the moment called The Tyranny of Merit about the kinds of effects that it has in society to think that, you know, people can just are self-made and we should judge them accordingly. And so I recommend that.

But so the first thing to say is unless we have randomized control trials that show that these loans are actually, these algorithms are making better decisions and getting better returns, then, you know, we shouldn't believe it. We shouldn't believe it, you know, because somebody says that we need randomized control trials. So that's the first thing. But secondly, let's say that they do work like that. and let's say that they are more effective. We also have to ask ourselves, what kind of society do we want to live in? Do we want to live in a society in which because somebody gets cancer, they can't get a loan? Shouldn't we have rules for that?

Just not be a super harsh society in which it's really hard to thrive and becomes kind of unlivable. Or maybe, you know, the government then should step in and give those loans to people who might have more trouble. Or I don't know, we have to do something. But we just can't leave it to the market and think that we're going to end up with this, wonderful and fair society, because we're not. And moving forward, if AI continues the way it is doing and the way it's been predicted to do, that's going to have a huge effect on the labor market, isn't it? There's going to be significant swaves of jobs that are just going to be non-existent.

Yeah, there's a huge controversy about it. Some people think that we're going to lose jobs and we're never going to recover them. Other people think that, no, no, no, in the past, this has happened before. And what happens is that people just change their jobs. So when technology gets developed, you know, in farming, you used to take care of the cows and the horses, and now you take care of the tractors. But some people think that because AI is a different kind of technology in the sense that we're trying for it to be autonomous, for it to not need input, then we are going to lose a lot of jobs. Then there are people who think that actually the best kind of AI works in tandem with a human being. AI is really stupid on its own.

If you've had a conversation with Syria or Alexa, you will have noticed that. So many people think that a more realistic future is one in which we design AI to work with human beings. So, yeah, that's a huge controversy. And I don't think, you know, I'm kind of neutral about it. I'm not confident enough to say, yeah, we'll go one way or the other. One of the fascinating things that I remember reading about recently is the idea that in order to program, AI to make decisions, for example, driverless cars, you're going to have to start to make philosophical decisions about right and wrong because let's say a car that is not driven by a person,

but is driven by AI, has to make a decision, do you crash into that car or do you crash into this car because you're in the trap? Do you kill three people there or do you kill one person there? Like, how are we as humanity going to resolve some of these moral dilemmas that pop up? That's very difficult. one hand, you're right that we have to make sure that we understand that whenever we make a tool, it has values within it. Technology is not neutral. And we need to make sure that the right values are in place so that we end up with the kind of society that we want. But it turns out that

ethics arguably is something that you cannot code into a computer. So we're not, a normal human being doesn't act morally in a way that can be coded easily into a program. So there are many questions about how do we deal with this. One way to deal with it is to try to teach AI as if we were teaching a small child. So just like we teach AI to recognize images by giving them, say, you know, millions of images of what is a dog and what is a horse or whatever it is, then we should just give them millions of cases of like what's the right thing to do and what's the wrong thing to do. Now, other people say, well, but actually people are really bad at being moral. I mean, we see examples of immorality every day.

all the time. So maybe we, you know, we should be more ambitious and actually have an AI that is perfect, not just like human level, which is kind of pathetic, but that is actually virtuous. But then, you know, what kind of morality do we do we accept? So just to give an example and something I've been thinking about lately, utilitarianism is a very attractive moral philosophy for many people. The main idea of utilitarianism is that you should maximize good consequences. You should maximize utility. How we cash out what utility depends, you know, can vary. But let's say we should maximize well-being, something like that. And that sounds great. And if you ever meet a human utilitarian, they're very imperfect, right? Because they think, you know, they think they should do one

thing, but it's very hard to do. So, for instance, they think that in order to maximize well-being, they should donate most of their money to charity, to the poor, because that's going to save lives and that's going to be better for the world. But when they have a family and they have a kid, it's really hard to do that, and they tend to prioritize their kid and, you know, pay for college. And the way they explain that is, I'm imperfect. You know, I know I shouldn't do that, but I have these psychological constraints as a human being, and I can't help myself, and I'm sorry about it. And, you know, for them, that's a bad thing.

For most other people, it's kind of a relief that they can't be a true utilitarian because a true utilitarian would always be strategic, right? So when they are your friend, they would only do what maximizes well-being in the world. So they wouldn't favor you as a friend. And so I'm really relieved that I don't have a friend who's a perfect utilitarian. But with AI, we could turn it into a perfect utilitarian. They don't have those psychological constraints of loving someone and being partial to someone. They could be a perfect utitarian.

And that's kind of a frightening thought. Well, you make it more dystopian if you want, because what if the truth is, let's say, that we know statistically most crimes in society are committed by a small number of people, many of them have psychopathic disorders. Now, what if you were perfect AI utilitarian, you wanted to maximize well-being by, killing all those people, right? And then, you know, you kill off 2% of the population, everybody else is really happy.

We get rid of Boris Johnson. Yeah, not to mention that, you know, the easiest way to get rid of suffering is just to explode the world, right? So, yeah, we have to make really, really sure that we're sensible in how we program AI. But I think at the moment, we should be focusing more on the short-term challenges

on the long-term ones because they're so urgent and it's not going well. No, because it's also as well, it opens a whole Pandora's box when it comes to the law. You know, for instance, if a driverless car malfunctions, swerves into the road, kills six people, or onto the sidewalk or pavement, I should say. You know, who is ultimately responsible for that? Because it's not a human being, you know, it's very simple. If a human being does that, you go, well, the human being is in charge of the technology.

But who is in charge of that technology? Is it the person programming the algorithm? Is it the company? Was there a malfunction? Was it the factory that produced it? Doesn't this open a whole minefield legally? It does. It creates a lot of what's called responsibility gaps

in which something goes terribly wrong and everybody can say it wasn't me. And then we don't have incentives for people to be careful with these things because nobody pays the price. And, you know, that example with the cars are a very good one, but there are so many others. So, for instance, a few years ago,

the Michigan Unemployment Agency used an algorithm to detect fraud and it accused thousands of people, about 34,000 people of fraud falsely. There were false accusations. So these were people who were very, in a very precarious situation already. They take their checks away and these are people who lost their families, they lost their homes, some of them committed suicide. Two years later, they realized that the algorithm got it wrong 93% of the time. And who goes to jail? No one. Because nobody knows who's responsible. Was it that, you know, they set the objectives wrong?

was it that they hired their wrong programmer, which programmer, because a lot of people work. These algorithms sometimes have millions of lines of code. And so one of the things we need to do is, before an AI project starts, we have to design a chain of responsibility and say exactly who's responsible for what, before anything goes wrong.

So that when things go wrong, we can actually know who to turn to. Carissa, are you optimistic about the future? I'm cautiously optimistic. I think it can go both ways. I think we're in a 50-50 kind of place. But I think we can turn things around. And I think we have done in the past.

So, you know, one example I gave in the book is how we managed to recover the ozone layer. We were really in danger and about to lose it. And we were using CFCs all over the place. And we learned what we were doing. We regulated. And now the ozone layer is recovering. And it's going to completely recover in a few years. So I think, you know, I think it's possible.

we can do it. My only hope is that we do it in time before something really bad happens. I think eventually we're going to get it right. The question is, are we going to get it right now before something really bad happens? Or are we going to wait for something like, you know, the Nazis using personal data for genocide in the Second World War in the West before we get our act together? So just before we finish, so we're talking, and I'm miserable and a pessimist, so that's my position. So you talk about something really bad happening, you've given the example of the Nazis, you know, and a lot of people will then say, well, look, you know, that's, you know, a once in a however many generation thing. Could you give

other examples of what this type of misuse of AI and algorithms could actually lead to? Sure. So one example of personal data being misused was in the Rwanda genocide in 1994 as well. personal data was misused in the Second World War with Japanese people in the United States but also more recently with algorithms the best example is probably China using it against the Uyghurs and this is a minority in China they are usually Muslims

and China has developed algorithms to detect facial features that are racial and they have put these people in camps and they are essentially persecuted and there's no reason why that couldn't happen in the West. Another example is Hong Kong. Hong Kong was a very high-tech society

that was really in favour of democracy. And suddenly they realized that a lot of the tech that they had developed was being used against them when China changed their policies and wanted to gain some control back. So we had these really impressive images of people trying to take down cameras with facial recognition in the street or people queuing for blocks and blocks to buy their subway ticket with cash instead of all the machines that were much more prevalent.

So that's one example in which, you know, once things go wrong, it's really hard to go back. So we have to make sure that we have a system in place as a buffer in case things go wrong. And I was going to say as well, in terms of personal things that we can do, I understand the regulatory stuff that you're talking about. As individuals, what can we do to protect ourselves? There's a lot we can do because these companies depend on us on our data. So choose privacy-friendly apps and devices. Don't ever buy a device from a company like Google that earns their keep through collecting personal data.

Instead of using WhatsApp, use Signal, instead of using Google search, use Dock. Instead of using Gmail, use something like Proton Mail. Instead of using Dropbox, use Jota Cloud. There are many alternatives out there. protect other people's privacy, contact your political representatives and tell them that you care about this. And in general, let's kind of create a culture of privacy and not a culture of exposure. And don't use that facial recognition software. Isn't that one that you should absolutely avoid with the apps? Yeah, if you can definitely avoid biometrics whenever possible.

There we go. Carissa, thank you so much. Your book is called Privacy's Power. Thanks for coming on. Where can people find your other work online if they want to follow you after this interview? I rant on Twitter a lot at Carissa Belize, and they can find me on my website. It's just my name.com. Fantastic. Thank you so much.

We'll ask a couple of questions for our local supporters, but for now, thank you for coming on and thank you guys for watching at home. We'll see you very soon with another brilliant interview like this one or Orr Show. All of them go out at 7pm UK time, which is 2 p.m. Eastern. Take care and see you soon, guys. We hope you've enjoyed this incredible interview. Remember to subscribe and hit the bell. button so that you never miss another fantastic episode. And if you believe that the work we do

here at trigonometry is important, support us by joining our locals community using the link below.