Technology, Connected - AI Is Becoming A Franz Kafka Novel

Episode Date: November 13, 2025

Carissa Veliz joins us again to speak about AI Ethics, a mirage straight from a Kafka novel. Questions of justice, principles and the rule of law are incompatible with machine learning. Machine learning is statistical analysis of data that outputs responses human beings are likely to find attractive, not true or ethical. That is not a good way to design ethics.Carissa Veliz joins Makr & Jeremy to Think On Paper. She outlines how AI depends on surveillance and statistical pattern-matching that can’t meet the basic standards of a democracy: clear rules and the ability to appeal a decision.AI thinking clashes with the foundations of a liberal democracy: public rules, transparency, and the right to challenge decisions that shape your life.We cover:😀 Why machine-learning decisions are opaque😀 Why that conflicts with the rule of law😀 How surveillance sits beneath modern AI systemsPlease enjoy the show.Cheers, Mark & JeremyOther ways to connect with us:⁠Listen to every podcast⁠Follow us on ⁠Instagram⁠Follow us on ⁠X⁠Follow Mark on ⁠LinkedIn⁠Follow Jeremy on ⁠LinkedIn⁠Read our ⁠Substack⁠Email: [e-mail]

Questions of justice and principles and the rule of law are incompatible with machine learning. So machine learning is a statistical analysis of data and it outputs likely responses. Or actually, responses that human beings are likely to find attractive. Not true, not ethical, attractive for human beings. And that's not a good way to design ethics. It's just not. So I'll give you an example because this is sounding a bit of, abstract, okay? You go to a bank, you ask for a loan. If we use AI, machine learning, then you get a

response, yes or no. Why did the algorithm say yes or no? We don't know. Does the data scientist no? No. Does the banker know? No. It analyzed data. Your data, it compared it to a lot of other data of people who have paid their don'ts or not, and then came up with our response. And sometimes when you look into it, it turns out that it was tracking your race, that it was tracking your name, that it was tracking things that it shouldn't track. Now, how do we make an ethical decision based on our values? We put transparent criteria. We say you have to have X amount of dollars in your bank account.

And if you don't, then you know how to fix it. But that's not machine learning. And it's not the kind of AI that we're using now. And so how do we align AI with our values? We don't. We don't. We either program it, program our values. We set a set of transparent criteria that people can know and can appeal.

And then we just make it obey it. But we don't use it to come up with those rules. And again, in a rule of law, in a liberal democracy, people need to know what the rules are. Rules need to be public. Laws are public. Right now, we are ruled by rules. that we don't know and we don't understand and we can't appeal. And that is the definition of Kafkaesque.

Codifying kind of human values to integrate into these models. What would be three of those values be to show a machine what the important things about being human are and what the important things for that machine to recognize about humanity to make it safer for us. I don't think it works out like that. You know, there aren't three things that you can codify easily.

But a good start would be human rights, you know, the list of universe of human rights. And one of those is privacy. And just with that, I mean, it would mean a huge difference, right? Because at the moment, all of this technology is based on surveillance. That's kind of mind-blowing.

We've become used to it. But if you told someone in the 1950s, we're going to have this system. It's going to rely on surveillance the way we think it's okay to surveil a criminal, but not anyone else in the 1950s, people would be shocked for good reason. And that's why I say, like,

Big Tech is so successful at the narratives, but it's how they package the product. If you package it like one of those electronic, devices that people on parole have to use, then it looks a lot less attractive. But it really is quite similar to that. And not only that, it's ranking you all the time. Even if we don't have an official social credit score,

that is essentially what we're doing, even though it's not as centralized as it would be in a country like China. All right. So what did we just watch right there? We watch Thinking on Paper, bite-sized, a shot of technological tequila to your prefrontal cortex. It's just a taste at a smorgasbord of what awaits you with the full thinking on paper interviews. There really is nothing to like it out there at the moment connecting the dots of all these technologies. So subscribe where you're listening.

Check the long former interviews out. And remember, stay curious. Be disruptive. Keep thinking on paper.