Your Undivided Attention - Flock is Just the Beginning: Inside the Era of AI-Powered Policing

Episode Date: September 10, 2026

In a 2018 TED Talk, Yuval Harari argued that democracy prevailed over fascism and communism in the 20th century not because of any inherent advantage, but because the technology of that age favored open and democratic societies. It was simply too difficult and too costly to centralize power using pre-digital technology.But, Yuval warned, with the rise of AI, the calculus might flip: it might actually become far more efficient to run highly centralized authoritarian governments.Eight years later, we're starting to see what Yuval was warning about. Right now, police across all levels of government, from small-town sheriffs to massive federal agencies, are adopting AI at a staggering rate and using it to build the kind of efficient, centralized surveillance-state apparatus that Yuval warned about.Take, for example, the recent deployment of hundreds of thousands of automated license plate readers (ALPRs) across the country by companies like Flock and Axon. These cameras have made headlines in recent weeks, but as you’ll hear in this episode, they are just the most visible frontier of an AI-powered digital dragnet that gives police the ability to analyze extraordinary amounts of data about private citizens.Today on the show, Tristan speaks with two experts on how police are using AI technology today, what we can expect in the near future, and what protections we need to create now before it's too late. Andrew Guthrie Ferguson is a law professor at George Washington University who studies how policing technology collides with constitutional law. He's the author of Your Data Will Be Used Against You: Policing in the Age of Self-Surveillance.Garance Burke is an investigative journalist at The Associated Press, where she leads high-impact projects on the intersection of technology and society. Her investigations into the global surveillance industry were part of a package that won the 2026 Pulitzer Prize for International Reporting. RECOMMENDED MEDIA Your Data Will Be Used Against You by Andrew Guthrie FergusonGarance Burke’s AP homepageYuval Noah Harari’s 2018 TED talk: “Why fascism is so tempting -- and how your data could power it”Garance Burke and Byron Tau’s reporting on the use of AI-enabled tools by DHS in Minneapolis The AP’s reporting on how Silicon Valley helped build China’s surveillance stateFurther reading on Estonia’s data tracker tool RECOMMENDED YUA EPISODESWhy Are Migrants Becoming AI Test Subjects? With Petra MolnarAmerica and China Are Racing to Different AI FuturesLaughing at Power: A Troublemaker’s Guide to Changing TechCorrectionsAndrew said that real-time crime centers are in 300 cities across the US, which matches the findings of the National Real Time Crime Center Association; however, this figure is disputed, with the Electronic Frontier Foundation putting it at 150.Garance cited her reporting that there are nearly two hundred AI systems currently in use by law enforcement. According to recent public accounting, that number has grown considerably and is now at 238.Andrew incorrectly described the resolution of the case of a man misidentified by AI in New Orleans. He said the man was acquitted, but in fact the charges were dropped. ClarificationsAs noted in the episode, Andrew’s discussion of agentic AI was a theoretical account of a tool that police may use in the future, not something actively used on the ground today.Garance referred to the protests in Ferguson as the impetus for police-worn body cameras. Body cameras were being piloted in the US and UK prior to Ferguson, but federal grants in the wake of the protests made them much more common.     Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Hey, everyone, it's Tristan. And we're gearing up to do a new Ask Us Anything episode for our podcast. The stakes of the AI race have never been higher. The danger is increasing. We know you have a lot of questions. We'd love to hear them. Please send us your questions at undivided at humanetech.com. That's undivided at humane tech.com.

Thanks. I spoke with a man named Luis Martinez, who was on his way to work. It was a cold Minneapolis morning. And all of a sudden, massed federal agents boxed in the SUV that he was driving. and he had to come to a dead stop in the middle of the street. Now this mass agent came up, told him to pull down his window, and then scanned his face using his phone. Lees didn't know what was going on,

and the agent kept asking, are you a U.S. citizen? Now, the system that the agent was using on his cell phone to scan Lois' face, picked out his irises, the shape of his face, the curve of his lips, but it did not correctly identify that he was a U.S. citizen, The only reason Luis was not taken to a detention center was because he happened to have his U.S. passport in his glove compartment and pulled it out and was able to prove his citizenship. This app is called Mobile Fortify. It's being used right now by ICE. And what we found is that according to a lawsuit that was filed against DHS by the state of Illinois and the city of Chicago in January is that this mobile fortify app had already been used in the field more than that. than 100,000 times in January. Hey, everyone, this is Tristan Harris, and welcome to your undivided attention.

So the conversation you're about to hear, I recorded while I was in Berlin, just a few blocks away from the Stasi Museum, which was dedicated to the history of East Germany's surveillance state. And what struck me about the kind of surveillance that's on display is just how laborious it all was. Every file had to be recorded. It had to be examined by a person. every tape was listened to and transcribed by hand.

Every informant was someone who had to be recruited, paid, and managed. It was expensive and time-consuming work. And it reminded me of an argument that my friend Yuval Harari made in a TED Talk in 2018, that democracy prevailed over fascism and communism in the 20th century, not because of any inherent advantage, but because the technology of that age favored open democratic societies. It was simply too difficult, too costly, to centralize power with pre-digital technology. But you've all warned that with the rise of AI, that calculus might flip,

that it might actually become far more efficient to run highly centralized authoritarian governments. The greatest danger that now faces liberal democracy is that the revolution in information technology will make dictatorships more efficient. than democracies. Now, eight years later, I worry that we're starting to see exactly what you've all was warning about. Right now, police across all levels of government, from small town sheriffs to massive federal agencies, are adopting AI technology at a staggering rate. And they're using it to build out the kind of efficient centralized surveillance state apparatus that you've all warned about.

So take, for example, you know, the recent deployment of hundreds of thousands of automated license plate readers or ALPRs, across the country from companies like flock or axon. And on their face, these cameras are just there to help police track cars by license plates. But as you hear in this conversation, they actually are just the most visible frontier of an AI-powered digital dragnet that gives police the ability to analyze extraordinary amounts of data about private citizens. So today on the show, we've invited two experts on how police are using AI technology today and what we can expect in the near future and what protections we need to create. now before it's too late.

Andrew Ferguson is a law professor at George Washington University who studies how policing technology collides with constitutional law. He's the author of Your Data Will Be Used Against You, Policing in the Age of Self Surveillance. And Garantzberg is an investigative journalist at the Associated Press, where she leads high-impact projects on the intersection of technology and society. And her investigations about the global surveillance industry were part of a package that won the 2006 Pulitzer Prize for International. National Reporting. Andrew and Garantz, welcome to your undivided attention.

Thank you. Thanks so much for having me. So before we dive in and just want to ask you both just to share a little bit about your background, how did you come to this topic? And Andrew, we'll start with you. Sure. I used to be a public defender in Washington, D.C., trying cases in front of judges and juries. And a lot of times in my trial work, it was determined. that my client was in a quote-unquote high-crime area. That's a term of art in the Fourth Amendment. It involves data, involves crime, involves numbers. And I thought to myself, someone should probably be keeping track of this information. And so in my cases, I would go subpoena the crime analyst to

figure out whether this was a quote-unquote high-crime area. I would cross-examine police officers. I'd convince judges about the role of data in policing. And no one else was doing this at the time. And so when I transitioned into academia, I started studying the impact. of data and new technologies on policing. I happened to arrive in academia at the rise of predictive policing, so I followed that, saw how it transitioned into facial recognition and the Internet of Things and smart surveillance. And now we're in their AI age.

As the technology has changed, the law has pretty much stayed almost the same. And so my academic career has been sort of studying that puzzle between a new technology and an old law and all of the interesting issues that arise from it. And Garan, how about you? Well, so I actually began my work as a journalist living in Mexico and then came back to the Bay Area where I did a whole course of study in statistics and economics and had always really had a data-driven bent to my investigative reporting.

But then in 2018, I began noticing there were just so many billboards up around San Francisco talking about AI, the promise that it would bring to our daily lives. And I was able to take a year out to study at Stanford where I took a lot of programming classes and really delved into AI and its intersections with civil rights, human rights, and many, many different areas of our lives. And so since that time, I've been leading investigations on that very intersection at the associated press. Well, and you're referencing something, which is just the felt experience of driving through San Francisco and seeing all the billboards of the promise of AI.

Well, I hope that, you know, what we do here is we both exposed for listeners, just where is this really going? What are all these technologies? How are they being applied? You know, where is the dark side of all this? And then hopefully have a vision as well of what would a good version of this look like that is preserving the values of open societies? So I want to start our conversation first by taking people to the frontier of the technology.

And people may not know this, but every year there are these massive expos where police departments and federal agencies, they go and they get the latest tech. And I want to just kind of put our listeners at that frontier. You know, if you were to go to one of these expos in 2026, you know, what kinds of technologies would you expect to see and what would surprise people the most? And Garantz, I'll start with you. Well, I think it's interesting for folks to understand just how pervasive these technologies are these days. if you were to go to something like the border security summit that's coming up in San Diego next month,

you would see lots of vendors with little booths advertising all kinds of surveillance technology to be used on the U.S.-Mexico border. That includes surveillance towers that can be driven around on trucks, ground sensors to detect people and vehicles, biometric technologies that capture the shape of people's irises or quadrants of their cheeks. I think it's just so important for people to understand that these technologies are very much being deployed in our communities today.

And this is increasingly spread far from the U.S.-Mexico border and they're embedded in a lot of our cities and towns and used daily. And, Andrew, how about you? some of the frontier technologies that you see police investing in today that maybe people haven't heard as much about? I think the best way to visualize where we are on the frontier right now is to go inside a real-time crime center.

So if you can picture a centralized police building filled with screens and computers, camera feeds coming in, and you see the convergence of three forms of AI power. The first is sensor data. So automated license plate readers, ALPRs, gunshot detection. Second, you'll see camera feeds. So camera feeds from CCTV cameras,

camera feeds from police body cameras, from drones, and the rest. And you'll see that these feeds are now going to be captured and analyzed with AI analytics. So every object within those video feeds, every sensor, every historical fact, can now be recorded, separated out,

and analyze in ways that we've never seen before. So you can find a particular car in a particular place. You can find an object across a city. You can track a backpack across a city just by object recognition technologies. And then you have like the physical AI of robotics. Like drones are robots.

You literally have some not terribly successful robots patrolling the streets. And you see these three different types of AI converging all in a centralized real-time crime center, which are now in over 300 cities in America, big cities and small, where police have a new tactical advantage, but also have a new surveillance advantage

to watch everyone everywhere all the time. I would just add to that as well, one of the ways that I keep track of all of the AI tools that the Department of Homeland Security is deploying, sometimes in these real-time crime centers, sometimes along the U.S.-Mexico border, is I look at what the agency calls its AI use, case inventory. Now, this is a public accounting of the AI tools that you can look at yourself,

and there's nearly 200 AI systems that are used currently, including some in law enforcement activities, and that includes nearly 50 AI tools that the agency itself said were high risk that are currently being piloted or in development or already out on our streets. The agency calls those tools high risk if they're key in making decisions, or, or if they have a legal effect on rights and safety. Well, one of the things as you were speaking about that first, Andrew, was that I realized I have no idea. I never heard of this called a real-time crime center.

Yeah, they're in 300 cities across America. It's basically a centralized command center where analysts and police officers can watch the city using video feeds. And if you watch what happens, there can be a call where 911 call says, man with a gun at a corner. And police can click on the can. that is nearest to the call and see the scene. They can fly a drone with a camera running to be able to get a tactical advantage before

police arrive. And then they can click back and forth between those cameras and the cameras on the police officer's chest, the police worn body camera and the car camera, to see in real time what is happening and have a two-way communication where they're feeding information about what the officer might see before they get there, which again, from a police department perspective, is incredibly helpful to be able to have. that kind of tactical insight, not just about what you're seeing there, but also what's happened in the last weeks or months because they have historical doubt. They might know this is a location

of a person who has like mental health issues. And so maybe they don't need to send the SWAT team in because they've already had three calls at the same time. All that data is now collected and can be analyzed in different ways and different forms. Well, so someone hearing this might say, well, that sounds great. We can just better catch the bad guys. We can better distinguish. distinguish that person who's been, who's just mentally ill, who's not actually a threat from someone who's actually a criminal. What's the real problem here? How would you articulate what the problem here is, Carranz? Well, I think one of the things that we've found over time is that

the data that often feeds these systems contains some of our own human biases. If you're looking, for example, at historical arrest records, you would see more people of color in that data. than you would proportionally to white members of the community. And then if that data then powers the AI system and is used to train the model to figure out where these high crime areas are, it will likely be sending officers right back to the same street corners, in essence, you know, hardening that bias into the system. I think also when you're talking about object recognition, you know,

say the model is trained to look for a gun. Well, there are a lot of things that could look like a gun that are bulky in somebody's pocket, right? And so what we've seen, for example, when some of those object recognition models have been deployed inside schools, you'll have alerts that go off. So those are a couple things I would point out. I'm sure you have other examples as well, Andrew. Yeah, I mean, there's a case where there was a video analytics system running in New Orleans and around the different parishes.

And it's such that they could do what they called a virtual patrol. They could go from camera to camera to see what might be suspicious on the streets. And just pause there. That means they could watch all the streets all the time. And they saw a guy who they thought had a bulge. Turned out it wasn't a bulge. But they used that pretext to actually have actual officers, stop him, chase him, arrest him.

And he was eventually found not to be guilty of what they accused him of. But the reason he was tackled in tased was because of this. sort of virtual patrol. But to your question of, isn't this a good thing? I think, you know, the point of this discussion is that there is a duality to this. Like, we are giving police incredible new powers, and they can use it for good in the sense of there's a carjack and they want to find and get people on the street there. But that same surveillance technology can watch the abortion clinic, the mental health clinic, the protest against police brutality. Because in order to work, in order to have a real-time crime center

that watches the streets with censors and video, you have to be watching the streets all the time. And that kind of flips the default. It means that everyone is being surveilled. And 99.9% of those people are absolutely innocent. They're just going around their business. But they're now also being watched by police without set rules, without regulations,

without even a whole lot of transparency. And thus, we've kind of reversed the default. We are in that world where you can be watched all the time and the police can choose when and to whom they want to sort of target their surveillance. And the technology exists to do that now. Yeah, that was what was coming up when Grants was speaking earlier, of just increasing technologies are being used on us. The 200 technologies you mentioned, the 50 that are high risk,

most people have no idea about this. So there's not a symmetry of understanding. Those in law enforcement have way more information, and federal agencies are way more information about us. We don't have information about what they know and how they know what the increasing technologies that they're using. How do we just continue to sort of map out an ontology of these different risks that we're trying to address here?

And then as we get to the end of the episode, we talk about solutions. I want to make sure that we're dealing with each one. I would just add one more to your list of, Glenn, it is this idea of what changes when everyone is being surveilled and watched all the time. How does that chill democratic action, association, where you go and who you go with and what you're doing?

There's an interesting story on social media where a gentleman who owned a gun range had to apologize to his customer because he had bought one of these ALPRs because he thought it was good for safety. Not realizing that, it meant that all of the people going to the gun range were now under surveillance. Hey, Tristan here. Just a reminder that ALPR stands for automated license plate reader. These are the AI-enabled cameras from flock and others that have popped up all around the country in recent months. as we'll discuss, they do a lot more than just read license plates. But again, if you think about putting it together, you don't need a database of all the people who own guns if you have an automated license rate reader outside the gun range.

You know who owns a gun. And you might even actually find someone who's involved in criminal activity if someone who owns that car isn't supposed to own a gun and is at the gun range. You might have just created the evidence against your clientele simply by purchasing a technology. And again, it just reverses that default of revealing who we are and what we're doing

without rule sets about whether it can only be used in the cases that maybe we could agree it should be used for. And I would add to that as well. I think in a democratic society, these tools can be used in ways that really would surprise people. They can be trained to optimize for things that, you know, we might not find palatable in this country, for example, in China. Some of the tools used in Xinjiang predict if a person had facial hair or a Muslim-sounding name, then they would be more likely to be deemed to be a terrorist.

And also, I'd say another risk is sometimes the tools just simply don't work as well in one situation as they may in another. So if we're thinking about AI-powered transcription software, which many of us use in our daily, lives, it works much better in English than in other languages. But yet if that same transcription software is used to translate important surveillance intercepts, say, for example, audio in Gaza, and it mistranslate something in Arabic, well, then that ends up being a big problem if that software is part of the data that's feeding a military targeting system, right? So I think thinking about, you know, what these systems are optimizing for, or or predicting on is another way in which the public can think about the kinds of risk that may be present,

depending on where they're deployed. Grants, is there a story from your reporting that just illustrates how this tech is being used by police, something that illustrates more of the human stakes of what we're talking about? So one thing that my colleague and I, Byron Tau, found, was pretty surprising, was back in late January, when the public, I'm sure, remembers the immigration enforcement push in Minneapolis. I spoke with a man named Luis Martinez who was on his way to work. It was a cold Minneapolis morning. And all of a sudden, masked federal agents boxed in the SUV that he was driving.

And he had to come to a dead stop in the middle of the street. Now, this mass agent came up, told him to pull down his window, and then scanned his face using his phone. Least didn't know what was going on. And the agent kept asking, are you a U.S. citizen? Now, the system that the agent was using on his cell phone to scan Lois' face, picked out his irises, the shape of his face, the curve of his lips, but it did not correctly identify that he was a U.S. citizen.

The only reason Luis was not taken to a detention center was because he happened to have his U.S. passport in his glove compartment and pulled it out and was able to prove his citizenship. This app is called Mobile Fortify. It's being used right now by ICE. And I think what that pointed out to us was that even if these tools have already been deployed, it doesn't necessarily mean that they work as advertised, right? It was a cold, dark morning.

Luis is a man of color. It somehow didn't pick up his citizenship. It could be that some of the databases that ICE is pinging to, It might not be shared properly, right? But I think that what we found is that according to a lawsuit that was filed against DHS by the state of Illinois and the city of Chicago in January is that this mobile fortify app had already been used in the field more than 100,000 times in January. I think that encounter really kind of encapsulates some of the concerns that have arisen around the Trump administration's immigration crackdown in Minneapolis, which was described at the time as the largest. of its kind and drew national scrutiny after federal agents shot and killed two U.S. citizens. Now, DHS says that its enforcement efforts are really targeted and focused on serious offenders,

but what we're seeing increasingly is federal agents working in tandem with local officers leaning heavily on this kind of biometric surveillance and these interconnected databases in order to determine people's citizenship, which of course, you know, then can end up having much bigger consequences. And that would have gone totally differently if he didn't have the passport in his glove compartment. Nigarant, are there other examples that we should be aware of? You know, one of the things that we were looking at when trying to understand

the Border Patrol's predictive policing program was how it was used out on the highway. And through going through court records in Texas, we found the case of a father in the San Antonio suburbs named Alex Schott, who was pulled over actually under the Biden administration. This predictive policing program has been present since then. And he basically had his truck ripped apart for no apparent reason. He was pulled over by a local law enforcement officer and wanted to know more about why he was found to be suspicious. So he filed suit. And as a part of that discovery, we ended up understanding more about how the Border Patrol and local law enforcement were coordinating, doing what they call whisper stops that kind of wall off the reason for why this man and others like him are pulled over.

And it was basically Border Patrol and local law enforcement were trading messages on a WhatsApp thread saying, hey, you know, I see this truck traveling southbound at this time, you know, local law enforcement, can you pull the truck over for me, basically concealing Border Patrol's involvement? And I think that what's been so interesting about that particular case is this San Antonio man basically says, you know, hey, I'm nothing to hide. You know, there was no reason for them to pull over my truck. What I object to is this surveillance program that even concealed the fact that the Border Patrol was involved in it in the first place. Now, Border Patrol says that they monitor, you know, driver's patterns. He had been close to the border.

But there were no other indicia, according to what, we've seen in the lawsuit thus far that would have signaled that he should have been pulled over. So I think that case really illustrated to us that as Border Patrol's capacity extends far from the frontier, you know, it can basically be anybody who finds themselves suddenly in this kind of digital dragnet. One of the interesting things about that story is that it shows how the technology can be hidden from the court system. So a lot of times this technology is created for law enforcement for a particular use.

So we want to find a car or a pattern of travel, whatever it is. But they don't want to have that exposed in court, but of course some of these cases do end up in court. So for example, if that truck did have drugs in it or whatever it is, it might actually end up in court. And what was interesting is why the police stopped was largely hidden. But in a court, it might actually get revealed that there was this predictive policing pattern recognition system. that was part of it. And then they'd have to justify about whether it was reliable, whether it made sense, whether it had any basis there. And that can be part of the problem. A lot of times the technologies are built for a particular use case, but not really thought about how it might play out in court. So maybe that system also has exculpatory evidence or impeaching evidence that might undermine the case. And if the system isn't built to find that and surface it for police and prosecutors as it goes to the court system, you may well be. creating a tool that's very helpful for investigation, but will never allow cases to go forward into the court system for a conviction because you haven't built it to withstand the scrutiny of

a judge or the lawyers who would be litigating it. Back in summer of 2024, during the previous administration, we actually interviewed on this podcast, immigration lawyer Petra Mulnar, and she argued that the border between countries was often a laboratory for the most dystopying in the surveillance technology because it had the most gray area in what, you know, what you could do, but that they would eventually be turned inward on all of us. And how does that prediction ring true for you, Garanz? Well, I think, you know, one important thing that's been happening in the last couple of years is some of the enhanced role that the Customs and Border Protection agencies could play

at the border have increasingly been showing up in the interior of the country. So, for example, the Border Patrol was once limited to policing the nation's boundaries, but the surveillance system that the Border Patrol runs that my colleagues and I investigated last year that relies on automated license plate readers is actually been deployed far from the U.S. Mexico border. It can pick up motorists near Gary, Indiana, or Chicago, or Michigan. And so I think that we're also seeing this sort of quiet transformation of some of these agencies that might have been confined to the U.S.-Mexico border or maybe 100 miles from the frontier now

into something that's more like a domestic intelligence operation. Federal authorities can now monitor American cities at a scale that would have been difficult to imagine just a decade or so ago. Agents can identify people on the street through facial recognition, trace their movements through license plate readers, and in some cases, use commercially available location data to reconstruct their daily routines and associations. Yeah, I just want to kind of outline what AI is doing to exacerbate the things we've talked

about so far. Is there anything else you'd add to that set of equation? I think there's going to be, you know, the next stage may well be sort of the agentic AI. It's taking the existing data that we already have and allowing police to use it. The strange thing, it's like we've had a lot of this information before. We've had the data, but it was either siloed in the sense of police couldn't connect the dots or it was too expensive or we just didn't have the capacity to take a police department, which is usually not your most tech savvy of institutions and have them be able to use it. Again, a great just concrete example is we sort of built these platforms for policing using tech

companies like Axon and other companies that are sophisticated data players. Axon and other companies sell AI-assisted police reports. So the audio from a police body camera, right? So as you are speaking to the officer, it's being recorded, that audio can be put through a chat GPT turbo-like LLM and turned into the police report that the officer reviews and signs and hands into court that becomes a document for the prosecution.

And that's a good example of how AI is taking the data we had. We've always had the audio from police body cameras. in something that has been there since we've had police body cameras. But now we can do something new with it. We can do something in terms of generating the police report that then becomes the affidavit, the search warrant affidavit, just because it's the same documentation in a different format, that then becomes how the prosecutor evaluates the case and might in a criminal justice system where 95% of cases are resolved by pleas might be actually the sole source of information

that puts someone in jail for a long period of time. Let's double-click on the idea of agentic policing, because I feel like that's a sneak preview into the future, Andrew, that most people aren't really tracking. And we all heard about AI, you know, 2025, 2026 is going to be the year of agents. We're going to have agents that are doing all of our work. You know, you could say, start a whole business for me, file the IP, create a brand for it by the website, built by the social media ads. And suddenly, you know, a thousand AI agents get to work, right? The legal agent that drafts the LLC for the corporation. the programming agent that writes the code for the website,

the social media agent that does like these things. So people are used to this idea of agents giving us all these magical powers. But what happens when a police force or a single police officer can fire up an army of a thousand AI agents to do this kind of long form what would normally take a month of research and work? What can go wrong with that? Sure. I mean, I think the promise of AI agents is to create the answer machine

for police investigators, right? So if you think about how AI works and the way you use it almost every day, it used to be when you Googled something, you were searching for something. You got a whole bunch of links that gave you a way of finding the information. And now when you Google something, you get the answer, right?

So the same thing happens in policing, right? So if the police have all those databases, we were talking about ALPR data, video databases, people's addresses, 911 calls, gang members, all the social networking connections that Grants was talking. talking about. It used to be that an officer would have to track down each one of those clues and sort of connect the dots. In a world of agentic policing, you ask for the answer. So take a

concrete example. Like you have a carjacking, you have a great car and you have a partial license plate, right? So you've got to figure that out. So in a traditional world, a police officer would go look at all the partial license plate that they could identify, all the great cars, try to link them up with maybe people who might be involved in carjacking and those things. And now in a world of agentic policing, you can either, like, create an agent to do each one of those things, which is like the simple version, or you can just ask for the answers. Like, give me the 10 most likely suspects of people using the database. You may not even know how the agents will go about and figure out which of the data sets

are going to and bringing back your information. And like in today's world with Google, like if you ask Google for a question, you get an answer, like it is smart to go double-check the links to make sure that it actually is actually and didn't just find some Reddit blog and give it back to you, that it's actually the answer. And you would hope that police might do the same thing, that they would do the double check. But what is happening is that we are having officers now, like, work backwards. We're hoping that they will do that double check. We're hoping that they will make sure that of the top 10 people who might have done this carjacking,

that they've double checked through the data to make sure the person that the agent thought was most likely, was in fact most likely. And that radically changes what police do. It creates a whole host of questions about, well, how do we think about error and bias and exculpatory evidence that might be in the system? Really open questions that are just now being thought about, or at least I'm thinking about them because I spent the summer writing a lot of your article on it, but we'll be in the future because we've seen it in all these other industries. Like, this is the future. We know we have the technology. And the question is, when does it get applied to the policing data sets that we also know we have?

And is the gentic policing something that police departments are using right now? I don't think they're calling at that per se, although I'm calling at that. But I think even, you know, Wired magazine just has this great expose of how Flock is using pattern recognition within its Flock cameras,

at least they have the capacity to do so. I don't know if they're actually doing it. They sort of walked back where they were doing it. But they said, look, we can find all of the cars in this block that have also passed this street and this street. and they have these sort of pre-programmed prompts already in their systems, even at this early stage, which means they were thinking about it like months ago

or whenever they created the system, if not years ago. And so we are seeing the seeds of this being built out, in part because it's what police do anyway. If you ask an off-state, okay, you get a car-checking, what are you going to do? They could tell you the five steps that they take in every carjacking. And there's no reason why you can't create an AI workflow that sort of automates that in a way that is doing just what they are doing and have been doing for decades. It's not rocket science to recognize that this is what police will be doing

because it's what they have been doing. What we're seeing now is companies are kind of putting it within the architecture of their systems because now AI can make useful all of the data that has been collected. So I just want to stop here for a moment and really absorb what Andrew's talking about here. There's one surveillance story that's all about data, tech companies, data brokers, the government, they have a mountain of data on every citizen, and now they can use AI to analyze that data in real time to build out a surveillance network.

But I think Andrew's pointing to another version of a surveillance story that's actually all about AI agents, where you enter into a world where police and law enforcement are going to have access to thousands of agents or analysts that they can deploy to do their surveillance for them, all running at superhuman speed, never sleeping, never complaining. So just imagine like the classic, classic detective's cork board. You know, there's the person sitting there, the detective,

and they've got a cork board up, and they've got photos and pins and red yarn that's running from one node to another. And up until now, a human being had to sit down and make all those connections, you know, one at a time, over weeks or months or years. But now imagine that there was a thousand detectives with millions of red yarn strands connecting every possible point to every other point. So, you know, today, police could search for using flock, you know, a blue pickup truck with a kayak in the bed. But in the future, they can have an AI agent search for a blue pickup truck belonging to someone who voted a particular way,

whose dog has a microchip registered at this address, whose phone was at this intersection last Thursday. So this isn't just a story of more surveillance. It's a story of a different kind of surveillance entirely. Because for most of human history, the constraint on surveillance wasn't law. It's been the expense and time required to do it. And once that constraint is removed with AI's, is going to create an entirely different paradigm shift.

So I just want you to hold that in your head as we proceed with the episode. So far, we've been talking a lot about the United States and how technology has been applied here. But Garans, I know in your reporting, you've written a lot about how American companies have actually been building out a surveillance apparatus for the rest of the world. Ironically, we talk about China being this dystopian thing and we'll never build that here. But the U.S. may be helping to build that surveillance state in a way. Yeah, I mean, I think particularly in this moment of the AI race between the U.S. and China was important for us to pick apart some of that history of how these not only AI models were built over time, but some of the very basic computing infrastructure that also fueled surveillance networks. And it turns out that American technology companies, notably IBM, to a large degree, actually designed, and built China's surveillance state.

So some of our reporting uncovered just how Silicon Valley had played a greater role in enabling human rights abuses than was previously known. We also looked at how the American Congress had, through Democratic and Republican administrations alike, really failed to curb American firms from selling technology to Chinese police, government agencies and surveillance companies. And then also we took a look at how AI is now powering warfare. So to look at how models that have been developed right here in California have been used in the war in Gaza to help make targeting decisions, as well as how some of this surveillance technology has later been turned back into the United States, sort of with new Chinese amalgamations and used to track down dissidents here. in the U.S. So I think part of what's been important is to understand how systems are maybe piloted in one particular location in the globe and then, you know, adopted much more widely after that.

These are really global technologies. And so just because they surface in one place for one use case doesn't mean that they can't be applied or, you know, misapplied in another kind of of very troubling scenario. Which gets back to the power of these tools divorced from their context. Normally, it's laws and norms and transparency that protect us from those tools being misapplied in context to create worlds that we wouldn't want to have. And it gets back to the question, Andrew, of what are therefore the protections, norms, or transparency measures that we need in order to deal with the range of threats

we've talked about so far? Yeah, I mean, I think that in addition to constitutional protection, where courts can continue to sort of strengthen the Fourth Amendment in response to these new threats. And again, just to keep talking about ALPRs, there's ongoing litigation about whether a system of ALPR surveillance, these license plate reader cameras, is a violation of the Fourth Amendment because it reveals location data and where you are and what you're doing, whether you're at a health clinic or a gun range, it reveals where you are. and Congress or states could easily take a lead in sort of regulating this, regulating either requirements for audits or transparency, which we don't even have in most of these situations.

We don't even know if AI is being used. Two states have regulated whether AI is being used in police reports, Utah and California. But again, having, you know, knowledge about whether AI is being used or at least be relatively easy and benign first step. And then you could set up a warrant requirement. by statute. You don't even have to go to the Fourth Amendment floor of the Constitution and say, look, you know, this is obviously a very dangerous technology that could be misused. It has good uses, but maybe we only want to use it to find serious offenders. We could have a system of laws in place

that require that kind of warrant requirement. Say, hey, look, you want to go figure out which car was at this location this time. Go convince a judge why you need it. It's not perfect. It might not be all that protective, but at least it's something that we're going to be. we don't have right now. So that's just a couple ideas about how we can sort of update our laws and rules and regulations around this technology, which we haven't done. Graz, I'm sure you speak to a lot of legal experts and advocates in your work. What would you add? What have you been hearing from them about what we need as well? I think, you know, at this point, AI has gotten increasingly political, right? And so the current

administration is not in favor of those kinds of... state and local AI regulations. And I think we're going to see that playing out as the administration matures in its AI policy, we'll see what kinds of restrictions may come up at a federal level. But I think, again, what's been interesting in some of the reporting that I've done is to see that these very personal issues of privacy should authorities know where I am as a result of where my car is sometimes do really unite both Republican and Democratic lawmakers. For example, after the Border Patrol investigation, I mentioned, we had lawmakers from both

sides calling for more scrutiny of the Border Patrol's use of this ALPR predictive policing system. And I think that the way that flock safety, one of the companies at the center of this public debate, frames it, is that it's all about... Privacy versus security. How much privacy will you give up in order to get safety or security? But of course, that looks very different for different people. What safety feels like in one community may be very different from another. And I think that what's been interesting to watch is that this is an area where Democrats and Republicans kind of come together. We found both parties tend to value privacy a certain amount, which is why this is. kind of kicked up in recent months and weeks.

But of course, automated license plate readers can be linked up to all kinds of other information that track people's shopping habits, that track the driver's licenses of the people who the cars are registered to, their voting records that track their citizenship. And so it's kind of this, I think it's become a touch point in part

because in the United States, many of us have cars, and that's often seen as like a symbol of freedom. But in fact, it's just one way in which we're all being tracked through this mass surveillance. I do think that there is a moment now that because, maybe ironically, because of AI and because people are afraid of AI, the same surveillance technologies that have largely been in existence for the last five years are getting a fresh look. Honestly, we've had ALPR systems for a long, long time. They gave tickets in D.C. for over decades. But the fact that AI has changed the conversation in terms of a new fear, a new power, it has given some of these systems of surveillance a different look.

And I think adding to that is the overreach by the federal government in the context of immigration. I think the flock debate would be different if we had not seen ALPR. used to target protesters in Minneapolis. I think everyone realizes that the surveillance aperture could be turned against them in the future. Our cars, our smart cars, our connected cars, are also revealing that exact same data. If you are driving any modern car, you're being tracked everywhere you go. And the Bluetooth signal from your car, the GPS that allows you to press a button and get emergency services, guess what? That's all trackable as well. And so it's interesting to me that we've had

this new moment of a particular company, in particular technology, which we've actually had for years now. We've had ALPRs for years, has sort of created a spark where people are starting to push back on this idea of mass surveillance, even though there are so many other places in our lives, like every smart device you own is a surveillance device. Like we're putting, you know, wiretaps in our home called Alexis, and we have ring doorbells that are capturing ourselves, and all these things. And so we are actually trackable almost everywhere we go because we have these wonderful digital technologies everywhere. And ALPRs are a piece of that puzzle, but it's only one piece. Well, and there may be a response by listeners of like, yeah, well, I have a fitness

band and it tracks all these things for me. But, you know, that fitness band's business model is not to sell that data to other surveillance companies. It's interesting. how this actually interlinks with other new AI risks, which is that the assumption that the data that is tracked about me is safe is suddenly completely in question because you have AI systems like mythos that can hack into literally the classified systems of the United States government have been demonstrated as such. But you don't even have to get there.

Every single thing you've created in a digital world is available to the government with a warrant and sometimes without a warrant. There is nothing, I just wrote this book called Your Data Will Be Used Against You. And the takeaway is there is nothing too secret, too private that the police cannot obtain with a warrant. They can go, you know, your smart bed,

your period tracking app, your car, the smart heart stent in your heart, detectives can go to your doctor with a warrant and get that information and use against you in a court of law. That's the rule. That's the law as it exists. And we don't think about that way, but it's part of it. And so all of that

exists now, and when you add AI, which just allows you to make more use of these digital trails that exist everywhere, we really are in a very vulnerable situation vis-à-vis the government. What are some of the tools that citizens should know about today to protect themselves? So I'm thinking of things like deflock, which I believe is a map of where flock is deployed so that there's actually at least some transparency, because most people just don't even know that there are flock cameras in their town. What are some other tools that people should be aware of to protect themselves

and fight back? One thing I think it's interesting about the de-flok movement and the maps that have that is that I think part of the reason people are reacting to that is that it's like a tangible visualization of a fear about surveillance. You literally can see the camera. We have cameras all over the place, right behind those AI billboards in San Francisco, a whole series of cameras in downtown San Francisco

that go directly to the Real Time Crime Center in San Francisco, including all the bells and whistles of new policing that are there. But it's hard to see that. You don't actually see that as a physical manifestation of surveillance, and these cameras just symbolize, are standing in for that fear. So I think one thing is to open your eyes about how vulnerable you are to this world of self-surveillance, whether that's democratically mediated self-surveillance, meaning that it's

your tax dollars that are paying for those cameras. It's being done in your name. And because it's being done in your name, it means you have a rule to do something about it. You actually can call your Congresspeople. It is fascinating to me as someone who studied this whole industry for over 15. that they're ordinary people in city councils who are taking the time of their busy lives to go protest a surveillance camera. And I think part of that was education. We saw a movement of educating yourself about how these cameras work, how they can be misused.

And I think if we expand that to other areas of new policing technologies, we might have a stronger conversation and movement for our elected leaders to do something about it. So we spent most of this conversation focused on what's been going wrong with AI and surveillance, but now I want to ask you both the opposite question, which is, what is an alternative vision for how open societies and AI policing tools would work in a way that's not dystopian, but actually desirable? You know, not the sort of robocop or minority report vision, but the vision that I don't know if we actually have. So, Andrew, I wanted to start with you. It's not an easy question because the system, that could be designed requires a lot of trust in how police will use that power and that new

power. And many communities do not trust the police to use that power wisely. And the concern is that if you build a system that can be used for good reasons in a sense of like finding the murderer or whatever in the city, it can also be misused if the category of crime you're targeting, say people who criticize the president, gets changed. And that's what the danger is. That's really where the concerns are. I do think there's a compromise that could be made, even though I also think that that compromise may very well not work as well as we'd want, which is to essentially assume the worst and begin from there. I end my book saying that we should focus our attention on police surveillance technologies by envisioning the tyrant.

We should have, I call it the tyrant test. And we should picture the tyrant reading your most embarrassing Google search. And then begin from there. Like, what would you do to protect that information from being used against you in a way that is either, you know, criminal or embarrassing or a way to silence you politically? What steps would you take? And there isn't an easy answer, but it has to be in all of the above approach, including legislative changes, like I discussed from legislators, court decisions that enhance sort of the privacy of our information and our communications,

has to be sort of community-based because it might be different in different places. People have different sort of senses about their levels of privacy and their trust in police. And you have to kind of have an all-of-the-above approach and, you know, with checks and balances. And if that sounds sort of familiar, well, you know, America was kind of built on the tyrant test

because we realize that trusting power and one king was a problem. And so we built a whole system that tried to limit that power. I think we should do the same thing with policing technology and policing information that police can use or misuse too easily right now. Garan, have you seen any models of AI police tech that would actually help serve justice and protect our rights? You know, I think one thing that I keep coming back to is body-worn cameras.

So body-worn cameras were rolled out. if you remember, after the protests in Ferguson in 2014, President Obama at the time said that that would bring more transparency to communities who were really trying to understand what police were doing. So that was really held up as a way for communities to engage with the actions of law enforcement. Now, if you add AI-powered facial recognition to those body-worn cameras enabling police to identify people on the street, that brings in a whole host of other privacy concerns, right? I was going to mention a story that was not part of the big surveillance package last year, but that I thought was important, which was about the city of Edmonton in Canada,

that decided to run a pilot of live facial recognition in body cameras. So that was with the company Axon Enterprise. And it's thought to be the first time that a police force has done live facial recognition inside body cams. And I think that that raises a lot of really interesting questions because body cams, of course, were introduced to bring more transparency to policing. And now there's a sort of novel use of those very cameras that were meant to bring about more community transparency, right? Canada, you know, of course, was one of the first countries that had an AI strategy back in 2017, but I don't think that the AI strategy quite contemplated exactly what happened in Edmondon, right? So I think that one of the things that I think is important, again, is just for the general public to keep pace with what law enforcement tools are being used, where they're being used, you know, what law enforcement says those tools.

tools will accomplish and to really engage in that conversation because it can change from Ferguson, Missouri to the city of Edmonton in Canada, two very different uses for essentially a similar technology now with AI layered on top of it. One other interesting point on AI-assisted body cameras is you actually can enhance police accountability. I just wrote an article with Joanna Schwartz from UCLA that says that the ability to watch police encounters with citizens as they go about their daily business can be automated through AI to find problematic behaviors of police. There has been a rather difficult path to enforce civil rights lawsuits against police

for police misconduct, brutality, whatever. It's because it's hard to find a pattern and practice of police misconduct. But companies like Trulio and other sorts of companies that sell AI analytics for police body cameras can actually identify those very problems automatically. And so in one of the early studies, essentially every time the officer used a racial epithet or a curse word or used handcuffs or there's a use of force, it would be picked up by the AI. And so all of these like millions of hours of body camera footage could be isolated to find the problematic behaviors that was then forwarded to the supervisor.

So the supervisor now had noticed that their officer had gone rogue, and they could look into it for police accountability for better training and the rest. And that's a use of AI to improve policing through the same technology that exists. That might be a way of also improving the way police practice works every day. That's an interesting example because I think that it all depends on how technological progress is defined. You know, how these tools are built to optimize or predict actually shapes these very real-world outcomes. If you have a tool that is aimed at bringing more accountability to police pattern and practice,

as opposed to analyzing historical arrest data that may have biases built into it, that's two very different use cases for arguably a similar tool, right? So I think that it's in those kinds of questions that will continue to grapple with these bigger issues of civil rights, democracy, power and accountability. Yeah, I mean, it's sort of the classic, who watches the watchers, and this is essentially AI that helps watchers. This is also bringing to mind Estonia, which has some laws that could provide a decent model for this.

They've had a data tracker, apparently, since 2017, where citizens can see which agency access their data and for what reason. So citizens basically are not passive users of services. They own their own data and unauthorized access. is actually a criminal offense. And their criminal code also has a post-surveillance notification requirement. So once a surveillance authorization expires, the agency must immediately notify both the person that was surveilled

and any person whose private or family life was significantly affected and who was identified during the proceedings. I'm just curious if either do you have ever heard or thought about this example. What's so fascinating about that is that we have the technology now to do that. In fact, when Flock got pushback, they just changed their sort of default policies about ALPRs. And one of them was to say, you know, we actually have this audit system that can look at every time the police are using this for inappropriate ways. And we can catch them and fire them if they do so.

And the question, of course, is, well, why weren't we leaning into that to begin with, right? And maybe the answer to that is we don't have a law that's on the books. We're sort of relying on the tech companies to sort of make sure this audit ability is enabled and used. And I think the way's so interesting about that set of regulations in Estonia is that it really sounds like it empowers the public to understand how these technologies are being used. Whereas in the United States, mostly what we have is opt-outs. That's right. So you can decide that you don't want your data shared with XYZ agency or XYZ tech company or ad broker. But that doesn't actually give you the information of how, you know, this data is other.

otherwise being used, which in Estonia, it sounds like you're just getting much more transparency into how these systems work, which in turn makes for a better informed public. Yeah, I love that example. Just by having transparency and more of this visibility, it's actually educating the public about the nature of what is surveilling them and the kinds of information that can be inferred from that. One sort of basic rule or principle that came to mind when thinking about this is, is the AI tool or the technology that's being employed? Is it used to check a human decision or is it used to make a human decision? And I feel like that is what at least one distinction as well that can help with steering us towards that more pro-human, democratic, open

society that we want to steer towards. Well, Garan's and Andrew, thank you so much for coming on your undivided attention and lightning everybody about how these technology trends and the law are intersecting in this new world. Thank you so much. Thank you. Pleasure to be with you. Your undivided attention is produced by the Center for Humane Technology. We're a non-profit working to catalyze a humane future. Our senior producer is Julia Scott. Our executive producer is Josh Lash. Mixing on this episode by Jeff Sudaken with original music by Ryan and Hayes Holiday.

And a special thanks to the whole Center for Humane Technology team for making this show possible. You can find transcripts from our interviews and bonus content on our substack and much more at HumaneTech.com. liked this episode, we'd be truly grateful if you could rate us on Apple Podcasts or Spotify. It really makes a huge difference in helping others discover this podcast and join the movement for a more humane future. And if you made it all the way here, let me give one more thank you to you for giving us your undivided attention.