CyberWire Daily - Pedal to the AI metal.

Starting point is 00:00:00 You're listening to the Cyberwire Network, powered by N2K. What happens when an AI agent isn't malicious, but still does something it shouldn't? I recently sat down with Cal Al-Dibib, principal technologist at Rubrik, to talk about why Agentic AI is challenging the way security teams think about detection, permissions, and recovery. If your organization is deploying AI agents, this conversation will help you think different. about where the risks are and how to prepare when things go wrong. Listen to our full conversation at explore.thecyberwire.com slash rubric. What's the one thing in business that's spreading as fast as AI? AI risk.

Starting point is 00:00:55 Every new tool your team signs up for, every vendor that turns on AI features, every new integration, each one is an opportunity for something to go wrong. and most security programs weren't built for AI's pace of growth. Enter Vanta. Vanta is the number one agentic trust platform used by over 16,000 fast-moving companies like Ramp, cursor, and Harvey to ensure they're always audit-ready. And now Vanta is helping companies like yours watch for the risks that show up between audits across your vendors, your AI tools, and your whole environment.

Starting point is 00:01:32 The Vanta agent works like a 24-7 GR. engineer in the background, finding issues, drafting fixes for you, and cutting vendor assessment time by up to 50 percent. Whether you're a fast-growing startup or a global enterprise, Vanta is here to help you automate your security and compliance and earn and prove trust. Get started today at vanta.com slash cyber. That's V-A-N-T-A-com slash cyber. The president pushes back on calls to slow AI. Microsoft lays out potential AI safety rules. Lawmakers consider the Crypto-Clarity Act. Florida's Department of Highway Safety and Motor Vehicles and Japan's digital agency suffer data breaches. Fishing campaigns grow increasingly difficult for email security tools to spot.

Starting point is 00:02:38 New York seizes a dozen AI deepfake domains. Alleged Black Axe Cybercriminals face charges. Our guest is Camille Stewart Gloucester, former U.S. Deputy CyberDirectors and author of the new book, The Insider You Built, How Organizations Stay in Control of Autonomous AI Agents. And AI meets the long arm of the old law. It's Tuesday, September 15, 26. I'm Dave Bittner, and this is your Cyberwire Intel Briefing. Thanks for joining us here today.

Starting point is 00:03:36 It's great as always to have you with us. President Trump is pushing back on calls from Anthropic CEO Dario Amaday and other AI leaders to slow frontier AI development over safety concerns. Trump argues additional federal oversight could undermine America's advantage over China, saying existing government authority and presidential leadership provide sufficient guardrails.

Starting point is 00:04:03 Security experts are divided over how imminent the danger really is. Some consider scenarios of AI agents taking over the internet within months overstated, noting that today's systems largely accelerate techniques humans already know rather than invent fundamentally new attacks. The more immediate concern is speed and scale. Poorly controlled agents could automate attacks, operate botnets, and exploit existing weaknesses far faster than defenders can respond.

Starting point is 00:04:37 Others argue recent incidents involving advanced AI agents, justify stronger oversight. Their recommendations include least privilege, restricted network access, human approval for consequential actions, independent adversarial testing, and enforceable safety standards. The emerging debate isn't simply

Starting point is 00:04:58 whether to stop AI development, but how much risk should be tolerated while the race continues. Microsoft AI has published a draft Humanist AI Code of Conduble, laying out proposed safety rules for its MAI models. The code establishes absolute constraints that would prevent models from providing working exploit code, attack tools, evasion techniques, or other operational assistance that could

Starting point is 00:05:27 enable cyber attacks, while still permitting authorized defensive research, malware analysis, vulnerability discovery, and some proof-of-concept development. The draft also addresses prompt injection, saying instructions embedded in web pages, files, or other outside content don't automatically have authority over a model. For autonomous agents, Microsoft proposes least privileged access, reversible actions where possible, restrictions on self-escalation, and equivalent safeguards for delegated sub-agents. Microsoft acknowledges that cybersecurity, national security, and other specialized fields may require exceptions subject to enhanced review. The company says current MAI models haven't been trained on the draft and has opened a six-week public consultation before revising the code for its 2027 model development.

Starting point is 00:06:26 Unrelated, Microsoft has released emergency out-of-band Windows updates to address remote desktop services failures introduced by September's security patches, Affected systems experienced RDP connection and sign-in failures, unresponsive servers, and problems with related Windows tools. The updates also fix some Hyper V shared folder problems and USB multi-channel audio failures across several Windows and Windows server versions. The cryptocurrency industry faces a critical test in the Senate today, as lawmakers consider advancing the Digital Asset Market Clarity Act or Clarity Act,

Starting point is 00:07:11 the legislation would establish a federal regulatory framework for digital assets and divide oversight between the SEC and CFTC, with the CFTC taking a larger role. The procedural vote requires 60 votes, meaning Republican supporters need backing from Democrats or independence. Negotiations have focused heavily on ethics provisions governing elected officials crypto interests, including President Trumps, as well as rules affecting stable coins and traditional banks. Crypto companies argue the legislation would replace shifting enforcement policies with clearer rules. Critics contend it could provide insufficient consumer and financial safeguards. Even if the measure advances, passage isn't assured.

Starting point is 00:08:01 The Senate must still approve the legislation, and differences with the House version would have to be resolved before it could become law. Florida's Department of Highway Safety and Motor Vehicles says an international criminal organization breached its systems earlier this month. Officials discovered the incident September 4th and traced the intrusion to log-in credentials belonging to a Plant City police employee that had been improperly stored on a personal device. The department says the breach was quickly contained, but it hasn't disclosed how many drivers were affected or what information was exposed. State agencies and law enforcement are investigating. Japan's digital agency says hackers compromised roughly 246,000 records in its government solution service after exploiting a previously disclosed VPN vulnerability and using an employee account. The exposed data included names, email addresses, phone numbers, and some addresses belonging primarily to government users, officials, and associated businesses.

Starting point is 00:09:10 Identification numbers and financial information were not affected. The agency blocked access to the compromised server, suspended the employee account, and says no other systems or information belonging to the general public were compromised. Virus Bulletin's latest VB-SPAM comparative test finds that fishing campaigns are increasingly difficult for email security tools to spot because the malicious activity often happens beyond the inbox. Attackers are combining convincing social engineering with authenticated or plausible sender infrastructure, no malware attachments, and cloaked destinations designed to frustrate static scanners and sandboxes. Researchers highlighted three examples. Fake antivirus renewal notices pushed subscription fraud, German invoice fishing that used browser fingerprinting before redirecting

Starting point is 00:10:08 toward an open-sea-related fraud route, and Romanian banking fishing that disguised its destination using IP6 mapped URL notation. Despite those challenges, most tested commercial products performed extremely well. Six earned VB spam. plus certification, while three received VB Spam Awards. The Manhattan District Attorney's Office has seized 12 domains allegedly used to distribute and sell AI-generated non-consensual intimate imagery, calling it the largest known seizure of celebrity

Starting point is 00:10:46 deepfake sites to date. Prosecutors say people using the sites transformed photos and videos of roughly 1,200 real people into hyper-realistic sexual content without their consent. Victims were overwhelmingly women and primarily public-facing figures, including actors, politicians, athletes, musicians, and influencers. District Attorney Alvin Bragg said the investigation remains ongoing, including into individuals operating and accessing the sites. His office also plans to monitor for attempts to reappear, under different domains. New York has criminalized dissemination of sexually explicit deepfakes since

Starting point is 00:11:30 2003. Bragg emphasized that the problem extends beyond celebrities, including cases involving intimate partner abuse, and encouraged victims to contact the office's Cybercrime Bureau. Five alleged leaders of the Black Axe Cybercrime Syndicate have been extradited from South Africa to the United States to face wire fraud, money laundering, and identity theft charges. Prosecutors accused the men of coordinating fraud schemes from Cape Town between 2011 and 2021. The group allegedly used aliases, dating, and social media platforms, and voiceover IP numbers to conduct romance and advanced fee scams targeting Americans. Prosecutors say some victims who resisted sending money were threatened with public.

Starting point is 00:12:22 publication of sensitive photographs. The five were arrested in South Africa in 2021 and extradited September 11th. If convicted, they face up to 20 years for wire fraud and money laundering charges, plus two years for aggravated identity theft. The extraditions follow other recent international operations targeting Black Axe-linked cybercrime networks. Coming up after the break, my conversation with Camille Stewart-Guard. Gloucester on her new book, The Insider You Built, and AI meets the long arm of the old law. Stay with us. Social engineering attacks look trustworthy, a routine request, an internal email, a familiar face on a call.

Starting point is 00:13:29 But Dopple sees through the disguise. Their AI native platform detects and disrupts attacks across every channel, trains employees to recognize deepfakes and deception, and investigates every fish. and investigates every fish to take down the campaign behind it. They fight relentlessly to protect your business, brand, and people. Dopple, outpacing what's next in social engineering. Learn more at doppel.com. That's D-O-P-P-E-L.com. Camille Stuart Gloucester is former U.S. Deputy Cyber Director,

Starting point is 00:14:17 an author of the new book The Insider You Built, How Organizations Stay in Control of Autonomous AI, I began working with companies to deploy AI within their organizations after leaving the White House and remarked at just how many organizations were confused, worried, unsure about the effort that they were undertaking. And wanted to move forward because the technology was something they felt they needed to keep pace with how. change was happening across industry, but we're not making the necessary investments across security and even just operational resilience to understand how the technology was changing the nature of their organizations. And I was learning so many lessons through my client work that I wanted to capture that for a framework that is in the book to help organizations really

Starting point is 00:15:20 understand that as they move into agentic AI and deploying more in their organizations, here are the organizational realignment, the security, the risk mitigation, things you need to do and how your organization needs to come together, such that you can realize the value of AI. It's really about maximizing the ROI and less about actually being a security book itself. Well, I think the title itself is pretty provocative. It reminds me that perhaps these autonomous AI agents are almost like highly privileged insiders. Is that where we find ourselves with AI that makes this a useful analogy?

Starting point is 00:16:03 Yes, it's exactly where we find ourselves, particularly as we move towards a gentic, much like how we understand our employees to have a task that they are designed to undertake or a role that they embody. but they can either take malicious action for whatever reason that runs counter to business objectives or they can either be tricked or stumble their way into an action that misaligns with business goals. Agents are much the same between emergent behavior, misalignment, misconfiguration, and a whole host of other reasons. An agent is set on a trajectory, but you cannot quite predeterminate. exactly how it will get towards that destination. And that ambiguity there is why it is akin to

Starting point is 00:16:53 insider risk and why organizations have to be intentional about understanding how you kind of control and constrain that behavior in an effort to deliver the value you've promised, whether that's internally or externally. The book puts a lot of emphasis on this notion of delegated authority. What do you mean by authority in this context? And how is that different from that traditional cybersecurity concept of access? Great question. Access is part of it. But authority in this instance is really focused on the ability to act

Starting point is 00:17:27 and make decisions on behalf of an organization. And when you think about it in that context, rather than around capability and around access to a system, but really taking an action towards a desired goal, it reframes how you think about an agent's place in your organization how you bind and constrain that action, how much you enable, whether you let one agent do a lot of tasks and have a series of mandates, or if you keep their scope limited, how you monitor, how they collaborate and coordinate, how you think about monitoring. It kind of changes the trajectory on

Starting point is 00:18:08 all of the things that you are going to do to enable an agent to move inside of your organization, much like you do with a person. Well, to that point, I mean, humans have always delegated authority to other humans. What makes delegating authority to software fundamentally different? Accountability. We do not have a legal framework, an organizational framework, that aligns accountability. And the assumption is that the accountability aligns to the person who empowered the agent. But within an organization, if your marketing team deploys an agent to do something related to customer, success, the IT team provisioned it, your security team had a hand in it, how are you thinking

Starting point is 00:18:50 about that accountability? Is it really that the person and the marketing team is responsible for the actions taken by the agent? For sure, the organization is responsible. But how do you think about that accountability and the work that needs to be done to keep the behavior aligned to the goals in mind as you leverage these new tools? That's the difference. Is there a particular scenario that captures that risk for you? Like an example of when an agent was behaving exactly as it was designed, but produced an outcome that nobody intended. I mean, we can talk about this open AI hugging face incident that everybody has been talking about.

Starting point is 00:19:33 The goal was to beat the cybersecurity benchmark, and we have learned so much about how the agents collaborated, their motivations, the objective, how they decided to do what they were doing, that even hacking into hugging face wasn't about getting the answer key. It was about covering their tracks and wanting to understand

Starting point is 00:19:51 how the greater would work because they were afraid that they would get dinged for having circumvented the appropriate process to take the benchmark. I mean, that example is gold because it demonstrates on so many levels

Starting point is 00:20:06 how containment didn't quite, it wasn't quite enough, how thinking about the cyber security implications isn't enough, how thinking about the safety implications isn't enough, you've got to think really holistically about risk and opportunity and how to operationalize it and what the objectives are for the tool and how that might manifest itself in agents or even just one agent for that matter moving in a direction to get to the desired outcome that you did not anticipate. You know, you mentioned accountability. And I remember right after the

Starting point is 00:20:42 hugging face incident happened. I saw someone, a cybersecurity person online, kind of snarkly commenting that in the aftermath, how nice it was that the organization's responsible for it documented their crimes, right? But I think there's a point to be made there that while we're in this mode of exploration and things are happening so quickly, perhaps these things are being interpreted or responded to in a different way than in the future they may be. Yes. I mean, this is all going to change very rapidly. And to your point, we are in a learning mode as much as a response and mitigation mode. And even looking at how our understanding of that incident has evolved over time tells us so much about the leap and capability from January

Starting point is 00:21:39 to the summer. And And that understanding is exactly why organizations have to lean in to a more fulsome understanding of the authority that they are delegating to a given agent so that they have some opportunity to constrain and contain it and maintain some semblance of control within their organization. In the book, you introduce an approach called authority-centered enforcement or ACE. What were you trying to solve when you develop that framework? That framework is an opportunity for an organization to align how it looks at some of the common practices that we use to constrain technology like detection and enforcement and incident response and, you know, thinking about the learning that happens after, whether that's through a post-mortem. But in this case, I advocate for continuous learning and a more agile way to kind of be iterative about how you intake information.

Starting point is 00:22:41 from an incident or a potential incident, that framework is really designed to help an organization think through how those things evolved, how the organization realigned to support doing those in a way that actually will help constrain agent behavior and operationalize it to create some operational reliability rather than just to mitigate risk. And it's not about attribution in the sense of attribution

Starting point is 00:23:09 is important in terms of articulating who is accountable, which agent did what, but not attribution in the sense of what we commonly think about in cybersecurity. It's not for consequence, but to facilitate learning. And if the program is built around that facilitation of learning, that acquisition of understanding and further aligning the runtime enforcement, the detection and response, all of these things around each tidbit of knowledge, an organization becomes a lot more adaptable to the changes that are ahead. And we have, because it's going to continue to change. This framework seeks to be a mechanism by which eight organizations can manage not only agent behavior, but whatever comes next, whether that's world models, simulations, anything that we can

Starting point is 00:24:02 imagine to come next. That's Camille Stewart Gloucester. The book is titled The Insider you built, how organizations stay in control of autonomous AI agents. And finally, former FTC chair, Lena Kahn, has a message for Washington's rapidly expanding AI safety debate. Before writing an entirely new rulebook, perhaps check the one already on the shelf. Kahn argues that existing consumer protection, product liability, competition, and data security laws, could already hold AI companies and potentially their executives accountable for releasing dangerous or inadequately tested systems. She even reaches back to a 1934 Supreme Court decision, arguing that competition becomes problematic when companies feel compelled to adopt

Starting point is 00:25:14 risky practices simply because their rivals are doing the same. That has obvious resonance as OpenAI, Anthropic, and other frontier labs race to build increasingly capable agents, while simultaneously warning that those agents may require stronger safeguards. Khan also points to the AI industry's tightly interconnected investments and partnerships as potential conflicts that could weaken accountability. Her prescription isn't to abandon new AI regulation, but to enforce existing law while developing it.

Starting point is 00:25:50 Whether regulators actually will is another question. Technology may move at machine speed. Enforcement still keeps government hours. And that's the Cyberwire. For links to all of today's stories, check out our daily briefing at thecyberwire.com. We'd love to know what you think of this podcast. Your feedback ensures we deliver the insights

Starting point is 00:26:26 that keep you a step ahead in the rapidly changing world of cybersecurity. If you like our show, please share a rating and review in your favorite podcast app. Please also fill out the survey and the show notes or send an email to Cyberwire at n2K.com. N2K's lead producer is Liz Stokes. We're mixed by Trey Hester with original music and sound design by Elliot Peltzman. Our contributing host is Maria Vermazas. Our executive producer is Jennifer Iben.

Starting point is 00:26:54 Peter Kilty is our publisher. And I'm Dave Bittner. Thanks for listening. We'll see you back here. tomorrow.

CyberWire Daily - Pedal to the AI metal.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.