The Good Tech Companies - AI Agents Aren’t Production Ready - and Access Control Might Be the Reason

Episode Date: April 16, 2025

This story was originally published on HackerNoon at: https://hackernoon.com/ai-agents-arent-production-ready-and-access-control-might-be-the-reason. Learn how to implement proper access control for AI agents in applications for production-ready AI systems. Check more stories related to cybersecurity at: https://hackernoon.com/c/cybersecurity. You can also check exclusive content about #ai-security, #access-control, #rag, #mcp, #ai-agents, #langchain, #fine-grained-authorization, #good-company, and more. This story was written by: @permit. Learn more about this writer by checking @permit's about page, and for more stories, please visit hackernoon.com. AI agents face critical security challenges due to dynamic data access and new attack vectors. This article presents the Four-Perimeter Framework, which secures AI workflows through prompt filtering, RAG data protection, external access control, and output validation—using fine-grained access control (ABAC & ReBAC) to enhance AI safety.

This audio is presented by Hacker Noon, where anyone can learn anything about any technology. AI agents aren't production ready, and access control might be the reason. By permit, EO, Daniel Bayes' enhancing application performance through the use of AI agents is becoming a very common practice, whether due to industry-wide hype or actual helpful use cases. Regardless, it helps automate workflows and enhance user experiences across various industries. These AI agents, however, often require access to highly sensitive data, introducing significant security risks. Without standardized access control solutions, development teams often build security measures from scratch, leading to cases where AI assistants expose sensitive patient information and several AI agents being compromised through sophisticated prompt injection attacks.

Protecting AI agents in this context is becoming just as important as the capabilities of AI agents themselves, if not more so. To address this problem, we at Permit.io have been working for a while on a structured security approach I wish to introduce to you today, the 4-perimeter framework. This method provides a comprehensive strategy for protecting AI agents by implementing fine-grained access control, ensuring compliance, and mitigating potential attack vectors. Before I go into the solution, let's talk more about the problem space. The problem, AI security risks in sensitive domains. Traditional application security measures and access control mechanisms fall short when it comes to handling AI agents. While conventional access control systems like Role-Based Access Control, RBAC, excel at managing static permissions, they struggle with the dynamic nature of AI responses. Unlike traditional applications where inputs map to predictable outputs and

access patterns are well defined, AI generates unique responses for each query. This means the access control and input validation techniques were used to cant anticipate all possible outputs. The way AI systems access data also doesn't fit security standards. To provide accurate and helpful responses, AI agents need broad access to various data sources and knowledge bases. Classic RBAC proves too rigid for this need, you can't simply assign static permissions when the data requirements change based in context. This calls for a more sophisticated, attribute-based approach that can make dynamic access decisions. Another concern is the new attack vectors that AI systems introduce. Conventional security measures weren't designed to handle prompt

injection attacks, model extraction attempts, or data poisoning attacks. These target vulnerabilities unique to AI systems, requiring entirely new protection methods. The solution comes in the form of a comprehensive security framework built around four essential perimeters the solution, the four perimeter framework. As AI applications differ from standard software in how they process inputs, retrieve data, execute actions, and generate outputs, they also introduce unique security risks. To address these challenges, we need a structured security approach that applies access control at every stage of AI interaction.

And that's exactly what I want to showcase here. The four-perimeter framework is designed to enforce identity-aware, fine-grained authorization throughout the AI lifecycle. It introduces security boundaries that govern what data AI models can access, what operations they can perform, and how their responses are validated. The framework consists of four parts. One, prompt filtering, ensuring only validated, safe inputs reach AI models. Two, RAG data protection, controlling AI access to external knowledge sources. Three, secure external access, defining AI agent permissions when interacting with external tools. 4. Response Enforcement, applying compliance checks and filtering I-generated outputs By applying fine-grained access control, FGA, principles across the separimeters, AI agents remain secure, auditable, and compliant,

without sacrificing their flexibility and functionality. Where does FGA come in? As mentioned earlier, traditional access control methods like RBAC fail to handle AI's dynamic, context-dependent behavior. This is where fine-grained authorization, FGA, comes into play, specifically through attribute-based access control, ABAC, and relationship-based access control, REBAC,

ABAC for prompt filtering AI models process unstructured inputs, making it difficult to apply traditional security rules. ABAC solves this by extracting structured attributes from AI prompts and using them in policy-based access decisions. REBAC for RAG data protection retrieval augmented generation, RAG, allows AI models to pull information from vector databases. Reback provides a natural way to enforce security in this setup by defining relationships between users, data sources, and AI agents. Instead of assigning static permissions, Reback dynamically grants or denies access based

on how data objects relate to one another, enabling context-aware retrieval permissions. By combining ABAC and REBAC, AI applications gain flexible, policy-driven access control mechanisms that adapt to different AI workflows without requiring manual intervention. What does a step-by-step implementation look like? Let's walk through a practical implementation

to see the four-perimeter framework in action. This example secures an AI agent by validating prompts, enforcing data protection, restricting external access, and moderating responses, leveraging permit. IO AI access control integrations at each step. One, implement prompt filtering. The first security perimeter focuses on validating and sanitizing AI inputs before they reach the model. We can enforce structured input handling and enforce access policies on prompts before they reach the AI model authorization checks are embedded into the prompt validation pipeline, preventing unauthorized data from influencing AI outputs. Role, attribute, and relationship-based

access control are enforced, ensuring only approved parameters pass to the AI. Any policy violations are logged and blocked in real-time, preserving an auditable trail of access decisions. Here are the three methods to filter prompts, from easiest to most advanced one. Token validation. An efficient method for simple checks like prompt length or the general allowance of running inputs in the system. Permission check functions in this method examine the structured attributes of textual prompts.

2. Pattern matching. In this method, the permission check examines patterns in the prompt text. Checking, for example, if the pattern matches in SQL query or code example. 3. AI classification. The most advanced method in this list. AI classification uses AI to analyze the prompt using a dedicated system prompt, classifying it into a structured input that can be precisely analyzed in the permissions check. Prompt filtering with Pydantic EIDO best demonstrate the implementation of efficient prompt filtering. Here's an example of using Pydantic AI tools to filter user inputs to AI agents. Pydantic AI is an agent framework that takes advantage of Python's famous Pydantic library and its static typing capabilities and turns it into a structured AI framework that handles unstructured data.

The following GitHub repository contains a full implementation of the framework for Pydantic AI, github.com, permissio, permit Pydantic AI to enforce data protection next, we secure retrieval augmented generation, RAG, queries by ensuring AI models can only access authorized knowledge sources. This can be achieved through fine-grained filtering of the AI agent data on behalf of the user within the AI workflow. The fine-grained policy uses advanced data filtering models, such as relationship-based access control, to allow advanced query filtering to the policy engine. The vector and graph sources connected to the AI agents have metadata that can help the engine manipulate

the unstructured query, filtering it to include only allowed data. Every time the agent fetches data from the RAG knowledge base, it filters the results per the user's permission. When the agent fetches data from the RAG knowledge base, it filters the results per the user's permission. When the agent application performs RAG filtering, it uses one of the following methods. In this method, the graph application fetches all the relevant data from the RAG and then filters it per user permissions. This function's advantage

is that it keeps the policy engine stateless and unaware of RAG data. In this method, the agent provides the RAG with an unstructured query. The RAG then calls the function, appending a filter query to the unstructured RAG query. This allows the query to be filtered to only the resources the user can access. RAG data protection with Langchain Langchain, the famous AI application framework, is known also for its retriever components that provide great support in custom retrievers that can be assembled with any kind of data source, making the implementation of secure RAGC. For example, with the, you can wrap the usual rag with a function to twill append the filtered

data to the query. Here's an example of the Permit self-query retriever that gets the user, action, and the type of the RAG resources to filter the results for the LLM. The following repository contains the full implementation of the framework for Langchain. https://github.com, permissio, langchain, permit3. Secure external access AI agents often interact with external APIs, services, and automation tools. Without proper access control, they risk executing unauthorized actions.

Thus, we must ensure I-driven operations are restricted based on machine identity policies. Here's how this can be achieved using models like MCP, which defines a server-client model where AI agents interact with external services, is a great step in enabling security for AI external access, e.g. databases, APIs, payment systems. The AI action infrastructure can use a service like PERMIT, IO to perform authorization checks at the action level, checking who's making a request and what action they want to perform. Developers are able to assign machine identities to AI agents, limiting their capabilities to pre-approved functions only. While GPTs and natural language interfaces aren't new,

letting them perform actions on behalf of users is a fairly new challenge many engineers struggle with. We can notice three distinct levels of access control important to ensure here. One, direct access security, the simplest level. A user asks an AI agent to perform an operation, such as an HTTP call. The AI agent knows to check if the operation is allowed, considering the current context and user. 2.

Agent to agent communication. In this next level, we require a cascading flow of identities and permissions that allow agents to perform actions themselves based on the principle of least privilege. At this level, we would want to use the relationship between identities, human and non-human, to cascade different levels of permissions. 3. Access Request Flows The most exciting level of external access is when the AI agent understands the need to request access directly from a human user when it wants to perform an operation. The agent knows which user to ask for permission, and the operation will only be performed once this permission is granted. MCP, Human in the Loop, and External Access Model Context Protocol is a new protocol introduced by

Anthropic, which solve a steep problem of letting AI agents perform proactive actions and serves as the greatest enabler for securing external access. In the following example, we show an MCP server that knows how to manage access request flows by bringing humans into the loop and uses permit. Ios APIs to ask for permissions. You can see the full code example for the MCP access request server here. https://github.com, Permitio, Permit MCP 4. Validate I responses The final perimeter enforces content moderation and compliance on AI generated responses. This can be done by creating a workflow that applies policy based filtering before delivering AI outputs a permission check can be added to the agent application,

enabling policy enforcement at each step of the chain before the user actually gets a response. Combined data validation and authorization workflows ensure that only validated and permitted data can move forward. The response that returns to the users can be edited based on predefined limitations, thus allowing for data masking or notifying the user of use limitations. Response filtering with Langflow in the following example, we use Langflow, a visual no-code AI applications editor, to create a permissions check component that's positioned before any chat response to the user. Using Langflow's friendly flow modeling capabilities, you can easily append another component that masks unwanted details in returned responses. This repository contains all the access control components required for secured Langflow applications.

Conclusion, AI systems are rapidly integrating into applications across industries, but hair security frameworks remain underdeveloped compared to traditional software. Without proper identity-aware access control, AI agents risk exposing sensitive data, executing unauthorized operations, and generating responses that fall outside compliance guidelines. The four-perimeter framework offers a structured way to secure AI workflows at every stage, from validating prompts and protecting retrieval augmented generation, RAG, data to controlling eye-driven external actions and enforcing response compliance.

By leveraging fine-grained authorization, FGA, with ABA canned reback, developers can ensure AI agents remain both functional and secure, adapting dynamically to real-world security scenarios. Instead of building access control from scratch, Permit, IOS AI access control integrations with Pydantic AI, Langchain, MCP, and Langflow allowed teams to embed security directly into their AI applications, without disrupting development workflows. Try it yourself here, that said, this field is still relatively new, and best practices for securing AI agents are still evolving. The 4-perimeter framework is an experimental approach, and best practices for securing AI agents are still evolving.

The 4-perimeter framework is an experimental approach, and while it provides a strong foundation, I recognize that real-world deployments always require further refinement and adaptation. I'd love to hear your feedback, thoughts, and ideas on how to improve this framework to make it even better for production-ready AI systems, comment here, or hit me up in our slack community. Thank you for listening to this Hacker Noon story, read by Artificial Intelligence. Visit HackerNoon.com to read, write, learn and publish.