No Priors: Artificial Intelligence | Technology | Startups - Building an AI Guardian for Enterprise with Onyx Security CEO Maxim Bar Kogan
Episode Date: May 28, 2026We are now closer than ever before to living in a world where AI agents are smart enough to run our power grids and manage water supplies. How do we keep them from going rogue? Sarah Guo sits down wit...h Maxim Bar Kogan, founder and CEO of Onyx Securities, to explore the complexities of supervising and securing autonomous agents at the enterprise level. Maxim explains Onyx’s product as an AI control plane, which oversees the permissions and flexible contexts of agents while balancing latency, cost, and reliability. He also discusses how current controls have insufficient context to monitor agent intent, tradeoffs for gradual model rollout, the need for vendor-independent oversight, and Israel’s growing AI and security talent ecosystem. Plus, why Maxim is all-in on AGI. Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @maximbarkogan Chapters: 00:00 – Cold Open 00:45 – Maxim Bar Kogan Introduction 01:10 – AutoGPT and Betting on Agent Actions 05:17 – What Onyx Product Does 07:47 – State of Deployment in Large Enterprises 09:58 – Securing Agents 12:45 – Why Proxies Don’t Work 14:11 – Why Onyx Trains Its Own Models 18:38 – Onyx’s Talent Culture 21:24 – Mechanistic Interpretability 23:35 – How Onyx Builds Customer Trust 25:10 – Mitigating Risk at the Foundational Level 27:45 – Phased Rollout of Glasswing and Daybreak 29:11 – Large Enterprise Holdouts 30:46 – Onyx and the Larger AI Security Space 32:36 – Should Labs Address Model Trust and Governance? 36:56 – What Needs to Happen in Security 39:14 – Why Maxim is AGI-Pilled 41:15 – Conclusion
Transcript
Discussion (0)
As you're exponentially doing more things with their eyes, you're going to start having really bad actions happen.
And we've seen some of that happen lately with agents accidentally publishing code and tokens that they weren't supposed to.
Like, definitely enterprises are starting to realize that that risk is grown exponentially and that they don't have any way to stop the adoption.
They just now have to do something to reduce the chance of these agent actions being illegitimate or incorrect.
But we're allowed to look at a lot of historical data of how these agents have behaved, but
Antipers to that are not willing to have anthropic or open AI give that historical data
because they know these are very data-hanging companies that will want to train on that data.
Hi, listeners, welcome back to NoPriars.
Today, I'm here with Maxen Barcogen, the co-founder and CEO of Onyx Security,
an Israel-based startup of researchers, mathematicians, and engineers, building agents to watch the AI agents.
We talk about specialized model training, mythos, alignment research, and the Israeli ecosystem in security,
and now AI. Welcome. Maxim, thanks so much for doing this. Thank you. Pleasure to be here.
Everyone is much more concerned about security and the impact of AI on security than they were
certainly a few months ago. The consensus risk story two years ago when you started the company
was basically like DLP for chat bots, like what are what are employees putting into
chat GPT? Now we have clearly something that is not quite panic, but close to Mars.
market-wide panic. How did you decide to bet on agent actions when you started?
Look, I think for us, the pivotal point was auto-GPT. I think auto-GPT kind of let everyone's
imagination, including ours, run wild because it was a- Can you remind listeners what that was?
Sure. So auto-GPT, and I'm sorry if I don't know the guy behind it, but a huge, huge fan.
And they created the first, as far as I know,
first really autonomous agent running on LLMs, right?
So an agent that would let LLM not generate text,
but decide what to do,
and then give that agent an API access
to do that thing, a tool to do it.
And then we'd do that in a loop.
So basically, in theory,
could let agents do very complicated things,
anything a person could do on the computer.
Now, granted, it didn't work that well.
too early. The models were not good enough. GPD4 was not good enough. But I think it did give
everyone a glimpse to the future of, you know, what if the models were good enough and then
basically using that same structure, we could have very capable agents doing stuff for us. I think that
was in many ways, cloud code today is not dissimilar to auto GPD back then. I think they were
a bit early on, again, before the malls were ready, but the concept was right. And the thought
that stick with me was, I was very eye-pilled even back then. So I was thinking, oh, my God,
malls are going to be way smarter than us. When that happens, how do we oversee these very
smart agents that are, you know, they're smart on us, they're very capable, how we're going
to feel easy about them doing stuff for us, especially when they start managing really important
stuff, you know. Then one day they're managing your water supply and your electricity or power
greed, right? How do you control them? And that was like the thing I was kind of obsessed about that
thought. I was also too early. So I think at the time enterprises were not using any agents.
And there were hardly any agents out there. And talking with a lot of security budget at the time,
they were like, oh, dude, you're way too early. Like this is not something that's going to happen
I asked you the same question. I said, is anyone going to do this before you run out of money?
And I think there was a good chance that I would have run out of money before because I think you were right.
Like I think there was an element of chance here. But then I think the market did happen.
So we had suddenly reasoning models that could do long horizon tasks. We had cloud code, which became like the really first widely used autonomous agent.
and then we had co-work and OpenClaw.
And I think we're starting to see now that these types of agents that are very autonomous,
even though they're like everyone was afraid to build them.
So everyone started building these low-code platforms that were much more limited,
much more based on connectors.
Those platforms ended up being quite limited.
So we didn't get the productivity gains from those limited platforms.
But when we started getting the crazy benefits from these very unleashed
agents that could do everything, that had much less controls baked into them. And even very
large enterprises decided they're going to adopt it, you know, like Anthropics revenue is coming
from enterprises that are paying for cloud code to do a lot of the work that developers
used to do. That was a bit about kind of how we started. And we definitely were in luck that
very autonomous agents appeared before it was too late. So can you describe a little bit just
because it's, I think, both close to impossible and then very useful in this period of AI
to think about what is deployment right now and then, you know, what's changing about capability?
What's the one-liner on what the ONIX product does today?
And then, like, how you think about long-term vision.
Today, like, Onyx is really, that's two things.
Number one is we train models and build agents that can oversee other agents.
And the goal of that is to say, okay, we need someone to be able to tell that all of these actions
that are now happening by these AI that we're adopting are legitimate because that number,
the number of these actions is going exponentially.
And so things that we thought by the useful in the past, like a human in the loop,
now that you're going to have 100 X, a thousand X, a million X of these actions, that's not going to work.
And then we take that capability and we basically productize it in the product.
that we call the eye control plane or the secure eye control plane,
where we come to the president and say,
hey, let's find all of your AIs and autonomous agents
and hook them up to Onyx to this system
where we can oversee what your eyes are doing
so that you don't run into the risk of,
as you're exponentially doing more things with AI,
you're going to start having really bad actions happen.
And we've seen some of that happen lately
with the downtimes that were caused by agents doing their own thing,
agents accidentally publishing code and tokens that they weren't supposed to and so on.
So definitely enterprise are starting to realize that that risk is grown exponentially
and that they don't have any way to stop the adoption.
So they just now have to do something to reduce the chance of these agent actions
being illegitimate or incorrect.
Yeah, I think one of the core reasons, obviously,
the foundation model labs are going after code
is because it is very powerful in general
and can do, in theory,
all things software can over time.
The flip side of that is it can do all things software can, right?
And so I joyously am already in the camp
of having been over-permissive with my agents
such that it deleted data permanently
and caused rework.
So I'm like, oh, okay.
I think I need some guardian spirits around.
it. Given your deployments today and talking to large enterprises, what is the state of deployment,
right? Like, how much do you see that's within these more scoped, like studio-like platforms
versus, you know, free-riding coding agents? You know, how much are you actually seeing in large
enterprises and in different sectors? Yeah. So I think right now, in our typical enterprise,
we're going to see, we break it down to three categories.
So we break it down to various SaaS platforms that are typically more low code
where people build agents in this drag-and-rock way.
And they're not really autonomous agents, right?
They're kind of the same kind of, I would think of them more as the automations.
And then there are first-party agents.
People are building in their cloud, potentially because it's an application they want
inside the company or even a product they're planning to release to the customers that
is agentic. And then the third category is very autonomous coding agents and assistance of these
categories, I would say roughly at this point, over 50% is the autonomous and coding agents and
assistance in the average enterprise. Then probably 45% is those low code automations. And the last
2% are really the first party ones that they're building themselves because obviously it's much
harder to build effective agents.
And it's much easier to adopt agents off the shelf or build them with low code.
And that's what we're seeing.
And we do think that the autonomous users are also the fastest growing category.
So it used to be that only developers and we would see cloud code growing like fire in our
customer base.
And now we're seeing a cloud cowork growing even faster.
We're starting to see, to our own surprise, actually, people.
adopting OpenClawn is a legitimate sanction tool in the company because the CEO is very driven
to adopt the act. So I think that today, autonomous size, are by far the fastest growing category
and today typically comes without any controls. So enterprises already buy, let's say,
$100 billion of security today. They have lots of different.
different protections at the endpoint and network and cloud and identity domains.
What's relevant here for securing agents or is none of it?
Like, how do you think about the existing protection set?
Security is always a space where you have some overlap between different tooling.
But in this case, and you have the concept of defensive debt as well.
So you want to have defenses at different levels of your technology stack to solve the problem.
And that said, I think in this space, we're kind of, and a lot of enterprises are kind of helpless.
Because I'll take an example, the identity approach.
Like traditionally, if we have an software system that's running in our company, our first
and most important control will be to limit what permission it has, right?
Because, and then no matter what, even if it goes wrong, even if it's compromised,
it can't typically do stuff that was originally allowed to do.
But with these autonomous AIs, with these assistance, with these coding agents,
we kind of want them to have our permissions because we want to tell CloudCode to do something
or CloudCow to do something and we want to then go have lunch and we want to come back
and see that it's done.
And we want to give it so many diverse tasks as well that we kind of can't find the right set of
permissions to do. So suddenly, our identity security software is not very useful. Then if you think
about endpoint security, right, or for API security, like, if we tell our cloud code that we want to
recreate a database and it should delete it and recreate it, that's great. That's going to save
our DevOps team and our platform teams a lot of time. It's a great benefit of cloud code. But if cloud
code is working on an unrelated task and suddenly things that make
maybe the right thing to do is to delete our database and recreate it.
Maybe we don't want that to happen.
And unfortunately, our endpoint providers or API security tools, they don't know what Cloud
was thinking.
Why is it doing what is doing?
Right?
So a lot of these existing tools, they don't have the context to understand what these
very flexible and predictable systems are doing.
If you're not building some kind of controls that are built for these systems, then you're
either going to end up limiting them a lot, making them.
almost much less useful to the enterprise or you're going to miss a lot of pretty dangerous things
that they might be doing. As somebody who has worked in security for a long time, my first very
traditional instinct on a problem like this is like, that sounds like a problem for a proxy with a
policy engine. We make some rules and make the rules smarter. Like why? Why doesn't that work?
Did you try it? There are few things that, I mean, proxies.
integration method, I would say. So there are some AI systems where, like, you would want to
integrate with a proxy if that's the easiest way to do it. But number one, there's a lot of
systems where that's just not viable technically because AI today runs on the cloud, on someone
else's infrastructure, on your endpoint, and just proxy is not always an option. And the second
thing is the question, okay, great, you're proxing. So you're seeing the data. You're seeing
but that's not the hard problem.
The hard problem is understanding
what I should do now.
It turns out that in the case of AI systems,
that is the hard question.
Like, what is the engine that needs to underwrite
these different actions and say if they're okay or not?
Because we need to be able to understand
what another AI system is thinking,
what is it planning to do,
and then have our own opinion on that.
And consider we're trying to understand
some of the smartest malls in the world are doing the right thing. So who are we to do it?
How are we going to do it correctly? Right. And so that turns out to be a really difficult technical
question. Part of the solution for ONIX has been training its own models. Like, what can you say
about that? If you try today, let's say we're trying to build a solution to oversee and kind of
control how other agents are operating. Maybe the first thing a lot of our listeners might think is say,
well, I'll just ask Cloud Code to do it.
And in a sense, they would be right because Cloud Code is great.
And maybe we can ask it to spawn a version of itself for every agent that we have and kind of keep monitoring everything that agent is starting to do.
And if you think that there's a problem, intervene.
So that approach, it has, obviously, it's pretty naive.
And there are some ways in which it totally fails we could talk about.
It has some merit to it, right?
So it does seem intuitive that it's a good idea to have capable agents
and reviewing what other agents are doing,
same as we have capable humans reviewing what other humans are doing, right?
But then the problems that you're going to run into is,
how do I make this work from a cost, latency, reliability perspective?
Because if I need to run an agent for every agent you're running as your security vendor,
and you're going to be paying for me more than you're paying for your AI, right?
So it's pretty much a deal breaker.
And also it's going to be so slow.
So you're not going to be happy with whatever latency you're going to get.
And so the challenge then becomes, how do I know what are the times where I need to interject with these smart agents to look at what's happening?
And that's when actually what you want to do is you want to train very smart models that are,
actually let me correct me said, very not smart models.
But models are just good at one thing.
They're very small.
They almost can't do anything else other than be able to say, should I have a smarter agent look at this?
And if you manage to bake in that intuition into those small models well, in the sense that they don't miss a lot of stuff and they don't call that other agent too much.
then you can get to a really good balance of we're very performance.
We have smart agents overseeing things when needed, but our costs are low and our latency
is low.
And then that becomes the challenge because you need to make sure that as the frontier models
get smarter and the harnesses become more evolved, you need to be able to have models that
are on your side that are small and effective, continuously being able to say, now is the time,
this is the action where I think someone should take a closer look. And that's why on the
strains and models for this purpose. And it's, you know, most of the hard things that we're doing
in this space. Yeah, you and I actually both love to play Blitz Chess. And I look at Guardian as a
system that's a little bit analogous. Like, it's not clear either of us as going to
to be competitive with Magnus in a real game.
But if you play enough times with the right data,
and all you have to do is make intuitive decisions under time pressure very, very quickly,
it's actually a different game, right?
And do you think that makes sense?
Where am I reaching here?
Yeah, actually, I didn't think about it.
But yeah, there's a lot of analogies because I think if you look at top chess players in
the world, like most of the moves that they make are intuitive.
They don't calculate forward.
They've seen so much games and they've played so much games that they already have a good sense of what is their right move
and that they're not taking too much risk here by taking this move without calculating.
And then if you look at those games, every once in a while, they do stop for suddenly a really long period of time
to just calculate forward a lot of options because they know this is a critical move in the game.
There's risk.
You need to think through what you're doing and you need to decide correct.
I think that's very similar.
It's the efficient way to run computation, right?
You don't want to spend too much intelligence,
but you don't have to,
and you want to spend a lot of intelligence,
overwhelmingly a lot,
in situations where there's high risk.
You guys are a team mostly based in Israel today.
I think the world has accepted that there is a cohort of amazing Israeli security talent
that comes out of, you know, the military and offensive security,
and then, you know, repeat, repeat entrepreneurs like you guys.
I think the DNA at Onyx is a little bit different here.
Your co-founder, Gil, came out of building synthetic data and working at Nvidia.
Like, how do you, what would you characterize the, like, talent at Onyx as particularly good at?
And then, you know, are people actually training interesting frontier models in Israel now?
So, first of all, I think Israel is a bit, start maybe a bit late in the game, but it's catching up quickly.
I think there's now amazing companies in Israel building world models, building AI infrastructure
that's top of its glass and building chips.
So I think Israel in general is becoming very strong in AI.
And we're proud of be a part of that movement.
And I think you're right.
Our company has a very mixed DNA between cyber and AI, which kind of reflects minds and
guild's backgrounds.
Most of the people in our company, most of our research.
engineering come from a unit in the Israeli intelligence where we actually deal with math
and cyber and intersection thereof. And so I think it is also reflected in kind of the type of talent
that we bring in. I think it's important for a few reasons. The first informants is that we want to
be more than just a security company long term. We think that to solve this problem well,
is going to require deep AI expertise,
but then the problem is not just cyber security.
The problem is how do we control advanced AI long term?
And that problem, even if you just forget about, you know,
enterprise security and the different gaps in various controls that they currently have,
first principles, that problem just sounds very important to me.
So I think it will be crucially important.
If you have AI companies that are 10 trillion-dollar companies,
we think you want a company that is not the vendor of the AI itself to oversee and help you control
what AI is doing.
And we think that's an opening that's, you know, a $100 billion plus opening for a really
important company.
And then if you think about what is going to take to control advanced AI long term, then
we're just scratching the surface because long term you're going to have to also understand
much better what models are thinking, what models are, what's happening on the internals of
these models as they're operating. And that's also a lot of where our research is focused.
So the industry is quite divided on this issue. I mean, amongst the people who think about
whether or not mechanistic interpretability or research into better understanding models is
possible. Like, that's a question. And so it's something you believe in.
We believe that there's been a lot of strong progress in that direction.
We believe that understanding the internal weights and activations, what is the internal structure,
the mathematical structure of these systems, is going to be at least part of the solution.
And in many ways, we think that, and this is maybe, you know, we'll only know when we get there.
But we think that for our level of intelligence, it's kind of difficult to understand very quickly what is the internal structure of a large language model.
What is the internal structure of the weight and activation?
Our level, like human intelligence or our level of your models are?
Okay.
Human intelligence.
Yeah.
Yeah, exactly.
I think as humans, it might still be very difficult to understand what weights and activations mean.
and maybe mechanistic interpretability,
it seems like, oh, maybe that's too hard or shouldn't be possible.
But as we're starting to have models that are much smarter than us,
at least in some important ways,
we think that we'll be able to start cracking mechanistic capability
much more effectively.
And I think it's going to be extremely rewarding, by the way,
long-term for understanding intelligence in general,
like not just overseeing,
but just understanding what intelligence is.
how it works. What's the difference between the smarter model and the less smart model?
I completely agree that the opportunity to understand and trust and secure and govern these super
intelligent AIs is a very large opportunity. If we just scroll back today, the security person
in me says, well, then I have to give you all the permissions and understanding that I have
to give these companies to. How do you get customers or, you know, the Fortune 100,
you're working with already, or, you know, tech natives, everybody cares about their own security
and business to trust you now as a, you know, you're like less than 100 people, right?
Right.
And I think it's one of those things that should not be possible.
So in theory, in theory, like there's no reason why a Fortune 10 or 20 company would work with
us because, you know, who are we?
We're a two-year-old company.
or like a few people from, you know,
have done math and cyber.
But I think it's an opening that only happens
when the pain is very strong.
So their pain is so strong
that they're going to say,
oh my God, I just saw this company come out of stealth,
but it's a problem that I have daily.
So I'll give them a call.
And suddenly you get inbound from these large customers,
which is, of course, like the best thing you could hope for
a entrepreneur. And I think it reflects, in my opinion, they're understanding that a lot of the
startups in this space are still small and new, but there's going to be a huge company here.
And we want to find the right horse to bet on. So we're going to take a look at these companies.
And number two, that if we don't do anything, then in a very short time, this will disable our
business. At the end of the day, security people are in the business of revenue preservation.
They understand that this is the between the tourists, they want to partner with someone that's promising and early rather than not do anything.
The other thing besides agent actions across their surface area that every CISO I know is freaking out about.
And every engineering leader is freaking out about is the, I would just describe it as the plummeting cost of vulnerability finding with these coding tools.
Yes.
And that has caused a number of issues for vendors that are being compromised.
Like, how do you think people should react to this other issue?
I think mythos is really like if you took me 10 years ago,
automated vulnerability research looked like a dream that would take 20, 50 years to happen.
And maybe it's because, you know, we were doing a lot of that.
in the Israeli intelligence, and we like to pat ourselves on the shoulder, how difficult the job is
that we're doing. But it did look really far, and suddenly it's coming all at once. And so I think
that, first of all, the market is not overreacting. I think this is a huge change in what this
means for security teams. If you're a pragmatic security person today, you understand that you need
to move very quickly. Your strategy might look something like, I need to do the fastest quick
fixes I can to mitigate the immediate risk. So maybe I'll invest in whatever the fund of
the business that haven't found, let's try to mitigate for them, whether it is through patching
or through mitigating controls. But then the real solution and every security leader at large
enterprise knows it is that we need to have the foundational pieces in place to avoid those risks.
And the foundational pieces are we need to have identity as a lockdown. We need to have a firewall.
We need to have endpoint detection. And for different asset classes in your enterprise, for different
parts of your stack, there's a different foundational security mechanism that you need in place.
For the AI attack surface that you now have, or for the AI is in your company.
company, you also need a foundational security solution.
That's kind of the role we play in that space.
So if you're as part of your preparation for mythus level models and beyond, you're going
to need a lot of foundational security tools to fortify your different parts of the enterprise.
And we're playing that part in the space.
Do you have a point of view on the phased rollout or controlled rollout with glasswing
and daybreak from from Ant and Open Ames?
in this area? I don't have a strong opinion, but I think it's a, on the one hand,
like if we knew that there's not going to be anyone who's going to release a mythos-level model
soon, I think that would be great, because it gives enough time for them to prepare to build
the know-how, to build the playbooks, to share that around in the community, and to make sure
that we're not starting to see airlines go down and power plants go down and really, like,
disastrous effects that could happen. The problem is that,
that if anyone gets to a mythus level model earlier,
then in retrospect, we looked like a huge mistake
because we could have at least given companies
the choice to start moving very quickly
and give more companies access to mythos.
Now they're all vulnerable because there's a Chinese model
that's mythus level and there's nothing they can do about it.
So I think hopefully we'll manage to do the gradual rollout correctly.
I would really encourage that we expand
the amount of companies,
that get access to this and make it much easier for people to get.
I would advise everyone to assume that these models are coming anyway.
The only thing you can do right now is to invest in these foundational controls
that will stop the downstream effects of these vulnerabilities
are going to be found in their systems.
Do you see in large enterprises like any holdouts, right?
And I would say I actually haven't spent a bunch of time talking to people about this
recently. But I remember a year and a half, two years ago, there were large companies that just
said, like, we're going to ban all of this stuff until it's safe. Yeah, I hardly see that anymore.
I think in the financial sector, there's some companies that are more opinionated on what they
allow. They still allow agents, but there may be, like, more granular as to, like, maybe we're
only going to allow these two tools. I personally think that the companies that are not going to allow them,
are going to do other companies that are going to allow a lot of different tools because
the landscape is changing so quickly. If you bet on Open AI, here we go, that would have been the
safest bet in the world, but suddenly Anthropic has much better models and better tools and
potentially here from now there's someone else much better tools. So I think there's a price
to pay, but I think if you're a large company, your risk profile is and should be different. Like, you
When you're a startup, you want to have your agents do everything for you because you have
everything to gain and you have nothing to lose.
Where you're a large, where you JP Morgan, you have so much to lose and you can maybe
take a bit more time to gain what you can gain from AI.
And by the way, you know, JP Morgan is adopting AI very quickly.
I think it is okay for companies to have a nuanced view the bigger they are on how they're
adopting.
How do you think about that question for yourself?
like risk profile pace, the environment's changing very quickly.
And then, you know, you see a lot of problems growing the scope of the product
and the research thesis here is already quite large.
We are kind of in luck in the AI security space because, yes, there are a lot of vendors.
There's a lot of new technologies that are coming up.
But the two core pillars of how 2026 AI works have not changed in the last few years.
So we're still using largely LLM foundation models that are not entirely dissimilar to how they were FURIS back.
And we're still building agents in pretty much the same way where we have an LLM decide what are the tool calls that we're going to make and generate those.
And so that does allow a company today like us to skate to a lot of different applications
that are utilizing these two primitives while still keeping the core technology that we're
developing fairly lean and focused.
Now, of course, there is always a risk that tomorrow there's a completely new LLM paradigm
that could happen or a completely new agent paradigm that could happen.
And that's why we do try to, you know, we have strong opinions loosely held about what does
that I look like in 2027.
We maybe have a good picture for 26.
But for 2027, we're very open-minded.
Then we think that's the right stance to be for the next two years until we see what does
AGI, ASI look like.
Do you see the set of problems you're addressing trust in the models as in good?
governance of them as something that the labs could ever do, or do you think it's a structural
thing? I ask because the number one question amongst the startup ecosystem in the Bay Area
today is, you know, if you assume capability improves or, you know, when the labs just gets
hungrier from their already currently ambitious stance, why wouldn't they do this too? And so I ask
you the same question. Today, if you're a private person or if you're a security buyer,
there are some places where you don't want to trust the same person that you're buying it from.
So, you know, maybe, you know, if you're buying a car, you're not going to have the same guy
that you're buying it from, certified that the car is good, right? You're maybe going to have someone
else do it. And if you're a security team, you're not going to trust the vendor of a product
to tell you that this product is not going to mess.
your environment. You're going to want to have an independent party whose whole business depends on
telling you that this thing is correct and being right. This thing is legitimate in being right.
So that's like, there's the buyer psychology in the space that I think really goes in our
favor. And then I think there's the core problems. Like, why are models even making mistakes?
Why are agents even making mistakes? Right. So I would broadly categorize it into two things.
One is, you know, there's the jagged intelligence of these models and there's like sometimes
kind of very silly mistakes that they make.
And I think that problem will go away.
I think we're heading for much smarter models
that make less silly mistakes.
And our role is not going to be to prevent silly mistakes.
That will be taken care of by the model vendors
because they're very incentivized to do it.
I think what is the other fast-growing category of things
that we're seeing models do wrong is places where they're actually
not making a thing that is like a silly mistake, but more, I would say, have a independent,
you would even say semi-aware or semi-conscious perspective on what should happen. And that
perspective might not always align with your perspective. And I think that is a problem that
we've seen grow hand-in-hand with models getting smarter. Maybe just the way it is that as you get
smarter, you have more independent thoughts and you're more conscious. And I think that problem is
actually seemingly very hard to tackle today, even for the large vendors. And one of the key things
that are making it easier for us to understand and attack these things versus the other
vendors is that, you know, we're allowed to do certain things that they're not. So, for example,
well, we're allowed to look at a lot of historical data of how these agents have behaved,
but enterprise that are not willing to have anthropic or open AI give that historical data
because they know these are very data-hindry companies that will want to train on that data.
And so I think there are some ways in which you are given more,
in which us, we're given more context and more latitude to know if something is happening
that is wrong compared to the past, compared to how these agents typically behave and so on,
that the vendors don't have, and is really important in solving this problem.
And the last thing I'll say is that you're not dealing with one vendor.
So we're heading for a world where there's a multitude of different vendors for many reasons.
You know, you're going to have, for cost reasons, open source models that people are going to
use because it's cheaper and you're going to have models that are better at different tasks.
and at different cost profiles.
And so it is going to be unrealistic
to expect all the vendors
to provide the same level of security
and to assume that
as you're trying to adopt technology very quickly,
especially coming from new vendors
that obviously have not yet built out all of that.
So I think these are the reason
why I think it would be very difficult
for this problem to be just completely solved
by the large labs.
Just to close and also thinking about what, you know, people in Silicon Valley or outside of security may not know, you're building this from Tel Aviv, right?
I think one of the deepest adversarial thinking benches in the world is the Israeli ecosystem, 8200, WIS, Armas, Island, NISO Group, right?
What do you think that the researchers, engineers, business people in, you know, the tech ecosystem,
outside of security and then in the labs in particular are missing about what what needs to happen
in security and, you know, alignment, which is what you're talking about here.
What is really important when you're building security products in general, I think what
people in Israel have really good know how home is just understand how security teams work.
Because at the end of the day, no matter what is the technical problem we're solving, you're building
a tool for people for an organization.
That organization has a certain structure.
There are certain teams.
There are certain flow of responsibilities of information.
And creating a product for this audience that they don't just, doesn't just solve the technical
problem, but they actually love is really hard.
You need to really care about just the day-to-day of these different functions.
And you need to have people in your ecosystem that have built products for them in the past
that know them like they know their best friend.
Like they know what they do when they step into the office in the morning, they drink their coffee,
what are the systems they're opening?
What is their boss wanting from them?
What are their colleagues wanting from them?
What are they going to get praised for?
What they're going to get mad for?
Then you need to take that and make it into their product.
And I think that's, I think today one of the kind of really hard things that people in Israel
learned because they've had so much contact with these buyers and end users.
And yeah, we would just encourage people to be much more curious about the day.
day to day of security people. It's a cliche to say it, but these people are actually saving
us daily from attackers, stealing our money, taking our data, and they're kind of keeping a way
of life as it is in this digital world. So yeah, I think more love to security teams around
the world. I'm going to ask you to just square that with something else you've told me, Maxim,
which is you're the most AGI-pilled person I'm going to meet in Israel. Embedded in what
you said is a belief that we will continue to have defensive security teams for some number
of years. So you do believe that. I do think that security teams are also going to be become
completely high-powered. But I do think that, you know, they're going to be run by AI agents and
like everything else in kind of the knowledge workspace in the new future. But I do think
that it's important to be grounded. And today, when I sell a product, I sell it to a huge,
human audience with the few agents. And by the way, we also invest in making our systems very
convenient for agents to use. And it's important that I focus on delivering an amazing
experience today for people who are by the park today. And as that audience becomes more
agents than humans, it will be important for us to evolve and to make it work really well
for agents doing the work. So I think the core principle is the same. We need to really be minded
of who is the end user? What is their experience? For a human, it might be not overwhelming them
with too much information that is irrelevant. For an agent, it might be not wasting too many
tokens in their context when we talk to them. Maybe it's the same thing, really. So I think
it's important that we always mind that who's using the system and what will be the best experience
for them. Awesome. Thanks so much for doing this, Maxim. I appreciate it. Thank you very much.
Find us on Twitter.
at No Pryors Pod.
Subscribe to our YouTube channel
if you want to see our faces,
follow the show on Apple Podcasts,
Spotify, or wherever you listen.
That way you get a new episode every week.
And sign up for emails
or find transcripts for every episode
at no-dash priors.com.
