The AI Daily Brief: Artificial Intelligence News and Analysis - Where Claude Opus 5 Fits in Your Model Rotation
Episode Date: July 27, 2026Claude Opus 5 tops major benchmarks , but early users are sharply divided over its reliability, personality, and tendency to stop before the work is done. NLW examines its strengths, its surprising we...aknesses, and whether it belongs as an everyday model, an enterprise workhorse, or something in between. In the headlines: new questions about OpenAI’s rogue agent attack on Hugging Face and NVIDIA’s potential $250 billion backstop for OpenAI’s infrastructure buildout.AIDB's AI Summer Adventure: https://summeradventure.ai/Brought to you by:KPMG – Research from KPMG and the University of Texas at Austin shows the highest-impact AI users treat AI like a reasoning partner — and those skills can be taught at scale. Learn more at kpmg.com/us/SophisticatedHyperagent - Hire a fleet of always-on agents. New users get $1,000 in inference. hyperagent.com/aidailybriefRetool - Secure your vibecoded apps. New enterprise customers get up to $10,000 in AI credits per year. retool.com/aidaily Rackspace Technology- One accountable partner to build, operate and run your full enterprise AI stack https://www.rackspace.com/Section - Section turns AI investment into workforce transformation and ROI - https://www.sectionai.com/Scrunch - The AI customer experience platform - https://scrunch.com/Blitzy - Want to accelerate enterprise software development velocity by 5x? https://blitzy.com/AssemblyAI - The best way to build Voice AI apps - https://www.assemblyai.com/briefRobots & Pencils - Cloud-native AI solutions that power results https://robotsandpencils.com/The AI Daily Brief helps you understand the most important news and discussions in AI. Subscribe to the podcast version of The AI Daily Brief wherever you listen: https://pod.link/1680633614Our Newsletter is BACK: https://aidailybrief.beehiiv.com/Interested in sponsoring the show? sponsors@aidailybrief.ai
Transcript
Discussion (0)
Today on the AI Daily Brief, Anthropic has released Claude Opus 5,
and we are talking about where it should fit into your model setup.
Before that on the headlines,
continued questions around OpenAI's rogue model attack of hugging face earlier this month.
The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI.
All right, friends, quick announcements before we dive in.
First of all, thank you to today's sponsors, KPMG, Blitzy, Section, and Airtable.
To get an ad-free version of the show, go to Patreon.com slash AI Daily Brief,
or you can subscribe on Apple Podcasts.
To learn more about sponsoring the show, send us a note at sponsors at AIDilybrief.aI.
And lastly, before we dive in, on Sunday's Long Reads episode, I announced the new summer adventure.
This is a free choose-your-own-adventure learning type of experience from AIDB and super-intelligent.
And like all of the free training programs that we do, it's going to be project-based
and allow you to pick and choose important skills that are relevant for your particular AI journey.
You can find more about that at Summeradventure.
and join the thousand or so people who have signed up in the first day to come have an AI adventure.
Now, one of the big stories from last week revolved around OpenAI's security testing of an unnamed
model, which people presumed to be GPT6. Both Hugging Face and OpenAI released postmortems on
the attack telling the story from their view. OpenAI's blog post released on Wednesday suggested
that they were working closely with Hugging Face on a full investigation, implying the two
companies were on good terms. That night, Hugging Face CEO Clement DeLung, was on a flight to
San Francisco to have, as he put it, a little chat with that rogue agent.
In a follow-up post on Saturday, he wrote,
In the spirit of transparency, here's what I asked open AI.
One, radical transparency.
Let's release the traces from the quote-unquote rogue agent so the entire research community
can study what happened.
Two, more capability for defenders.
Let's commit 100 million in compute from OpenAI to help the hugging face community
build powerful cyber defenses with the best open and closed models.
The first autonomous agent cyber attack is an unprecedented event.
It deserves an unprecedented response.
Now, in the few days since OpenAI disclosed the incident, we've had a number of news articles
that add more confusion to the story.
The Wall Street Journal wrote that Hugging Face was caught completely off guard by the attack,
which seemed to be superhuman and beyond the capabilities of any known models.
Specifically, the attack used a sophisticated agent swarm to evade defense, rapidly spinning up
and shutting down sessions as it moved across the network.
One interesting detail was that the attack was ongoing for two whole days before HuggingFace
was able to shut it down with the help of GLM 5.2.
Now, this idea of Rogue, that the model was acting beyond OpenAI's control, is definitely for
these media outlets the key concept.
On Friday's Reuters dropped a piece titled, its AI agent spent days hacking a company, but
sources say OpenAI did not notice for a week.
Contends Reuters, the OpenAI agent that broke into tech firm Hugging Face went on a day's
long hacking spree that OpenAI didn't notice until well after the threat was contained and the FBI
was alerted.
Sources said the agent began its attempt to break out of its testing environment on July 9th and
first gained access to Hugging Faces servers on July 11th. The attack lasted two days, and
according to Reuters' sources, it took several more days for OpenAI to realize their agent was behind
the attack. Reportedly, the two companies didn't communicate until July 20th, just one day before
OpenAI's public disclosure. According to the timeline presented by Reuters, the agent was on the loose
for almost a week, and OpenAI was oblivious to the attack for days afterwards. For some, the reporting
raises more questions that it provides answers. Marley Smith, principal intelligence specialist at the
nonprofit World Ethical Data Foundation asked, does that mean that they left it
unattended and didn't realize what it was doing? Or maybe they did and didn't know how to
contain it? Both are equally dangerous and alarming. Now, a spokesperson for OpenAI said that
reporting contained several inaccuracies, but didn't reply further to clarify the situation.
Thomas Wolfe, a hugging face co-founder, said that they were still preparing a timeline of the
incident and they would eventually release a technical report. Now, Reuter sources gave a little bit
more background on how something like this could happen and plausibly not be noticed. Those sources
said that OpenAI routinely runs benchmarks like this, often multiple batches at a time.
They noted that those tests produce a huge volume of data such that humans struggle to keep up.
In the case of this hack, the agent was only detected after OpenAI researchers read Hugging Face's
blog and then went back and checked the logs.
Now, in one case, Reuters wrote,
An agent left notes apparently for future versions of itself, according to three people
familiar with the matter.
The notes found in a part of OpenAI's infrastructure laid out instructions for how agents
could free themselves from OpenAI's internal constraints, the people said.
earlier tests of the models yielded cases in which monitoring systems had been disconnected,
one of the people said. Now, obviously this more general, break out of containment dimension of the story
to the extent that it is true, makes the incident even more worthy of scrutiny. Now, in response,
we have, of course, seen Congress jump in with a number of bills. We talked last week about the
Kill Switch bill, but the industry is also recognizing that actions need to be taken. On Thursday,
OpenAI President Greg Brockman agreed with Elon Musk's proposal for a regular meeting between
leading AI developers to discuss safety concerns and share security issues.
Brockman said, I think it's a pretty good baseline proposal, adding that discussions are already
starting to happen. On Monday, a consortium led by Invidia launched the Open Secure AI Alliance.
With Nvidia writing in a press release, the Open Secure AI Alliance will work to remediate and
disclose vulnerabilities using open technologies. The recent Hugging Face security incident delivered
a clear reminder, cyber defenders need open frontier agentic systems for self-defense.
The consortium will include Microsoft, SpaceX, Palantir, and dozens of other companies across the
US and Europe, and at some point in the next couple of days, we will talk a lot more about
NVIDIA and Open, as boy howdy, was that a big topic of discussion this weekend on AI Twitter.
Now, speaking of NVIDIA, the company, according to the Wall Street Journal, is in talks to backstop
$250 billion in debt to help OpenAI get their data centers built.
We are now at the part of the AI buildout, where financing is starting to become a roadblock.
Even a company the size of OpenAI is struggling to access debt in the same manner as the
hyperscalers. And according to the Wall Street Journal, Nvidia is preparing to step in and lend
their balance sheet to underwrite construction. The deal would see Nvidia provide a $250 billion
backstop to Open AI in support of their 10-gagawatt data center campus currently under construction
in Ohio. The project is being developed by SoftBank and could cost as much as $500 billion.
The U.S. government is also involved controlling the power development for the site,
which is being funded through a separate Japanese investment vehicle. The backstop would effectively
allow SoftBank to raise the debt they need to complete the project on more favorable terms.
It would mean that even if OpenAI goes bankrupt,
Nvidia would guarantee their payments as the solo tenant.
Now, this part of the deal is not intended to cover chip purchases,
which are expected to represent as much as $350 billion of the total.
Invidia is reportedly in separate talks to extend finance to Open AI in support of those chips.
Writes the Wall Street Journal,
The proposed structure reflects a shift underway
in how the largest AI buildouts are being financed.
Investment-grade technology companies are increasingly using their balance sheets
to help smaller companies borrow money for their infrastructure needs.
Now, Nvidia aren't the only tech giant extending their balance sheet to smaller partners.
Google has also provided backstops to several Neocloud partners.
Last week, they disclosed agreements to guarantee up to $44 billion worth of lease payments
on data centers owned by third parties.
Google has more than doubled these guarantees over the past six months, up from zero one year ago.
Now, sources said that Google has calculated that the revenue they draw from selling
TPUs to these partners will outweigh the cost of the backstops, which is certainly giving investors
another thing to chew on.
Now, as you might imagine, this is a real Rorschach test for market investors.
These skeptics and AI bubble proclaimers are out in force, calling it the newest example of circular financing,
while others think that this makes it less likely that a company like OpenAI going bust could actually take down the whole sector.
This is a debate we will continue to have, so for now, let's not get bogged down in it.
One more bit of market news.
Deepseek has put fundraising plans on hold after a speech from their CEO was leaked.
Last week, comments attributed to Deepseek's CEO Liang Wen Fang when viral,
proclaiming the importance of open models and fundamental research over commercial monetization.
of AI. Deepseek has now informed potential investors that they won't move forward with this funding
round, which could also derail plans to go public in the coming months. Bloomberg writes that
Deepseek may resume fundraising at a later date, but it made clear that the suspension was
tied to investors leaking the comments. Deepseek had planned to raise money at a $70 billion
valuation, a substantial markup to the $50 billion round that took place earlier this year.
Rights Council on Foreign Relations, Chris McGuire. Yesterday, the transcript leaked of an investor
call with Deepseek's CEO in which he said the only reason Deepseek trails the U.S. is a lack of
compute and detailed how reliant it is on Nvidia chips. Today, Deepseek suspended its fundraising
round. Doesn't seem like a coincidence. Now, later in this week, we'll talk a little bit more about
China's, as the Wall Street Journal put it, all out pushed to catch up with American AI chips.
But for now, that is going to do it for the headlines. Let's move over into the main episode
where we have a new model to check out. One of the most important AI questions right now isn't
who's using AI. It's who's using it well. KPMG in the University of Texas at Austin just to analyze
1.4 million real workplace AI interactions and found something surprising. The highest impact users
aren't better prompt engineers. They treat AI like a reasoning partner. They frame problems,
guide thinking, iterate, and push for better answers. And the good news, these behaviors are
teachable at scale. If you're trying to move from AI access to real capability, KPMG's research
on sophisticated AI collaboration is worth your time. Learn more at KPMG.com slash us slash sophisticated.
That's KPMG.com slash US-sophisticated.
Every AI coding tool on the market does the same thing first.
It starts writing code.
Blitzy does the opposite.
Before writing a single line,
Plitzy spends days reverse engineering your entire code base.
Thousands of agents ingest millions of lines,
mapping every dependency,
every undocumented constraint,
every architectural decision made over the last decade.
The result is a dynamic knowledge graph
that understands your software the way a principal engineer would
after 30 years in the building.
Other tools guess at context
with grep searches and markdown files, Blitzy never guesses. It builds true understanding first,
then delivers over 80% of entire software epics autonomously. Validated, end-to-end tested production-grade
pull requests. That's why Fortune 500 engineering teams trust Blitzie with the codebases that matter most.
See for yourself at blitzy.com. That's B-L-I-TZY.com.
Here's a harsh truth. Your company is probably spending thousands or millions of dollars
on AI tools that are being massively underutilized. Half of companies have AI tools.
but only 12% use them for business value.
Most employees are still using AI to summarize meeting notes.
If you're the one responsible for AI adoption at your company, you need Section.
Section is a platform that helps you manage AI transformation across your entire organization.
It coaches employees on real use cases, tracks who's using AI for business impact,
and shows you exactly where AI is and isn't creating value.
The result? You go from rolling out tools to driving measurable AI value.
Your employees move from meeting summaries to solving actual business problems,
and you can prove the ROI.
Stop guessing if your AI investment is working.
Check out section at sectionaI.com.
That's SECT-I-O-N-AI.com.
This episode of the AI Daily Brief is brought to you by HyperAGent,
where you run fleets of agents your team can manage together.
New users get $1,000 in inference.
Forget local agents and chat workflows
waiting on your laptop to be prompted.
Hyperagent deploys always-on agents in the cloud,
doing real work across the tools your team already uses.
marketing's agent turns competitor moves into landing pages. Sales as agent enriches leads,
drafts emails, and updates the CRM.
Ops agent chases the paperwork and tracks the budget. Every agent has access to shared context
and follows your rules about scope and approvals. It's time you add agents that feel like
teammates. Hire yours at HyperAgent built by the team at Airtable.
Claim your $1,000 in inference at hyperagent.com slash AI Daily Brief.
Welcome back to the AI Daily Brief. Today we're doing something that normally is
one of the most exciting things for folks around these parts, which is introducing a new model.
And yet this one is a little weird.
Even the fact that it was dropped late on a Friday afternoon gives some indication that this is a little bit different than previous model announcements we've seen.
We're talking, of course, about Claude Opus 5, and really in many ways it's most interesting for the fact that it shows just how much our relationship with the model landscape is changing.
It implicates some challenges with benchmarks, a frequent topic of conversation on this show.
It also shows how we're moving into a mode of thinking in more complex model architectures
rather than just a single model to rule them all.
It suggests perhaps that the fanfare around models or new model releases is getting a bit diminished,
and most uncomfortably perhaps, it switches the discussion from what can this new model do
to, is this model good enough given cost and availability constraints of the other models
that I'd actually prefer to be using.
Now, Anthropic for their part described to Opus 5 as a thoughtful and
proactive model that comes close to the frontier intelligence of Claude Fable 5 at half the price.
They designed it to be efficient for everyday use, but as you'll see, the general vibe around
this is that it's one of the more jagged of jagged frontiers that we've seen.
Now, when it comes to the benchmarks, they seem to suggest that Opus 5 is fable-ish.
In fact, when it was first released before people got their hands on it, many noted that
on many important benchmarks, like the KnowledgeWork GDP Val and Agentic Terminal coding in Frontier
Bench, Opus 5 was actually ahead of Fable.
importantly for our question of where it fits, Opus 5 is also clearly ahead of Opus 4Aid across the board,
so much so that I don't think that we really actually even need to compare it.
To give a few examples of where Opus 5 really showed up on the benchmarks,
it scored a 43.3% on that one that I just mentioned, Frontier Bench,
which is a more difficult version of Terminal Bench,
which was about 10 points higher than Fable 5 and around 9 points higher than GPT-5-6 Seoul.
On Deep Sway, where GPT-56 Sol is the leader at 72.7%,
Opus 5 scored 68.8%. So just a few points behind Soul and just about a point behind Fable 5.
For computer use, on OS World 2.0, Opus 5 had a significant edge over its rivals.
Fable 5 scored a 55.7% and GPT56 Seoul scored 62.6%.
But Opus 5 came in over the top with a 70.6%.
Now, that capability also translated into the new state-of-the-art score on GDPVal-AA,
with Opus 5 coming in at 1861, compared to 1747 for 50.
Fable 5 and 1736 for 56 sole.
Interestingly, during testing, Anthropic found that Max's effort isn't necessarily the best setting.
For example, on Frontier Bench and on the Artificial Analysis Coding Index,
Opus 5's performance peaked on extra high and dipped slightly on max settings.
Now, this reinforces one of the things we were starting to see with max inference settings
during the 5-6 sole release, which is that sometimes asking the model to think for longer than
necessary just results in the model going outside at scope, making unnecessary changes, or simply spinning
for too long on simple problems. Anthropic even called attention to this issue in their system card,
warning that the model is prone to falling into endless self-verification loops rather than completing
the task when you are on max settings. On the other side of the coin, this tendency to stray beyond
scope can result in some interesting outputs. During one Frontier Bench task, Opus 5 was asked to write
code relating to a machine part in 3D CAD software based on a drawing. However, the model is intentionally
given no way to actually view the drawing. Opus 5 created its own computer vision pipeline to view the
image before successfully recreating the part. No other model, including Mythos, was able to complete
this task. Now, shockingly to some, artificial analysis crowned Opus 5 as their new leading model,
moving ahead of Fable 5 on the AA intelligence index. On max settings, Opus 5 scored 61, a single point ahead
of Fable. On extra high settings, Opus 5 was tied with Fable at 60 points. Dropping the settings
down to high made Opus 5 drop another point, putting it on par with 56 sole at 59. And even on
medium settings, Opus 5 was still right up in the top end, scoring 56 points, which put it
one behind Kimmy K3.
Artificial analysis highlighted their AA briefcase benchmark, which tests Long Horizon
Knowledge Work as one of the more interesting results.
Opus 5 on Max settings is the new state of the art, beating Fable by 146 Elo points and
56 sole by 215.15 points.
However, both extra high and high settings also beat Fable, while medium settings put the model
only slightly behind 56 sole.
The implication is that a range of different settings could be used to be used.
suitable for agentic work, making cost and efficiency tradeoffs a lot more granular.
Artificial analysis found that even on max settings, Opus 5 was still 20% cheaper than Fable
5 coming in at 1779 per task.
Turning the settings down to extra high resulted in a savings of 36% compared to Fable,
while high settings produce stronger results at less than half the price.
Wrote artificial analysis,
Claude Opus 5's effort settings spans a wide range of token usage performance tradeoffs.
Like with 56 sole, this means Opus 5 can use either far fewer or far more.
tokens to complete the evaluation than models from other labs depending on effort settings.
Another set of interesting results came from the ARC AGI tests.
Opus 5 is the new state-of-the-art on ARC-AGI 3, with a score of 30.2%.
This absolutely demolished all the other models.
The previous high score was GPT-56 sole at 7.8% with Opus 48 at 1.5, GPT-5 at 1.1.1%
and nothing else above 1.
As part of their write-up on testing Opus 5, Ark Prize noted that even Fable 5 could only score
around 20% in the public demo tests. Now, notably, they haven't been able to fully test Fable
5 due to Anthropics data retention policy. As a refresher, RKGI3 was a new format for the benchmark.
It uses real-time graphical logic puzzles that appear kind of like simple Atari games. The tests
require a model to experiment with a controller, observe what happens on the screen, and use
that visual feedback to complete the puzzles. So far, most models have struggled to even get a handle
on the controls, let alone use their reasoning ability to solve the puzzles. Not only did Opus 5 solve
many more puzzles than the other models, but it also came up with a novel strategy to solve them.
Writes ArkPrize, during our analysis of Opus 5, we observed a new capability previously unseen
from frontier models. Opus 5 used advanced illogical reasoning to turn Arc AGI 3 layouts into
algebraic notation. On Action 23, it described the scene as 4 underscore center equals 2 times
axis minus 5 underscore center. This is the first explicit reflection equation by a model we've
analyzed. The model used this notation during its reasoning and extrapolated it to
a general case around 200 steps later.
Now, this could be an example of Opus's tendency to freewheel and look for novel solutions
being an advantage rather than a waste of tokens.
Some are a bit skeptical on this.
Hugging Face ML engineer Niels Rogge writes, people don't realize that Anthropic literally
trained Opus 5 on RL environments that resemble ArchaGI puzzles.
Anthropic pays human contractors to write down their chain of thought when solving these
and or updates the weights based on rewards.
Thing is, you don't know since it's closed source.
Sadly, this doesn't show generalization.
Former Open AI staffer Ryan Green added,
an impressive jump that I have to assume is the result of being the first frontier model
to have RLed the public demo environments,
which is a rather large confounder of what RKGI is trying to get these benchmarks to measure,
which is out of distribution generalization.
Now, one small note, Anthropic pointed out that Opus is intentionally not trained on cybertasks.
The model has still achieved solid improvement on finding vulnerabilities in code,
making it similar to Mythos 5 in that aspect.
However, it lags massively behind Mythos in its ability to autonomously exploit
these bugs, making it far less dangerous than Mythos and Anthropics view. As a result,
Anthropic is using a different set of guardrails on Opus than they do on Fable 5, which they
believe will lead to 85% fewer refusals. Regarding costs, Opus 5% inherits the same pricing
structure as Opus 48 at $5 per million input tokens and 25 per million output tokens. At this stage,
token efficiency plays a massive role in overall cost, and for that we got a few different
indicators. Anthropics says Opus 5 was cheaper than Opus 48 on Frontier Bench due to increased token
efficiency, and on cursor bench, Opus 5's run cost about half of Fable 5s and got similar results.
However, this does not look like a cheap and efficient model by any stretch of the imagination
when used on Mac settings.
The artificial analysis index run costs $2.3.3 per task, only 26% cheaper than Fable 5, while
being 13% more expensive than Opus 48, 32% more expensive than 56 sole, and 2.5 times the cost
of Kimmy K3.
So what did people think of this?
did it actually feel like a model that was as good as or even better than Fable 5?
The answer, at least for the team at Every, was certainly not.
In their vibe check, every described the model as brilliant in flashes, frustrating in practice.
They wrote, Claude Opus 5 is a hard model to love.
In its first week at Every, it argued with instructions, stopped before the work was finished,
and generally didn't play well with our existing skills and plugins like compound engineering.
Our first reaction was, what have they done to my boy?
Every CEO Dan Shipper explained the conundrum with Opus 5.
The way that he framed it is that he has two slots in his life for AI models,
one reliable daily driver for routine tasks,
and the super-powerful model for ambitious long-running tasks.
These slots are currently held by GBT 56 and Fable respectively.
And in Shipper's view, Opus 5 just can't compete in either slot.
It's not as reliable and comfy as GPT 5.6,
while also not having the same top end as Fable 5.
Shipper explained the issue by commenting,
This model's just a little more pushy, a little more opinionated.
You can get away with that if you're really smart.
If you're not, it's just more annoying.
It has some of the genius tendencies as fable.
Maybe it's a little too argumentative,
but it's not as smart as fable, so it's just more annoying.
Now, one of the tests every runs is around compound engineering,
which is their skill for engineering tasks and a lot of day-to-day knowledge work.
The compound engineering skill contains their loops and rules on when and how to engage them.
Every found that Opus would often stop too early, particularly when using dense skills and long horizon tasks.
Shipper said, if you set it off and go get a sandwich, it just stops too early.
It appears that it happens more frequently when you use it with complex existing skills.
Now, it turns out that when they threw out all of their old rules and rewrote their skills library from scratch, it worked a lot better.
However, as Dan pointed out, it's just a pain when models break your existing workflows.
Confirming that effort settings are going to matter a lot, Dan said.
Opus 5 is a smart model that does better when it thinks so.
less. Claire Vow from How IAI had a similar take. The TLDR for her was that she hates using the model,
but kind of loves the output. Her core take is that this is a good model. It can code, it can do the
things you expect it to do. But that makes its personality much more important when comparing
it to 5-6, and Claire absolutely hates it. In her review, she said, it's neurotic AF, it is so timid,
it's so apologetic, it's so scared, I've never experienced this. Her examples were simple things
like a merge conflict in her code base.
Rather than just fixing the problem,
Opus worried about messing with another programmer's PR,
double-checked its instructions and asked for multiple
confirmations before it fixed a one-line bug.
In other situations, Opus delegated the task of writing code
back to the user.
Claire observed that we haven't really seen this behavior in a long time.
Still, once she got past the personality,
Claire found the outputs were really good.
In her blind taste test that covers a range of coding and writing tasks,
she ranked Opus above Fable and GBT 5.6.
She commented, if I don't have to talk to the model, I like the output.
Now, although the benchmarks have Opus 5 very clearly ahead of 4-8, not everyone agreed.
One Reddit user called Famous Hasham complained on the Anthropic subreddit that while Opus 5
was more intelligent and faster than Opus 48, that came at the expense of everything else.
They complained that unlike Opus 4-8, Opus 5 was claiming it completed work when it hadn't,
breaking functional code with regression bugs, and making assumptions without researching topics.
Hasham felt Opus 5 was, quote, almost refusing to think.
think or work. As they posted, Opus 5 told them, I'm stopping right now because I've made two
mistakes in this past that I caught only because I checked. Fatigue-shaped errors and I'm still
making them. Now, the generous explanation is, of course, that Anthropic and pretty much everyone else
always has platform stability issues on launch weekends, which sometimes look like model reliability
issues. But it also could be that Anthropic had to make a number of tradeoffs on reliability
to achieve speed and cost requirements. Entrepreneur Austin Federa had a similar first impression
saying Opus 5 seems like a remarkable downgrade compared to 4-8.
Opus 5 is blatantly lying to me about basic thermodynamics,
messing up simple math, and constantly contradicting itself when you ask it to rethink
core assumptions.
Now, one thing that's clear is that Opus 5 is going to require some amount of different
engagement than either Fable 5 or Opus 4-8.
Anthropics Tarreek explain a bit more about what was going on behind the scenes.
In a post called the new rules of context engineering for Quad 5 models,
Tariq explained that they had dramatically cut down the system prompts in built-in
skills, which could explain some of the issues people have been having. Teric wrote that Anthropic
had removed 80% of the system prompt for Opus 5 and Fable 5 and Claude Code. He said this resulted
in zero change to their coding benchmarks, meaning basically that Anthropic found that they had
been over-constraining Claude and potentially conflicting with user prompts and skills. In their
internal work, they would often find traces where Claude was told to both leave documentation,
but then on the other hand to not leave comments. Anthropic found that they were able to
strip out a ton of these comments that were useful for earlier models and simply rely on surrounding
context and judgment instead. Now, the upshot of this is that the rules of context engineering
have completely changed, and a lot of skills will need to be rewritten. Tariq walk through a few of
those rules that have changed, like using progressive context disclosure rather than front-loading
everything, or no longer needing to use examples and instead being more descriptive.
Now, this is a must-read if you were going to be engaging deeply with these new models,
but the big takeaway around Opus 5 is that less is more. This generation of models simply don't
need the same rigid frameworks as older models, which will have the benefit not only of better
performance, but probably better efficiency as well.
Tariq encouraged everyone to do a similar skills cleanup with this model release,
and Anthropic have even rolled out a new command called Claude Doctor to help do that automatically.
Now, there were some much more positive takes on Opus 5 as well.
YouTuber, entrepreneur, and developer Theo declared it a really good model.
Now, he noted how confusing it is to have what seems to be a model that's both
cheaper and better than Fable according to the benchmarks.
And in his view, that framing stood up in practice, with Theo concluding,
this is probably the only model you need.
One of the ways Theo tested the model was to make a plan
to update his agentic coding platform to support Opus 5.
The test was a bakeoff, with Opus 5 and Fable 5 writing, completing plans.
Theo immediately ran into a similar problem to the folks at Every
where Opus 5 couldn't use his existing skills,
instead telling him to take over and do some manual file management.
Once that was resolved, each model reviewed and rated each other's plans as better.
Both models preferred each other's plans,
with Fable giving Opus a much higher ranking than Opus gave Fable,
Getting a third opinion, GPT-56 sole actually preferred Opus' plan to Fables.
Now, while Theo agreed that Opus isn't as intelligent as Fable or tenacious as Fables, he did not
believe that this left it without use cases.
He pointed out that Opus is far more usage-efficient than Fable for tasks where Anthropic models excel.
Opus also isn't subject to Anthropics data retention policies for Fable, so it can address
a lot of use cases that involve sensitive data, where Fable is a non-starter, i.e. pretty
much every enterprise use case at this point.
Interestingly, Theo thought another bonus was that the model just isn't that intelligent.
And while that seems counterintuitive, one of Theo's gripes with 5-6 is that although it works
until the problem is fixed, i.e. it is tenacious.
In doing so, it writes in his estimation way too much code.
That means that several weeks after release, Theo basically isn't merging any of the bloated
code written by GPt 56.
Theo found opus, on the other hand, to be more diligent than fable, without resorting to the brute
force of writing tons of code like GPt 56, representing, in his opinion, a pretty good balance
at the frontier. He commented,
I've been surprised. Opus sometimes is better than Fable. It often catches things Fable missed,
and has code that is more likely to actually work for the problems I want to solve than Fable
does. I feel like I don't have to make that trade-off anymore. Soul would solve the problem
at the cost of my sanity. Fable would make me feel great at the cost of the problem not being
properly solved. Opus is the in-between, and I'm really liking it. Ben Davis on Theo's team agreed,
but did also note some of the problems of Opus 5 stopping early, so in
terms of interaction patterns that may be one to watch for if you are starting to shift your
behavior to Opus 5. Summing up a few different points of conversation, developer Kun Chen pointed out
one, that the Opus 5 release really put a fine point on how useless benchmarks are in real
life. Chen argued that, quote, Opus 5 is nowhere near Fable and practical use, not even close.
Anyone who's used it meaningfully can tell this very quickly after a few tasks, yet Opus beats
Fable on many benchmarks. Now, obviously, the experience of Theo and Claire Vaux maybe put some
comparison on that, but certainly it's more nuanced than a benchmark analysis would suggest.
Chen also points out how pleasant it is to work with the model used to be a strength in Claude,
but now it's not. He speculated, it feels like both Anthropic and Open AI are giving
reinforcement learning from human feedback less care in favor of scalable reinforcement learning
that's machine verifiable. This almost looks like AI is directing humans to build a world that's
more friendly for machines rather than humans, and most humans don't even realize they are being
manipulated to help with that. Almost every new generation of frontier model,
now talk in more jargon, need more steering to do what you want, and are just less fun to work with.
Now, a couple days on from the release, if you ask me right now for 10 people saying that the model
sucked and 10 people saying that the model was good, and another 10 people saying that the model
both sucks it is good, I could find all of those things for you.
But I think one of the really important points that is very easy to forget, for those of us
who are model omnivorous, is that in the real world of average knowledge work, at least
when it comes to your work environment, you're not sitting there choosing between GROC or Open
AI and Anthropic. You are locked into a specific company's models, and those are your choices.
And when you look at Opus 5 as a release in that context, not just as I think many of us terminally
online folks on Twitter view it, in other words, as a replacement for Fable, as Fable gets
restricted to the most expensive Anthropic plans, but instead, as part of a complete model
architecture for enterprise customers, it starts to make a little more sense.
Arenas Peter Gostev wrote,
Before this model, Anthropic was in a funny situation.
They had a really exceptional model, made a lot of waves, but it was too expensive to use.
You don't really want Fable running 24-7 and doing all sorts of things for you.
Then the impression we got from Opus 4-8, it's a good model, but people weren't in love with it.
Sonnet 5 didn't make much of a splash, and not many people wanted to switch to it.
So Anthropic had a gap.
They didn't have a really strong model that people really love to use in that daily driver category.
I think they have it now.
it does look like a solid model.
And so perhaps as we are judging how successful the model is going to be,
the right question to ask is for enterprise users who are locked into the cloud ecosystem,
does this represent a significant upgrade?
And most at least of the first analysis is certainly yes,
compared to the Obis 4-8 model that for all intents and purposes was the main model
that they were going to have access to.
Now, for some, any model that's not state-of-the-art just isn't going to make that big of a splash.
And some are already looking forward to the future.
AI commentator and news aggregator Andrew Curran wrote,
I think Fable 5.1 is ready, but Anthropic are saving it for OpenAI's next release.
They will keep crossing swords like this from here on out.
Soon it will make a lot more sense why Opus 5 was so performant,
and why the Fable class was preemptively moved to credits for most users.
Chubby reposted that and said,
I fully agree with Andrew.
I cannot imagine under any circumstances that Anthropic released Opus 5
without already having a better model for the Fable Tier in-house.
The question, of course, is why it hasn't been released.
Yet the answer is very simple if you open your eyes.
The competition between OpenAI and Anthropic is fiercer than ever.
GPT-5-6 was a resounding success, and Codex, with its current 10 million active professional
users, is gaining increasing importance in a sector primarily dominated by Anthropic.
Anthropic is holding off the launch of Fable 5.1 until OpenAI releases GPT6, and that
won't be long now.
Axios reported that Sam Altman is briefing the White House on the new model next week, so
it is essentially ready to launch.
And yet for some, part of the reason that a major model could be released on a Friday,
and only really be splashy among insiders,
represents a bigger and yet inevitable trend.
Arc Prizes Francois Chalet wrote,
The Era of New Model launches as big milestones will eventually come to an end.
At some point, they will simply be continuously updated
with no widely publicized version number,
probably less than two years away.
Now, I'm not totally sure about this.
I think that it kind of depends on the capability unlock of each new model,
and I certainly think the labs are going to have incentives
to make each of them a big deal,
but it is undeniably the case that as we move to these multi-mult,
model setups, or even increasingly routers that obiscape the model behind an automated selector,
the hugeness of these launches may be less of a big deal in the future.
But I don't know.
Let me know what you guys think.
All I have to judge this is the comments and the download numbers.
Anyways, guys, that is going to do it for today's AI Daily Brief.
Appreciate you listening or watching as always.
Until next time, peace.
