The AI Daily Brief: Artificial Intelligence News and Analysis - Where Claude Opus 5 Fits in Your Model Rotation

Episode Date: July 27, 2026

Claude Opus 5 tops major benchmarks , but early users are sharply divided over its reliability, personality, and tendency to stop before the work is done. NLW examines its strengths, its surprising we...aknesses, and whether it belongs as an everyday model, an enterprise workhorse, or something in between. In the headlines: new questions about OpenAI’s rogue agent attack on Hugging Face and NVIDIA’s potential $250 billion backstop for OpenAI’s infrastructure buildout.AIDB's AI Summer Adventure: ⁠https://summeradventure.ai/Brought to you by:KPMG – Research from KPMG and the University of Texas at Austin shows the highest-impact AI users treat AI like a reasoning partner — and those skills can be taught at scale. Learn more at ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠kpmg.com/us/Sophisticated⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Hyperagent - Hire a fleet of always-on agents. New users get $1,000 in inference. ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠hyperagent.com/aidailybrief⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Retool - Secure your vibecoded apps. New enterprise customers get up to $10,000 in AI credits per year. ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠retool.com/aidaily ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Rackspace Technology- One accountable partner to build, operate and run your full enterprise AI stack ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.rackspace.com/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Section - Section turns AI investment into workforce transformation and ROI - ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.sectionai.com/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Scrunch - The AI customer experience platform - ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://scrunch.com/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Blitzy - Want to accelerate enterprise software development velocity by 5x? ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://blitzy.com/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠AssemblyAI - The best way to build Voice AI apps - ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://www.assemblyai.com/brief⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Robots & Pencils - Cloud-native AI solutions that power results ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://robotsandpencils.com/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠The AI Daily Brief helps you understand the most important news and discussions in AI. Subscribe to the podcast version of The AI Daily Brief wherever you listen: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://pod.link/1680633614⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Our Newsletter is BACK: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠https://aidailybrief.beehiiv.com/⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠Interested in sponsoring the show? sponsors@aidailybrief.ai

Transcript
Discussion (0)
Starting point is 00:00:00 Today on the AI Daily Brief, Anthropic has released Claude Opus 5, and we are talking about where it should fit into your model setup. Before that on the headlines, continued questions around OpenAI's rogue model attack of hugging face earlier this month. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. All right, friends, quick announcements before we dive in. First of all, thank you to today's sponsors, KPMG, Blitzy, Section, and Airtable. To get an ad-free version of the show, go to Patreon.com slash AI Daily Brief,
Starting point is 00:00:34 or you can subscribe on Apple Podcasts. To learn more about sponsoring the show, send us a note at sponsors at AIDilybrief.aI. And lastly, before we dive in, on Sunday's Long Reads episode, I announced the new summer adventure. This is a free choose-your-own-adventure learning type of experience from AIDB and super-intelligent. And like all of the free training programs that we do, it's going to be project-based and allow you to pick and choose important skills that are relevant for your particular AI journey. You can find more about that at Summeradventure. and join the thousand or so people who have signed up in the first day to come have an AI adventure.
Starting point is 00:01:07 Now, one of the big stories from last week revolved around OpenAI's security testing of an unnamed model, which people presumed to be GPT6. Both Hugging Face and OpenAI released postmortems on the attack telling the story from their view. OpenAI's blog post released on Wednesday suggested that they were working closely with Hugging Face on a full investigation, implying the two companies were on good terms. That night, Hugging Face CEO Clement DeLung, was on a flight to San Francisco to have, as he put it, a little chat with that rogue agent. In a follow-up post on Saturday, he wrote, In the spirit of transparency, here's what I asked open AI.
Starting point is 00:01:40 One, radical transparency. Let's release the traces from the quote-unquote rogue agent so the entire research community can study what happened. Two, more capability for defenders. Let's commit 100 million in compute from OpenAI to help the hugging face community build powerful cyber defenses with the best open and closed models. The first autonomous agent cyber attack is an unprecedented event. It deserves an unprecedented response.
Starting point is 00:02:01 Now, in the few days since OpenAI disclosed the incident, we've had a number of news articles that add more confusion to the story. The Wall Street Journal wrote that Hugging Face was caught completely off guard by the attack, which seemed to be superhuman and beyond the capabilities of any known models. Specifically, the attack used a sophisticated agent swarm to evade defense, rapidly spinning up and shutting down sessions as it moved across the network. One interesting detail was that the attack was ongoing for two whole days before HuggingFace was able to shut it down with the help of GLM 5.2.
Starting point is 00:02:28 Now, this idea of Rogue, that the model was acting beyond OpenAI's control, is definitely for these media outlets the key concept. On Friday's Reuters dropped a piece titled, its AI agent spent days hacking a company, but sources say OpenAI did not notice for a week. Contends Reuters, the OpenAI agent that broke into tech firm Hugging Face went on a day's long hacking spree that OpenAI didn't notice until well after the threat was contained and the FBI was alerted. Sources said the agent began its attempt to break out of its testing environment on July 9th and
Starting point is 00:02:58 first gained access to Hugging Faces servers on July 11th. The attack lasted two days, and according to Reuters' sources, it took several more days for OpenAI to realize their agent was behind the attack. Reportedly, the two companies didn't communicate until July 20th, just one day before OpenAI's public disclosure. According to the timeline presented by Reuters, the agent was on the loose for almost a week, and OpenAI was oblivious to the attack for days afterwards. For some, the reporting raises more questions that it provides answers. Marley Smith, principal intelligence specialist at the nonprofit World Ethical Data Foundation asked, does that mean that they left it unattended and didn't realize what it was doing? Or maybe they did and didn't know how to
Starting point is 00:03:32 contain it? Both are equally dangerous and alarming. Now, a spokesperson for OpenAI said that reporting contained several inaccuracies, but didn't reply further to clarify the situation. Thomas Wolfe, a hugging face co-founder, said that they were still preparing a timeline of the incident and they would eventually release a technical report. Now, Reuter sources gave a little bit more background on how something like this could happen and plausibly not be noticed. Those sources said that OpenAI routinely runs benchmarks like this, often multiple batches at a time. They noted that those tests produce a huge volume of data such that humans struggle to keep up. In the case of this hack, the agent was only detected after OpenAI researchers read Hugging Face's
Starting point is 00:04:07 blog and then went back and checked the logs. Now, in one case, Reuters wrote, An agent left notes apparently for future versions of itself, according to three people familiar with the matter. The notes found in a part of OpenAI's infrastructure laid out instructions for how agents could free themselves from OpenAI's internal constraints, the people said. earlier tests of the models yielded cases in which monitoring systems had been disconnected, one of the people said. Now, obviously this more general, break out of containment dimension of the story
Starting point is 00:04:31 to the extent that it is true, makes the incident even more worthy of scrutiny. Now, in response, we have, of course, seen Congress jump in with a number of bills. We talked last week about the Kill Switch bill, but the industry is also recognizing that actions need to be taken. On Thursday, OpenAI President Greg Brockman agreed with Elon Musk's proposal for a regular meeting between leading AI developers to discuss safety concerns and share security issues. Brockman said, I think it's a pretty good baseline proposal, adding that discussions are already starting to happen. On Monday, a consortium led by Invidia launched the Open Secure AI Alliance. With Nvidia writing in a press release, the Open Secure AI Alliance will work to remediate and
Starting point is 00:05:06 disclose vulnerabilities using open technologies. The recent Hugging Face security incident delivered a clear reminder, cyber defenders need open frontier agentic systems for self-defense. The consortium will include Microsoft, SpaceX, Palantir, and dozens of other companies across the US and Europe, and at some point in the next couple of days, we will talk a lot more about NVIDIA and Open, as boy howdy, was that a big topic of discussion this weekend on AI Twitter. Now, speaking of NVIDIA, the company, according to the Wall Street Journal, is in talks to backstop $250 billion in debt to help OpenAI get their data centers built. We are now at the part of the AI buildout, where financing is starting to become a roadblock.
Starting point is 00:05:42 Even a company the size of OpenAI is struggling to access debt in the same manner as the hyperscalers. And according to the Wall Street Journal, Nvidia is preparing to step in and lend their balance sheet to underwrite construction. The deal would see Nvidia provide a $250 billion backstop to Open AI in support of their 10-gagawatt data center campus currently under construction in Ohio. The project is being developed by SoftBank and could cost as much as $500 billion. The U.S. government is also involved controlling the power development for the site, which is being funded through a separate Japanese investment vehicle. The backstop would effectively allow SoftBank to raise the debt they need to complete the project on more favorable terms.
Starting point is 00:06:16 It would mean that even if OpenAI goes bankrupt, Nvidia would guarantee their payments as the solo tenant. Now, this part of the deal is not intended to cover chip purchases, which are expected to represent as much as $350 billion of the total. Invidia is reportedly in separate talks to extend finance to Open AI in support of those chips. Writes the Wall Street Journal, The proposed structure reflects a shift underway in how the largest AI buildouts are being financed.
Starting point is 00:06:39 Investment-grade technology companies are increasingly using their balance sheets to help smaller companies borrow money for their infrastructure needs. Now, Nvidia aren't the only tech giant extending their balance sheet to smaller partners. Google has also provided backstops to several Neocloud partners. Last week, they disclosed agreements to guarantee up to $44 billion worth of lease payments on data centers owned by third parties. Google has more than doubled these guarantees over the past six months, up from zero one year ago. Now, sources said that Google has calculated that the revenue they draw from selling
Starting point is 00:07:06 TPUs to these partners will outweigh the cost of the backstops, which is certainly giving investors another thing to chew on. Now, as you might imagine, this is a real Rorschach test for market investors. These skeptics and AI bubble proclaimers are out in force, calling it the newest example of circular financing, while others think that this makes it less likely that a company like OpenAI going bust could actually take down the whole sector. This is a debate we will continue to have, so for now, let's not get bogged down in it. One more bit of market news. Deepseek has put fundraising plans on hold after a speech from their CEO was leaked.
Starting point is 00:07:37 Last week, comments attributed to Deepseek's CEO Liang Wen Fang when viral, proclaiming the importance of open models and fundamental research over commercial monetization. of AI. Deepseek has now informed potential investors that they won't move forward with this funding round, which could also derail plans to go public in the coming months. Bloomberg writes that Deepseek may resume fundraising at a later date, but it made clear that the suspension was tied to investors leaking the comments. Deepseek had planned to raise money at a $70 billion valuation, a substantial markup to the $50 billion round that took place earlier this year. Rights Council on Foreign Relations, Chris McGuire. Yesterday, the transcript leaked of an investor
Starting point is 00:08:10 call with Deepseek's CEO in which he said the only reason Deepseek trails the U.S. is a lack of compute and detailed how reliant it is on Nvidia chips. Today, Deepseek suspended its fundraising round. Doesn't seem like a coincidence. Now, later in this week, we'll talk a little bit more about China's, as the Wall Street Journal put it, all out pushed to catch up with American AI chips. But for now, that is going to do it for the headlines. Let's move over into the main episode where we have a new model to check out. One of the most important AI questions right now isn't who's using AI. It's who's using it well. KPMG in the University of Texas at Austin just to analyze 1.4 million real workplace AI interactions and found something surprising. The highest impact users
Starting point is 00:08:55 aren't better prompt engineers. They treat AI like a reasoning partner. They frame problems, guide thinking, iterate, and push for better answers. And the good news, these behaviors are teachable at scale. If you're trying to move from AI access to real capability, KPMG's research on sophisticated AI collaboration is worth your time. Learn more at KPMG.com slash us slash sophisticated. That's KPMG.com slash US-sophisticated. Every AI coding tool on the market does the same thing first. It starts writing code. Blitzy does the opposite.
Starting point is 00:09:28 Before writing a single line, Plitzy spends days reverse engineering your entire code base. Thousands of agents ingest millions of lines, mapping every dependency, every undocumented constraint, every architectural decision made over the last decade. The result is a dynamic knowledge graph that understands your software the way a principal engineer would
Starting point is 00:09:44 after 30 years in the building. Other tools guess at context with grep searches and markdown files, Blitzy never guesses. It builds true understanding first, then delivers over 80% of entire software epics autonomously. Validated, end-to-end tested production-grade pull requests. That's why Fortune 500 engineering teams trust Blitzie with the codebases that matter most. See for yourself at blitzy.com. That's B-L-I-TZY.com. Here's a harsh truth. Your company is probably spending thousands or millions of dollars on AI tools that are being massively underutilized. Half of companies have AI tools.
Starting point is 00:10:17 but only 12% use them for business value. Most employees are still using AI to summarize meeting notes. If you're the one responsible for AI adoption at your company, you need Section. Section is a platform that helps you manage AI transformation across your entire organization. It coaches employees on real use cases, tracks who's using AI for business impact, and shows you exactly where AI is and isn't creating value. The result? You go from rolling out tools to driving measurable AI value. Your employees move from meeting summaries to solving actual business problems,
Starting point is 00:10:46 and you can prove the ROI. Stop guessing if your AI investment is working. Check out section at sectionaI.com. That's SECT-I-O-N-AI.com. This episode of the AI Daily Brief is brought to you by HyperAGent, where you run fleets of agents your team can manage together. New users get $1,000 in inference. Forget local agents and chat workflows
Starting point is 00:11:08 waiting on your laptop to be prompted. Hyperagent deploys always-on agents in the cloud, doing real work across the tools your team already uses. marketing's agent turns competitor moves into landing pages. Sales as agent enriches leads, drafts emails, and updates the CRM. Ops agent chases the paperwork and tracks the budget. Every agent has access to shared context and follows your rules about scope and approvals. It's time you add agents that feel like teammates. Hire yours at HyperAgent built by the team at Airtable.
Starting point is 00:11:34 Claim your $1,000 in inference at hyperagent.com slash AI Daily Brief. Welcome back to the AI Daily Brief. Today we're doing something that normally is one of the most exciting things for folks around these parts, which is introducing a new model. And yet this one is a little weird. Even the fact that it was dropped late on a Friday afternoon gives some indication that this is a little bit different than previous model announcements we've seen. We're talking, of course, about Claude Opus 5, and really in many ways it's most interesting for the fact that it shows just how much our relationship with the model landscape is changing. It implicates some challenges with benchmarks, a frequent topic of conversation on this show. It also shows how we're moving into a mode of thinking in more complex model architectures
Starting point is 00:12:19 rather than just a single model to rule them all. It suggests perhaps that the fanfare around models or new model releases is getting a bit diminished, and most uncomfortably perhaps, it switches the discussion from what can this new model do to, is this model good enough given cost and availability constraints of the other models that I'd actually prefer to be using. Now, Anthropic for their part described to Opus 5 as a thoughtful and proactive model that comes close to the frontier intelligence of Claude Fable 5 at half the price. They designed it to be efficient for everyday use, but as you'll see, the general vibe around
Starting point is 00:12:54 this is that it's one of the more jagged of jagged frontiers that we've seen. Now, when it comes to the benchmarks, they seem to suggest that Opus 5 is fable-ish. In fact, when it was first released before people got their hands on it, many noted that on many important benchmarks, like the KnowledgeWork GDP Val and Agentic Terminal coding in Frontier Bench, Opus 5 was actually ahead of Fable. importantly for our question of where it fits, Opus 5 is also clearly ahead of Opus 4Aid across the board, so much so that I don't think that we really actually even need to compare it. To give a few examples of where Opus 5 really showed up on the benchmarks,
Starting point is 00:13:27 it scored a 43.3% on that one that I just mentioned, Frontier Bench, which is a more difficult version of Terminal Bench, which was about 10 points higher than Fable 5 and around 9 points higher than GPT-5-6 Seoul. On Deep Sway, where GPT-56 Sol is the leader at 72.7%, Opus 5 scored 68.8%. So just a few points behind Soul and just about a point behind Fable 5. For computer use, on OS World 2.0, Opus 5 had a significant edge over its rivals. Fable 5 scored a 55.7% and GPT56 Seoul scored 62.6%. But Opus 5 came in over the top with a 70.6%.
Starting point is 00:14:03 Now, that capability also translated into the new state-of-the-art score on GDPVal-AA, with Opus 5 coming in at 1861, compared to 1747 for 50. Fable 5 and 1736 for 56 sole. Interestingly, during testing, Anthropic found that Max's effort isn't necessarily the best setting. For example, on Frontier Bench and on the Artificial Analysis Coding Index, Opus 5's performance peaked on extra high and dipped slightly on max settings. Now, this reinforces one of the things we were starting to see with max inference settings during the 5-6 sole release, which is that sometimes asking the model to think for longer than
Starting point is 00:14:37 necessary just results in the model going outside at scope, making unnecessary changes, or simply spinning for too long on simple problems. Anthropic even called attention to this issue in their system card, warning that the model is prone to falling into endless self-verification loops rather than completing the task when you are on max settings. On the other side of the coin, this tendency to stray beyond scope can result in some interesting outputs. During one Frontier Bench task, Opus 5 was asked to write code relating to a machine part in 3D CAD software based on a drawing. However, the model is intentionally given no way to actually view the drawing. Opus 5 created its own computer vision pipeline to view the image before successfully recreating the part. No other model, including Mythos, was able to complete
Starting point is 00:15:16 this task. Now, shockingly to some, artificial analysis crowned Opus 5 as their new leading model, moving ahead of Fable 5 on the AA intelligence index. On max settings, Opus 5 scored 61, a single point ahead of Fable. On extra high settings, Opus 5 was tied with Fable at 60 points. Dropping the settings down to high made Opus 5 drop another point, putting it on par with 56 sole at 59. And even on medium settings, Opus 5 was still right up in the top end, scoring 56 points, which put it one behind Kimmy K3. Artificial analysis highlighted their AA briefcase benchmark, which tests Long Horizon Knowledge Work as one of the more interesting results.
Starting point is 00:15:52 Opus 5 on Max settings is the new state of the art, beating Fable by 146 Elo points and 56 sole by 215.15 points. However, both extra high and high settings also beat Fable, while medium settings put the model only slightly behind 56 sole. The implication is that a range of different settings could be used to be used. suitable for agentic work, making cost and efficiency tradeoffs a lot more granular. Artificial analysis found that even on max settings, Opus 5 was still 20% cheaper than Fable 5 coming in at 1779 per task.
Starting point is 00:16:20 Turning the settings down to extra high resulted in a savings of 36% compared to Fable, while high settings produce stronger results at less than half the price. Wrote artificial analysis, Claude Opus 5's effort settings spans a wide range of token usage performance tradeoffs. Like with 56 sole, this means Opus 5 can use either far fewer or far more. tokens to complete the evaluation than models from other labs depending on effort settings. Another set of interesting results came from the ARC AGI tests. Opus 5 is the new state-of-the-art on ARC-AGI 3, with a score of 30.2%.
Starting point is 00:16:51 This absolutely demolished all the other models. The previous high score was GPT-56 sole at 7.8% with Opus 48 at 1.5, GPT-5 at 1.1.1% and nothing else above 1. As part of their write-up on testing Opus 5, Ark Prize noted that even Fable 5 could only score around 20% in the public demo tests. Now, notably, they haven't been able to fully test Fable 5 due to Anthropics data retention policy. As a refresher, RKGI3 was a new format for the benchmark. It uses real-time graphical logic puzzles that appear kind of like simple Atari games. The tests require a model to experiment with a controller, observe what happens on the screen, and use
Starting point is 00:17:28 that visual feedback to complete the puzzles. So far, most models have struggled to even get a handle on the controls, let alone use their reasoning ability to solve the puzzles. Not only did Opus 5 solve many more puzzles than the other models, but it also came up with a novel strategy to solve them. Writes ArkPrize, during our analysis of Opus 5, we observed a new capability previously unseen from frontier models. Opus 5 used advanced illogical reasoning to turn Arc AGI 3 layouts into algebraic notation. On Action 23, it described the scene as 4 underscore center equals 2 times axis minus 5 underscore center. This is the first explicit reflection equation by a model we've analyzed. The model used this notation during its reasoning and extrapolated it to
Starting point is 00:18:08 a general case around 200 steps later. Now, this could be an example of Opus's tendency to freewheel and look for novel solutions being an advantage rather than a waste of tokens. Some are a bit skeptical on this. Hugging Face ML engineer Niels Rogge writes, people don't realize that Anthropic literally trained Opus 5 on RL environments that resemble ArchaGI puzzles. Anthropic pays human contractors to write down their chain of thought when solving these and or updates the weights based on rewards.
Starting point is 00:18:33 Thing is, you don't know since it's closed source. Sadly, this doesn't show generalization. Former Open AI staffer Ryan Green added, an impressive jump that I have to assume is the result of being the first frontier model to have RLed the public demo environments, which is a rather large confounder of what RKGI is trying to get these benchmarks to measure, which is out of distribution generalization. Now, one small note, Anthropic pointed out that Opus is intentionally not trained on cybertasks.
Starting point is 00:18:57 The model has still achieved solid improvement on finding vulnerabilities in code, making it similar to Mythos 5 in that aspect. However, it lags massively behind Mythos in its ability to autonomously exploit these bugs, making it far less dangerous than Mythos and Anthropics view. As a result, Anthropic is using a different set of guardrails on Opus than they do on Fable 5, which they believe will lead to 85% fewer refusals. Regarding costs, Opus 5% inherits the same pricing structure as Opus 48 at $5 per million input tokens and 25 per million output tokens. At this stage, token efficiency plays a massive role in overall cost, and for that we got a few different
Starting point is 00:19:32 indicators. Anthropics says Opus 5 was cheaper than Opus 48 on Frontier Bench due to increased token efficiency, and on cursor bench, Opus 5's run cost about half of Fable 5s and got similar results. However, this does not look like a cheap and efficient model by any stretch of the imagination when used on Mac settings. The artificial analysis index run costs $2.3.3 per task, only 26% cheaper than Fable 5, while being 13% more expensive than Opus 48, 32% more expensive than 56 sole, and 2.5 times the cost of Kimmy K3. So what did people think of this?
Starting point is 00:20:04 did it actually feel like a model that was as good as or even better than Fable 5? The answer, at least for the team at Every, was certainly not. In their vibe check, every described the model as brilliant in flashes, frustrating in practice. They wrote, Claude Opus 5 is a hard model to love. In its first week at Every, it argued with instructions, stopped before the work was finished, and generally didn't play well with our existing skills and plugins like compound engineering. Our first reaction was, what have they done to my boy? Every CEO Dan Shipper explained the conundrum with Opus 5.
Starting point is 00:20:37 The way that he framed it is that he has two slots in his life for AI models, one reliable daily driver for routine tasks, and the super-powerful model for ambitious long-running tasks. These slots are currently held by GBT 56 and Fable respectively. And in Shipper's view, Opus 5 just can't compete in either slot. It's not as reliable and comfy as GPT 5.6, while also not having the same top end as Fable 5. Shipper explained the issue by commenting,
Starting point is 00:21:04 This model's just a little more pushy, a little more opinionated. You can get away with that if you're really smart. If you're not, it's just more annoying. It has some of the genius tendencies as fable. Maybe it's a little too argumentative, but it's not as smart as fable, so it's just more annoying. Now, one of the tests every runs is around compound engineering, which is their skill for engineering tasks and a lot of day-to-day knowledge work.
Starting point is 00:21:26 The compound engineering skill contains their loops and rules on when and how to engage them. Every found that Opus would often stop too early, particularly when using dense skills and long horizon tasks. Shipper said, if you set it off and go get a sandwich, it just stops too early. It appears that it happens more frequently when you use it with complex existing skills. Now, it turns out that when they threw out all of their old rules and rewrote their skills library from scratch, it worked a lot better. However, as Dan pointed out, it's just a pain when models break your existing workflows. Confirming that effort settings are going to matter a lot, Dan said. Opus 5 is a smart model that does better when it thinks so.
Starting point is 00:22:00 less. Claire Vow from How IAI had a similar take. The TLDR for her was that she hates using the model, but kind of loves the output. Her core take is that this is a good model. It can code, it can do the things you expect it to do. But that makes its personality much more important when comparing it to 5-6, and Claire absolutely hates it. In her review, she said, it's neurotic AF, it is so timid, it's so apologetic, it's so scared, I've never experienced this. Her examples were simple things like a merge conflict in her code base. Rather than just fixing the problem, Opus worried about messing with another programmer's PR,
Starting point is 00:22:34 double-checked its instructions and asked for multiple confirmations before it fixed a one-line bug. In other situations, Opus delegated the task of writing code back to the user. Claire observed that we haven't really seen this behavior in a long time. Still, once she got past the personality, Claire found the outputs were really good. In her blind taste test that covers a range of coding and writing tasks,
Starting point is 00:22:53 she ranked Opus above Fable and GBT 5.6. She commented, if I don't have to talk to the model, I like the output. Now, although the benchmarks have Opus 5 very clearly ahead of 4-8, not everyone agreed. One Reddit user called Famous Hasham complained on the Anthropic subreddit that while Opus 5 was more intelligent and faster than Opus 48, that came at the expense of everything else. They complained that unlike Opus 4-8, Opus 5 was claiming it completed work when it hadn't, breaking functional code with regression bugs, and making assumptions without researching topics. Hasham felt Opus 5 was, quote, almost refusing to think.
Starting point is 00:23:26 think or work. As they posted, Opus 5 told them, I'm stopping right now because I've made two mistakes in this past that I caught only because I checked. Fatigue-shaped errors and I'm still making them. Now, the generous explanation is, of course, that Anthropic and pretty much everyone else always has platform stability issues on launch weekends, which sometimes look like model reliability issues. But it also could be that Anthropic had to make a number of tradeoffs on reliability to achieve speed and cost requirements. Entrepreneur Austin Federa had a similar first impression saying Opus 5 seems like a remarkable downgrade compared to 4-8. Opus 5 is blatantly lying to me about basic thermodynamics,
Starting point is 00:24:00 messing up simple math, and constantly contradicting itself when you ask it to rethink core assumptions. Now, one thing that's clear is that Opus 5 is going to require some amount of different engagement than either Fable 5 or Opus 4-8. Anthropics Tarreek explain a bit more about what was going on behind the scenes. In a post called the new rules of context engineering for Quad 5 models, Tariq explained that they had dramatically cut down the system prompts in built-in skills, which could explain some of the issues people have been having. Teric wrote that Anthropic
Starting point is 00:24:27 had removed 80% of the system prompt for Opus 5 and Fable 5 and Claude Code. He said this resulted in zero change to their coding benchmarks, meaning basically that Anthropic found that they had been over-constraining Claude and potentially conflicting with user prompts and skills. In their internal work, they would often find traces where Claude was told to both leave documentation, but then on the other hand to not leave comments. Anthropic found that they were able to strip out a ton of these comments that were useful for earlier models and simply rely on surrounding context and judgment instead. Now, the upshot of this is that the rules of context engineering have completely changed, and a lot of skills will need to be rewritten. Tariq walk through a few of
Starting point is 00:25:02 those rules that have changed, like using progressive context disclosure rather than front-loading everything, or no longer needing to use examples and instead being more descriptive. Now, this is a must-read if you were going to be engaging deeply with these new models, but the big takeaway around Opus 5 is that less is more. This generation of models simply don't need the same rigid frameworks as older models, which will have the benefit not only of better performance, but probably better efficiency as well. Tariq encouraged everyone to do a similar skills cleanup with this model release, and Anthropic have even rolled out a new command called Claude Doctor to help do that automatically.
Starting point is 00:25:34 Now, there were some much more positive takes on Opus 5 as well. YouTuber, entrepreneur, and developer Theo declared it a really good model. Now, he noted how confusing it is to have what seems to be a model that's both cheaper and better than Fable according to the benchmarks. And in his view, that framing stood up in practice, with Theo concluding, this is probably the only model you need. One of the ways Theo tested the model was to make a plan to update his agentic coding platform to support Opus 5.
Starting point is 00:25:59 The test was a bakeoff, with Opus 5 and Fable 5 writing, completing plans. Theo immediately ran into a similar problem to the folks at Every where Opus 5 couldn't use his existing skills, instead telling him to take over and do some manual file management. Once that was resolved, each model reviewed and rated each other's plans as better. Both models preferred each other's plans, with Fable giving Opus a much higher ranking than Opus gave Fable, Getting a third opinion, GPT-56 sole actually preferred Opus' plan to Fables.
Starting point is 00:26:25 Now, while Theo agreed that Opus isn't as intelligent as Fable or tenacious as Fables, he did not believe that this left it without use cases. He pointed out that Opus is far more usage-efficient than Fable for tasks where Anthropic models excel. Opus also isn't subject to Anthropics data retention policies for Fable, so it can address a lot of use cases that involve sensitive data, where Fable is a non-starter, i.e. pretty much every enterprise use case at this point. Interestingly, Theo thought another bonus was that the model just isn't that intelligent. And while that seems counterintuitive, one of Theo's gripes with 5-6 is that although it works
Starting point is 00:26:59 until the problem is fixed, i.e. it is tenacious. In doing so, it writes in his estimation way too much code. That means that several weeks after release, Theo basically isn't merging any of the bloated code written by GPt 56. Theo found opus, on the other hand, to be more diligent than fable, without resorting to the brute force of writing tons of code like GPt 56, representing, in his opinion, a pretty good balance at the frontier. He commented, I've been surprised. Opus sometimes is better than Fable. It often catches things Fable missed,
Starting point is 00:27:28 and has code that is more likely to actually work for the problems I want to solve than Fable does. I feel like I don't have to make that trade-off anymore. Soul would solve the problem at the cost of my sanity. Fable would make me feel great at the cost of the problem not being properly solved. Opus is the in-between, and I'm really liking it. Ben Davis on Theo's team agreed, but did also note some of the problems of Opus 5 stopping early, so in terms of interaction patterns that may be one to watch for if you are starting to shift your behavior to Opus 5. Summing up a few different points of conversation, developer Kun Chen pointed out one, that the Opus 5 release really put a fine point on how useless benchmarks are in real
Starting point is 00:28:04 life. Chen argued that, quote, Opus 5 is nowhere near Fable and practical use, not even close. Anyone who's used it meaningfully can tell this very quickly after a few tasks, yet Opus beats Fable on many benchmarks. Now, obviously, the experience of Theo and Claire Vaux maybe put some comparison on that, but certainly it's more nuanced than a benchmark analysis would suggest. Chen also points out how pleasant it is to work with the model used to be a strength in Claude, but now it's not. He speculated, it feels like both Anthropic and Open AI are giving reinforcement learning from human feedback less care in favor of scalable reinforcement learning that's machine verifiable. This almost looks like AI is directing humans to build a world that's
Starting point is 00:28:41 more friendly for machines rather than humans, and most humans don't even realize they are being manipulated to help with that. Almost every new generation of frontier model, now talk in more jargon, need more steering to do what you want, and are just less fun to work with. Now, a couple days on from the release, if you ask me right now for 10 people saying that the model sucked and 10 people saying that the model was good, and another 10 people saying that the model both sucks it is good, I could find all of those things for you. But I think one of the really important points that is very easy to forget, for those of us who are model omnivorous, is that in the real world of average knowledge work, at least
Starting point is 00:29:16 when it comes to your work environment, you're not sitting there choosing between GROC or Open AI and Anthropic. You are locked into a specific company's models, and those are your choices. And when you look at Opus 5 as a release in that context, not just as I think many of us terminally online folks on Twitter view it, in other words, as a replacement for Fable, as Fable gets restricted to the most expensive Anthropic plans, but instead, as part of a complete model architecture for enterprise customers, it starts to make a little more sense. Arenas Peter Gostev wrote, Before this model, Anthropic was in a funny situation.
Starting point is 00:29:51 They had a really exceptional model, made a lot of waves, but it was too expensive to use. You don't really want Fable running 24-7 and doing all sorts of things for you. Then the impression we got from Opus 4-8, it's a good model, but people weren't in love with it. Sonnet 5 didn't make much of a splash, and not many people wanted to switch to it. So Anthropic had a gap. They didn't have a really strong model that people really love to use in that daily driver category. I think they have it now. it does look like a solid model.
Starting point is 00:30:19 And so perhaps as we are judging how successful the model is going to be, the right question to ask is for enterprise users who are locked into the cloud ecosystem, does this represent a significant upgrade? And most at least of the first analysis is certainly yes, compared to the Obis 4-8 model that for all intents and purposes was the main model that they were going to have access to. Now, for some, any model that's not state-of-the-art just isn't going to make that big of a splash. And some are already looking forward to the future.
Starting point is 00:30:45 AI commentator and news aggregator Andrew Curran wrote, I think Fable 5.1 is ready, but Anthropic are saving it for OpenAI's next release. They will keep crossing swords like this from here on out. Soon it will make a lot more sense why Opus 5 was so performant, and why the Fable class was preemptively moved to credits for most users. Chubby reposted that and said, I fully agree with Andrew. I cannot imagine under any circumstances that Anthropic released Opus 5
Starting point is 00:31:09 without already having a better model for the Fable Tier in-house. The question, of course, is why it hasn't been released. Yet the answer is very simple if you open your eyes. The competition between OpenAI and Anthropic is fiercer than ever. GPT-5-6 was a resounding success, and Codex, with its current 10 million active professional users, is gaining increasing importance in a sector primarily dominated by Anthropic. Anthropic is holding off the launch of Fable 5.1 until OpenAI releases GPT6, and that won't be long now.
Starting point is 00:31:35 Axios reported that Sam Altman is briefing the White House on the new model next week, so it is essentially ready to launch. And yet for some, part of the reason that a major model could be released on a Friday, and only really be splashy among insiders, represents a bigger and yet inevitable trend. Arc Prizes Francois Chalet wrote, The Era of New Model launches as big milestones will eventually come to an end. At some point, they will simply be continuously updated
Starting point is 00:31:58 with no widely publicized version number, probably less than two years away. Now, I'm not totally sure about this. I think that it kind of depends on the capability unlock of each new model, and I certainly think the labs are going to have incentives to make each of them a big deal, but it is undeniably the case that as we move to these multi-mult, model setups, or even increasingly routers that obiscape the model behind an automated selector,
Starting point is 00:32:20 the hugeness of these launches may be less of a big deal in the future. But I don't know. Let me know what you guys think. All I have to judge this is the comments and the download numbers. Anyways, guys, that is going to do it for today's AI Daily Brief. Appreciate you listening or watching as always. Until next time, peace.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.