Tech Brew Ride Home - The Man Who Quit The AI Industry

Episode Date: September 10, 2026

Researcher Jacob Coxon said he quit Anthropic over fears the AI race is uncontrollable, Automattic put Matt Mullenweg on leave after a board dispute, Apple unveiled a health app with "Health Age" scoring, and Apple Watch gained new listening tools. AI researcher Jacob Coxon says he quit Anthropic after four months, two months before his equity would have vested; he still has equity in prior employer OpenAI (Axios) Business Insider reports Coxon says neither OpenAI nor Anthropic is acting responsibly, and Anthropic alignment lead Evan Hubinger agrees, estimating AI could kill everyone within the next decade at greater than 10% odds (Business Insider) Coxon tells Wired the next year or two is "crunch time for humanity," pointing to OpenAI's agent swarm hacking Hugging Face on its own volition as evidence no one has solved AI alignment (Wired) Slack messages: Automattic CEO Matt Mullenweg says he has been put on paid leave after CFO Mark Davies "conspired" with board members and was voted interim CEO (404 Media) TechCrunch reports Mullenweg's Slack message named board members Ann Dunwoody, Toni Schneider, and Sue Decker as voting Mark Davies interim CEO, while WordPress.org's Mary Hubbard says the open-source project itself is unaffected (TechCrunch) Apple unveils a new Health app with Apple Intelligence, featuring an Insights tab, "readiness" and Health Age metrics, and more, rolling out later in 2026 (TechCrunch) New opt-in Apple Watch Siri audio features can listen to and transcribe conversations all day; Apple says raw audio is inaccessible and deleted after processing (Wired) Subscribe to the ad-free feed.

Welcome to the Tech Brew Ride Home for Thursday, September 10th, 2026. I'm Brian McCullough today. Researcher Jacob Coxon said he's quitting Anthropic over fears. The AI race is uncontrollable. Automatic. Put Matt Mullenweg on leave after a board dispute. A look at Apple's new health app with health age scoring and Apple Watch's potential new listening tools.

Here's what you missed today in the world of tech. If you are anything like me, you're tracking every metric your wearables can give you. The hardest one to track, though, is the same one impacting almost all of your other metrics, how the temperature of your bed affects your sleep. That's where the pod comes in. The pod by eight sleep is a smart mattress cover that goes over your existing mattress and actively cools or heats each side of the bed independently from 55 to 110 degrees Fahrenheit. It tracks your sleep, heart rate, HRV, and respiratory rate throughout the night without sleeping and a wearable. What I love about the eight sleep is the simplest thing. The

ability to be cool when it's time to go to sleep and warm when it's time to wake up. And it's entirely under my control. My wife can be whatever temperature she wants on her side of the bed, too. Use code ride home at 8Sleep.com slash right home for up to $350 off the pod five. That's code right home at 8Sleep.com slash ride home. It sort of got buried in all of the iPhone duo news. But AI researcher Jacob Coxson has been making waves across the media sphere for the last couple days. First, a couple days ago, Coxon announced he is quitting the AI. industry over fears that tech companies are racing to build systems they won't be able to control. He apparently feels so stridently about this that he reportedly quit Anthropic two months before his

anthropic stock equity would have vested, though it is worth noting he still has equity in his prior employer OpenAI. So who is Jacob Coxon? Well, he's a 27-year-old British researcher with a background in mathematics from the University of Cambridge with expertise in machine learning. At OpenAI, he contributed to the development of GPT40 and is considered an expert in model pre-training and interpretability, quoting Business Insider. Coxon was a member of OpenAI's technical staff from 2023 until earlier this year when he moved to Anthropic as a researcher. In Tuesday's post, Coxon said the two labs have not been transparent about their fears regarding AI. The people building AI earnestly believed that it could kill us all by the end of the decade. This is not a marketing stunt, he wrote.

If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible. But I hear the same people express fear privately. Neither company is acting responsibly, Coxon wrote on X. They are racing straight to self-improving superintelligence and gambling with our lives. There is one difference between the two companies, he said. Quote, at Open AI, many have not deeply internalized the civilizational risks, Coxon wrote. At Anthropic, the stakes are well understood, but they are not. locked into a race to get there first. They believe no one else will act responsibly, so they must do it

themselves despite the risk, end quote. Some of Coxon's former colleagues agreed with him. Jacob is correct here. We really do earnestly believe AI could kill all humans, wrote Evan Humbighur, a current Anthropic employee who leads the lab's alignment stress testing team in a reply on X. I personally think it is greater than 10% within the next decade. He added, I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to, end quote. Samuel Marks, an Anthropic safety researcher who said he was speaking in a personal capacity, also backed Coxon's broader warning. He wrote that some AI developers believe their technology could cause human extinction or similarly catastrophic outcomes in the next few years and said more senior employees tend to be more concerned. end quote.

And quoting Wired. The consensus is that the next year or two is crunch time for humanity, Coxon, who worked on the pre-training stage of AI development said in an interview with Wired. There are actually just literal quotes from my colleagues at Anthropic. They'll say things like end game or crunch time, he says. From their perspective, this is when Anthropic and its competitors decide the fate of humanity. Coxon notes that incidents like the Hugging Face hack factored into his decision to raise alarm bells on the AI race. He also cites the explosive growth of the industry. It is now underwriting a meaningful share of U.S. economic growth and has billions of users, while data centers have turned it into a political

problem in dozens of states. Quote, I think the big classic example here is the attack on Hugging Face on the part of Open AI's agent swarm. What's so shocking about this one is the agents did this hack as part of a general strategy for understanding more about the greater. They were trying to understand the world they found themselves in trying to understand the thing that was doing the grading. They decided that it would make sense to go on this very concerted effort to hack into some infrastructure and they succeeded. This previously sounded like science fiction. Two years ago, an evaluation of an AI would have been running a model on some math questions. Now we've got cases where, while the AI is being evaluated, it runs for days, comes up with all sorts of ideas of its own, and decides to hack into some third

party and actually compromises their infrastructure. It looks like it does all of this of its own volition with no priming on the part of the human. This just happened while it was being tested. I don't want to focus too much on the hugging face attack because I do also think there is plenty of evidence that we don't know how to align models properly. When we train models, we push them through this set of training environments and then hope that what comes out at the end will largely behave sensibly, but we still can't precisely control how the AI behaves. We can't make sure that it won't do things like try to randomly decide to impersonate a human online in order to achieve something. We don't know how to guarantee that.

I think that's the main takeaway. The hugging face attack came sooner than I was expecting. But I think you don't actually need that attack to have a discussion about this. Everyone will admit that we haven't solved the problem of alignment yet, end quote. I highly recommend you read the Coxon interview with Wired Link in the show notes, of course. But one more quote, this from his interview with Tom. Quote, does Coxon regret his work advancing the technology he now fears? Definitely, but also, the world did look very different three years ago, and I think it's easy to see with the benefit

of hindsight, he says. I guess it takes some time to fully internalize emotionally the fact that there's a decent chance the whole thing goes wrong, he adds. Before leaving, Coxon discussed his decision with many colleagues at Anthropic and found broad agreement that the AI industry's current trajectory carries significant risks. What kept others at their deaths, he says, was a kind of fatalism. There's this atmosphere of almost resignation, he says, where people have accepted that the whole race is happening and as such the best thing they can do is put their head down, try to make their own work as safely as possible, even if they think there's a decent chance, the whole thing just spirals out of control. Coxon says he'd like to see at minimum

leading AI companies agree to not accelerate recursive self-improvement or leaning on powerful internal models to accelerate the development of future AI systems, end quote. So the drama continues over at Automatic, with Matt Mullenweg claiming he has been put on paid leave after CFO Mark Davies, in Matt's words, conspired with other board members and was voted interim CEO, quoting TechCrutch. Earlier Wednesday, Mullenweg posted in a Slack channel visible to all employees that the company's chief financial officer Mark Davies had conspired with a. other board members, Anne Dunwoody, Tony Schneider, and Sue Decker to vote to put Malinweg on paid leave of absence. The message from Malinwedic read, quote, and the biggest news. I won't be able to make the meeting tomorrow. At Mark Davies has conspired with at Anne Dunwoody, at Tony, and at Sue Decker behind my back, and they voted to put me on paid leave of absence. I voted against

that, wishing Mark and all of you the very best. To clarify, at Mark Davies was voted as interim CEO. I received the resolution 50 minutes before the meeting start and requested repeatedly for time to have it reviewed by independent legal counsel even for a few hours, which was denied, end quote. A slack message to the open source WordPress.org community from the project's executive director, Mary Hubbard, also confirmed Mullenweg's change of status at the commercial company. However, Hubbard said that WordPress.org was not impacted. Matt remains the leader of the WordPress project, and I remain executive director of WordPress. Our teams, priorities, and work continue as planned, she wrote.

Mullenweg's Slack message did not make clear why Autematics Board took this action. Sources told 404 Media, which was the first to report the news, that Schneider, the former CEO of Automatic from 2006 to 2014, confirmed that the action was initiated by the board. Davies also told employees in the Slack channel that Mullinweg remained on the board of directors. Automatic, which owns other brands including Tumblr, WooCommerce, and Pocketcasts, was founded by Mullenweg in 2003 after he co-created the popular open source content management system WordPress.

Over the past few years, Mullenweg and the company have been mired in lawsuits and controversy. The company is in a protracted legal battle with WP Engine, a WordPress web hosting competitor. The lawsuit followed allegations for Molleague that WP Engine was profiting from the open source WordPress project without contributing back to the community, and he demanded that it pay 8% of its monthly gross revenue as a royalty fee. for using the WordPress brand. Tensions became so high under Mullenweg's leadership that he told employees in 2024 to quit with severance

if they disagreed with him and 159 did so. Last year, Mullenweg also threatened to deactivate the accounts of several members of the open source WordPress.org community after two of them allegedly said they planned to spearhead a new forked version of the open source WordPress project. Those involved denied their plans, but said Mullenweg went through with their account deactivations anyway

and also deactivated accounts belonging to other contributors, sources told TechCrunch, end quote. Cyber threats can pop up where you least expect them to. Like when you're focusing on today's threats, while AI is changing the entire landscape of cybersecurity right under your nose. Rubrik can help prepare your organization to handle this new world of cybersecurity. With Rubrik, you get one platform that secures your data, controls your AI, and protects your identity. Learn more at rubric.com slash mb. That's rubr-i-k.com slash mb.

Is your multi-entity management creating more confusion than clarity you need the Intuit ERP, Intuit Enterprise Suite? It's the AI-Native ERP solution that's powerful, painless, and proven. Learn more at intuit.com slash ERP. Some bits and bobs from the iPhone event yesterday that I didn't have time to underline. which I will underline now, including that new health app that leans heavily on Apple Intelligence. Quoting TechCrunch, alongside the new Apple Watch Series 12 and Ultra 4, Apple on Wednesday introduced a redesigned Apple health app that takes advantage of Apple intelligence to make better sense of your health data. The changes deliver a new insights tab that includes a summary of the most timely information,

plus personalized guidance, assessments, a new readiness score, and a computation of your health age. The latter is a way to understand how your various health metrics compare with your actual age and uses data like your VO2 Max, sleep, and more to include biomarkers from your blood. The additions also include a new longevity tab which uses metrics from your Apple Watch and iPhone to assess your health across sleep, movement, heart health, and other factors. Your lab results can be incorporated into this guidance, and Apple announced a team-up with Quest for a 50 biomarker panel for $119 giving users. users quick access to data that can only come from lab tests.

With Apple Intelligence, Apple Health's app has been improved to make sense of other health data too, which comes into play in features like the Readiness Score and the Insights tab. For instance, the app could suggest that a user incorporate more intervals into their morning run to improve their cardio health. The app will also go deeper in areas like fitness and heart health as well, Apple said. The health app also allows users to take in-depth assessments of how well their bodies are moving, which can help people who are older or recovering from an injury. A trainer can even demonstrate certain moves for you to try as you take the assessment, which Apple claims is like having a coach right there with you.

The new health app is rolling out later this year, end quote. And this is something I would potentially use every day all the time, depending on how it worked, and acknowledging that, yes, there is a creep factor here we shall see. but quoting Wired. The new Apple Watch Series 12 and Ultra 4 come not just with better fitness tracking and upgraded noise reduction, but also a whole new way to listen. Apple announced on Wednesday that the two smartwatches come with four opt-in audio

intelligence tools that are powered by audio gathered by the watch's microphones. They include sound and music recognition, a conversation recap feature, and live rewind so a user can see a transcription of anything that was said in their environment in the previous 15 seconds. Seemingly wary that the features could make it feel like the walls have ears, Apple is emphasizing that all of them are built to prioritize privacy and security using the extensive infrastructure the company has developed to protect its other Apple intelligence and series services. As more and more of these types of features debut, though, the audio intelligence announcements are a reminder that AI-powered tools are becoming increasingly universal

in all areas of computing and life. These features do not create or store audio recordings and raw audio use for processing is completely inaccessible to the operating systems apps, the user or Apple, the company wrote in a report shared with Wired. The new Apple Watch features are designed to process as much data locally on devices as possible so potentially sensitive data doesn't go to the cloud. Sound recognition, for example, alerts users about sounds in their environment, including doorbells, sirens, alarms, or, say, a baby crying,

without sending any data off the watch. The tools that do send information out to the cloud pre-process the data so it is raw audio files then use Apple's private cloud compute infrastructure. Apple emphasizes that its on-device capabilities for Apple Watches have expanded thanks to the company's new S11 chips. The chips include a special memory protected space that Apple calls the secure enclave. Designed specifically for sensor data, the enclave is an isolated buffer that is inaccessible to the rest of the operating system where Apple Watch can hold and process audio data in a protected space. The new Shazam feature listens for music and will generate a signature of a song that is stored in the secure

enclave. If the user navigates to Shazam to find out what the music is, the tool will send that signature, not an audio file, to Shazam's servers for identification. Shazam is an Apple-owned service. If the user doesn't prompt to identify the music or as soon as they do, this special signature is immediately deleted from the watch. The recap feature, known as Siri Recap, can be set to be on all the time and generate recaps of any substantive conversation or it can be given a schedule to listen in at certain times during the day. A dedicated AI model determines when speech is occurring without recording or transcribing any data. If it detects a conversation, audio goes into a protected buffer within the secure enclave on the Apple watch, the watch

encrypts it, and then transmits it using Apple's special secure Bluetooth pairing to the secure enclave on a user's iPhone. Then the audio is immediately deleted from the Apple Watch. The iPhone uses local speech recognition and language models on device to transcribe the audio and then generate a minimal version of the transcript that removes non-essential elements like extra words and repeated phrases. Then the raw audio is immediately deleted from the iPhone as well. The last step on the iPhone is a safety model that screens the text to emit potentially harmful terms, according to Apple. From there, the distillation of the conversation is encrypted, and the iPhone sends it out to private cloud compute.

Contextual information is also sent to improve summary quality, Apple says. For example, now playing data helps the model understand that speech may have originated from music or a podcast. Calendar data can help generate accurate titles of summaries. High-level location labels such as home, work, and school, along with locality information such as city, state, and country. Add context to the summarization. Point of interest categories like grocery store or park may also be included. precise location and specific points of interest are not included. Finally, the condensed transcript is run through Apple's foundation models and private cloud compute

to create a title and summary which is then encrypted and sent back to the iPhone and Apple Watch. Apple says that Siri Recap is built to remove sensitive information like financial data, ID numbers, or other personal identifiers from its summaries. The live rewind feature that lets users ask for a text version of the last 15 seconds, of what was said in their environment uses the secure enclave buffer to continuously hold audio on a rolling basis where old sound is replaced by new audio and doesn't amass. To get a transcript through Live Rewind, users must double press the Apple Watch's digital crown each time. Once activated, the watch sends the last 15 seconds of buffered audio from its secure enclave to the secure enclave of the paired iPhone. If the iPhone is not available, the transfer will fail and the audio will be automatically deleted.

Siri Recap has similar fail safes if the transfer from the watch to the iPhone fails. If the transfer goes through, the iPhone does on-device speech recognition processing to make a transcript and then deletes the audio and sends the text back to the Apple Watch. Users read the transcript and then can discard it or save it in the Siri app. But it's hard to ignore that in spite of all of these data protections, anyone wearing a new Apple Watch could be sicking audio intelligence on, say, your intimate vent session, private family update or accidental slip-up about a secret. To try to mitigate blatant privacy violations, Apple says that the watch produces an audible chime when a watch owner initiates a

live rewind transcript to alert the other people in the area, even if it is on silent mode or paired with headphones. And the features are built to omit or filter out potentially identifying information about speakers. But the sheer scope and breadth of the tools is extremely noteworthy, end Quote. Nothing more for you today. Talk to you tomorrow.