A New AI Safety Report (and Why the Media Loves the AI Extinction Narrative)
00:00Today on the AI Breakdown, a new report funded by the State Department advocates some very aggressive stances with regard to artificial intelligence.The AI breakdown is a daily podcast and video about the most important news and discussions in AI.Go to Breakdown.netnet for more information about our YouTube, our newsletter, and our Discord.Hello, friends, quick note here.I did the main episode first today, and it got quite a bit longer than my normal main episode.And frankly, I didn't even get into everything that I could have.So for that reason, we're going to skip the brief today. We will be back with our normal format, however, tomorrow, with a profile of a couple new pieces of software that some folks are getting very, very excited about, including what some are calling the most advanced AI coding app that they've ever seen. However, for this episode, we are talking all about this AI safety report, and even more, the media coverage surrounding these issues.Welcome back to the AI breakdown. A new report has been flying all over the media wires and the Twitter.
01:05Earth's fear, and probably the most referenced article about it, is this piece in Time magazinecalled Exclusive. U.S. must move decisively to avert extinction-level threat from AI,government-commissioned report says. I will be clear right up front that I went from,interested to see what this report had to say, to fairly disgusted with the media coveragearound it in the 24 hours that I've been paying attention. I'll explain a little bit more aboutwhy, but first, let's talk about what the report actually says. To get a sense of the position,time begins, the U.S. government must move, quote, quickly and decisively, to avert substantialnational security risks stemming from artificial intelligence, which could, in the worst case,
01:44cause an extinction-level threat to the human species, says a report commissioned by the U.S.government published on Monday. The report reads, current frontier AI development poses urgentand growing risks to national security. The rise of advanced AI and AGI has the potentialto destabilize global security in ways reminiscent of the introduction of nuclear weapons.The executive summary is called Defense in Depth, an action plan to in-eastern,increase the safety and security of advanced AI. The executive summary reads,The recent explosion of progress in advanced artificial intelligence has brought great opportunities,but it is also creating entirely new categories of weapons of mass destruction and WMD enabling
02:19catastrophic risks. A key driver of these risks is an acute competitive dynamic among the frontierAI labs that are building the world's most advanced AI systems. All of these labs have openlydeclared an intent or expectation to achieve human level and superhuman artificial generalintelligence, a transformative technology with profound implications for democratic governance andglobal security by the end of this decade or earlier. From there, they say that the frontier labexecutives have acknowledged these dangers, but continue to push anyways, and so they conclude,quote, there is a clear and urgent need for the U.S. government to intervene. This action plan,they write, is a blueprint for that intervention. Its aim is to increase the safety and security
02:52of advanced AI by countering catastrophic national security risks from AI weaponization and lossof control. So what actions do they propose? Well, they say they follow a sequence that,begins by establishing interim safeguards to stabilize advanced AI development, including export controls on the advanced AI supply chain,leverages the time gained to develop basic regulatory oversight and strengthen U.S. government capacity for later stages.Transitions into a domestic legal regime of responsible AI development and adoption,safeguarded by a new U.S. regulatory agency, and finally extends that regime to the multilateral and international domains.Here's the way the time sums it up.The finished document recommends a set of sweeping and unprecedented policy actions that, if enacted, would radically disrupts.
03:31the AI industry. Congress should make it illegal. The report recommends to train AI models usingmore than a certain level of computing power. The threshold the report recommends should be setby a new federal AI agency, although the report suggests as an example that the agency could set itjust above the levels of computing power used to train current cutting-edge models like OpenAIs,GPT4, and Google's Gemini. The new AI agency should require AI companies on the frontier of the industryto obtain government permission to train and deploy new models above a certain lower threshold.Authorities should also urgently consider outlawing the publication of the weights or inner workings of powerful AI models,for example under open source licenses, with violations possibly punishable by jail time.
04:07And the government should further tighten controls on the manufacture and export of AI chipsand channel federal funding toward alignment research that seeks to make advanced AI safer.Even Time calls the report's recommendations, quote, previously unthinkable.Time also talks about how the report hones in on hardware as a good area to regulate as a way to slow AI down.Quote, regulating the proliferation of this hardware, the report arguesmaybe the most important requirement to safeguard long-term global safety and security from AI.It says the government should explore tying chip export licenses to the presence of on-chip technologies,allowing monitoring of whether chips are being used in large AI training runs.
04:40Now, the response to this report has, as you might expect,fallen almost completely along the political dividing lines of the AI safety folks on the one handand the accelerationsist on the other.Prominent AI safety voice Max Tegmark writes,A report commissioned by the U.S. government finds that future AI poses an extinction level risk.I agree. Ed Newton Rex, who is the person who is in charge of audio at stability AI but left over their copyright thinking, tweets.This is an extremely important report. Commissioned by the U.S. government, it outlines two key existential risks from AI,weaponization by rogue actors and escaping the control of humans. And it strongly recommends governments take these threats seriously whatever the potential upsides of AI and act fast.
05:18On the flip side, there are folks like Riva who tweets,report commissioned by the U.S. government says that the U.S. government are best poised to manage AI.Shocker. Eric Hartford from Abacus writes,What's next? An AI Prohibition? AI speak-eas? I'm Jodd Mossad, the CEO of Replet, writes.To save you a click, the government wrote a 250K check to a handful of pre-committed AIDoomers to write a doom and gloom report, so they wrote one, to which George Hots responded,it's called Manufacturing Consent. And honestly, these were some of the tame responses.Daniel Jeffries writes, I'll keep it simple. Here's what I recommend.
05:48Take any report that seriously includes a discussion aboutAI killing us all and pitch it right in the trash. Never allow anyone with this delusional viewpointwithin 5,000 miles of a policymaker. But Nick Marta, the technical lead on AI governance at Mozilla,gets it one of the issues that I think is core here. Nick writes, I will now be referring toevery NSF funded paper as a government commissioned report. Brilliant communications, though. Just goaround saying the U.S. government commissioned this first ever action report, get a flattering newsarticle about it, and then monetize that traffic. What he's pointing to is the fact that to get a copy ofthe action plan, you have to sign up with your name and email.
06:21on the Gladstone website, which is the group that was behind this research.Now, let's talk about Gladstone for a minute.Here's how time describes the firm.Jeremy and Edward Harris, the CEO and CTO of Gladstone, respectively, have been briefing theU.S. government on the risks of AI since 2021.The duo, who are brothers, say that government officials who attended many of theirearliest briefings, agreed that the risks of AI were significant, but told them the responsibilityfor dealing with them fell to different teams or departments.
06:45In late 2021, the Harris' say Gladstone finally found an arm of the government with theresponsibility to address AI risks.the State Department's Bureau of International Security and non-proliferation. Teams within the Bureauhave an interagency mandate to address risks from emerging technologies, including chemical andbiological and biological and nuclear risks. Following briefings from Jeremy and Gladstone's thenCEO Mark Beale in October 2022, the Bureau put out a tender for report that could inform a decisionwhether to add AI to the list of other risks it monitors. The State Department did not respondto a request for comment on the outcome of that decision. The Gladstone team won that contract.
07:17So basically, this is a team of four people, or at least it was four,when they won this $250,000 contract in October 2022, as it appears that the third co-author of thereport, who was Mark Beale, a former Defense Department official, has subsequently left Gladstoneto start a super PAC aimed at advocating around AI policy. His PAC is called Americans for AISafety and also officially launched on Monday. Wright's time, it aims to make AI safety and securitya key issue in the 2024 elections, with the goal of passing AI safety legislation by the end of2024. The PAC did not disclose its funding commitments, but said it has, quote, set a goal of raisingmillions of dollars to accomplish its mission. Time also notes that the Harris brothers were previously
07:54the entrepreneurs behind a company that went through Y Combinator and writes, the pair brandishthese credentials as evidence that they have the industry's interests at heart, even as theirrecommendations if implemented, would append it. Now, let's read the tweet thread that Edward Harrisdid around this. He writes, here's what we've been working on for over a year. The first U.S.government commissioned assessment of catastrophic national security risks from AI, including systemson a path to AGI. TLDR, things are worse than we thought, and nobody's in control. Harris continues,started this work with concerns, but no preconceptions. We knew there were solid technical reasonsthat AI could eventually pose catastrophic risks. But we went in looking for reasons to change our
08:28minds. We found the opposite. Our overriding goal was to get to the truth. To do that, we hadto do more than just speak to policy and leadership at the AI labs. We also connected withindividual technical researchers, many of whom are way more concerned than their labs let on inpublic. Many of these folks came forward on condition of anonymity to share stories. He then goes onto tell a number of anecdotes and pitches the framework that they came up with. Now, to me, oneissue here is, of course, debate over the substantive issues herein. The policies advocated forhere are absolutely extreme relative to where most people are, but I don't have any problemwith people having these debates. It's totally fine for a group of researchers to dig into these
09:04issues and come to the conclusion that these are the right ideas. We live in a free societywhere everyone's positions, even extreme positions, gets to be debated. More broadly, I don't mindthere being a great, even fierce debate between the extremes of accelerationism and safetyismat either ends of the discussion.What's frustrating to me, and I think to many who are observing how this is being presented,is in fact, in the press coverage.I read you the Times headline.US must move decisively to avert extinction-level threat from AI,
09:30Government Commission Report says.Venturebeat says State Department-backed report provides action plan to avoid catastrophic AI risks.In short, these reports make it seem like President Biden himself commissioned this report,that this was a central and key asset in U.S. government decision-making.rather than what it appears to have been, which is a group that had a pre-existing concern around a particular issue,spent a bunch of time building relationships, and trying to find others who might resonate with that position,and then got a grant for a quarter million dollars to write a report based on that position,which, for all we know, might have either made waves inside the specific department within the state department that paid for the contract,
10:08or might have been chucked right in the trash can.What's irresponsible, in other words, is the presentation of this like it is the State Department and the U.S. government's position.it lends it an authority which it should not have, and which warps that debate that I'm so encouragingof their being. The issue, of course, is that this is not somehow out of the norm for how press andmedia treat these issues right now. Sometime around this time last year, the press went fromnever caring at all and never thinking about any sort of AI-related issues to being completelyand utterly in love with this extinction narrative. As that's happened, we've seen Americanattitudes about AI decline precipitously. And of course, I don't want to blame that entirely on
10:48the media. There are plenty of reasons why someone might be concerned about AI in terms of job lossor whatever other issue there might be, to say nothing of the overall trend of decliningtrust in big tech. But it certainly makes it very hard to have a rational, wide-rangingconversation about these issues when people are being pummeled constantly with the idea thatTerminator is going to come end their world. Now, I don't expect AI safety advocates to have anyissues with this. It's likely to me that they'll see this as simply winning the battle of ideas.But I find myself personally radicalizing in the other direction because I think people are gettingsuch a warped and one-sided conversation. And apparently I'm not the only one who feels that way.
11:24Just before I pressed record on this show, I noticed that a new organization called the Alliancefor the Future had just announced itself with a mission effectively to counter these types ofnarratives when it comes to artificial intelligence. John Asconis writes,It's time to take back our future. I'm proud to be a small part of the Alliance for the Future,a new political advocacy organization aiming to give the future,our future, our children's future, America's future,a voice in American politics and to push back against a growing conspiracy against it.Alliance for the Future is an alliance and it's a big tent.
11:52We've got EACC's, American Dynamists, Libertarian, Civil and Otherwise,China Hawks, Cypunks, and more on board.What we have in common is an enemy.The people who would rather cancel America's futurethan give up the power and wealth they currently enjoyor their neurotic desires to exert control.Today's battleground is AI.Most Americans have little idea about how AI actually
12:10works, how it differs from Hollywood fantasies, and about how the benefits they, and especiallyworking-class Americans could enjoy, are being canceled by an organized cabal of Baptist andbootleggers. Now, I had started to wonder at what point we were actually going to see acounter-response to the doom and gloom narrative, and here we have it. I still tend to thinkthat talk to the average person, and they're going to be way more in the middle than either ofthese extremes, and in that middle space, there's going to be lots of common-sense approachesthat involve neither prohibitionnor complete ignoral and ignorance,
12:42but until those more moderateor just still being formed opinionsget coverage in the media,I think we're a little bit doomed to be subjectto just endless back and forthbetween the narrative extremes.Hopefully this AI breakdown spacecan counteract that trend
12:55at least in small waysby at least presenting all sides of this,and yet, as much as this audience has grown,it remains still small, relatively speaking.But I certainly wouldn't want to be talking about anything else,and so I'm glad to have you all here on this journey.For now, that's going to do it for today's AI breakdown.Until next time, peace.
There aren't comments yet for this episode. Click any sentence in the transcript to leave one.