The AI Daily Brief: Artificial Intelligence News and Analysis - Can OpenAI and Anthropic's Frontier Model Forum Improve AI Safety?

Episode Date: July 26, 2023

OpenAI, Anthropic, Google and Microsoft have teamed up to create the Frontier Model Forum. Will it improve AI safety or is it just corporate windowdressing? Before that on the Brief: Alibaba Cloud has announced it will support Meta's Llama 2 model in China; Hugging Face, GitHub and others advocate for open source policy changes in the EU AI Act and people are livid about Netflix's $900k AI job listing. ABOUT THE AI BREAKDOWN The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI breakdown, we're looking at a new collaboration between OpenAI, Anthropic, Google, and Microsoft on a frontier model forum. Before that on the brief, some interesting questions around open source in China. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.network for more information about our Discord, YouTube, and newsletter. Welcome back to the AI breakdown brief. All the AI headline news you need in five-ish minutes or less. Today we kick off with a number of discussions in the open source realm. First of all, focusing on a policy perspective.

Now, so far, the most comprehensive AI legislation anywhere in the world is the EU's AI Act. A draft of the Act has been voted upon, but there is still some amount of room for dialogue, negotiation, and changes. One set of changes that some are advocating for is for more openness to open source in that legislation. A coalition of open source companies including HuggingFace, GitHub, Creative Commons, and more have released and shared with the EUA policy paper titled Supporting Open Source and Open Science in the EU AI Act. The aim they say is to, quote, ensure that open AI development practices are not confronted with obligations that are structurally impractical to comply with or that would be otherwise counterproductive. The executive summary of the paper lays out two benefits to EU citizens for thinking about and fostering the open source ecosystem. First, they say the values of sound research, reproducibility, and transparency fostered by open science are instrumental to the development of safe and accountable AI systems.

Second, open source development can enable competition and innovation by new entrants in smaller players, including in the EU. Now, rather than just a general position, the paper attempts to give five concrete suggestions for specific changes to the AI Act that would help support and protect open source. The first is to define AI components clearly. The second is to clarify that open source developers would be. not be subject to the same stringent requirements that closed source developers are. A third is to, quote, support the AI office's coordination and inclusive governance with the

open source ecosystem building on the Parliament's text. A fourth is to ensure the R&D exception is practical and effective. And a fifth is to set proportional requirements for foundation models, recognizing and distinctly treating different uses and development modalities. The proposal has been met with a lot of positivity. Tim Sweeney, the CEO and founder of Epic Games, tweeted, protecting open source is vital for the future of AI. and keeping its foundations open. Now, when it comes to the discussion of open source, one of the areas where it gets thorny,

particularly in U.S. policy circles, is with the question of China. All the way back in November, Protocol wrote a piece called Will Nationalism End the Golden Age of Global AI Collaboration? The subheader reads, until now, the borderless open source software movement that has helped bring together AI developers

and tech from the U.S. and China has risen above geopolitical tensions. Could national security crackdowns tear it apart? Now, certainly throughout this year, as AI has gotten more and more on the policy agenda, it is converged with broader U.S.-China policy in ways that are definitely more competitive than cooperative. For example, back at the end of June, the Wall Street Journal posted a piece about how the U.S. was considering even further prohibitions on AI chip exports to China.

Many of those prohibitions already existed, but the WS.J reported that the Biden administration was considering even further limits on AI chip exports, effectively prohibiting companies from selling to China at all. It seems every week there's another big op-ed like this one from July in the New York Times. Biden is beating China on chips. It may not be enough. So of course, with this is the backdrop, it all made the news that Alibaba would be supporting Meta's new Lama 2 model for clients all the more intriguing.

There are a couple interesting notes about this. First of all, this is based on META's new Lama 2 model, which has a commercial license available to it. They launched Lama 2 with Microsoft as their preferred partner, but are making it available to other companies around the world as well. Second, it's interesting because Alibaba has been developing its own AI models as well. Reuters explained why from a business standpoint both of these companies might be excited about this move. They write, the U.S. has been actively looking to restrict Chinese companies access to many U.S.

developed technologies related to AI, particularly in the area of AI semiconductors. The Lama 2 move would allow Alibaba to further its own AI ambitions by keeping abreast of the latest developments in the technology. For MEDA, whose Facebook social media platform has for years been banned in China, along with other Western platforms, it could bring closer ties with the world's second biggest economy. That is, of course, if this doesn't get caught up in the political drag net. AJ Button tweeted this morning, Alibaba Cloud gets access to Meta's Lama 2. The LLM empowers Chinese businesses to build generative AI apps.

Will this be blocked by Biden and Co? Speaking of markets, yesterday we had earnings calls from Microsoft and Google, and big surprise what the main theme was, it was, in fact, AI. AI, AI, AI, AI, AI, AI, AI, AI, AI, AI, AI, AI, somebody needs to turn that into a song, and I'm going to have nightmares tonight. That's it. Based off our count from both calls, Microsoft and Alphabet executives said AI, or a version of that, 145, combined times. Jokes aside, both companies report that they are going to invest heavily. We're talking billions and billions of dollars into this area, suggesting that at least for the big tech giants, this is not some passing fad. Meanwhile, when it comes to mainstream consciousness of AI, there continues to be a significant amount of focus on the Hollywood strikes, including both

the Screen Actors Guild and the Writers Guild of America. Today, we saw a number of breathless articles castigating Netflix for listing AI jobs that paid up to $900,000 as the strike was on ongoing. The Intercept writes, as Hollywood executives insist it is quote just not realistic to pay actors, 87% of whom earn less than 26,000 more, they are spending lavishly on AI programs. While entertainment firms like Disney have declined to go into specifics about the nature of their investments in artificial intelligence, job postings and financial disclosures reviewed by the Intercept reveal new details about the extent of these companies embrace of the technology. In one case, Netflix is offering as much as $900,000 for a single AI product manager. The job listing in question,

is for a product manager machine learning platform, and the description does give some amount of information about how they view machine learning and AI in the context of their company. They write, with more than 230 million members in over 190 countries, Netflix continues to shape the future of entertainment around the world. Machine learning and artificial intelligence

is powering innovation in all areas of the business, from helping us buy and create great content, helping members choose the right title for them through personalization, to optimizing our payment processing and other revenue-focused initiatives. Now, as you might tell, nothing in that description seems to suggest that this is about creating a new way to develop programming that would replace the traditional process of, you know, having a writer and a director and an actor.

But I don't think that those details matter so much as the culture war aspect that has taken hold in this particular circumstance. It continues to feel to me like this particular strike is the first resurgence of labor organization in a post-chatCHEPT world, but I have no doubt that it will not be the last. Anyways, guys, that is going to do it for today's AI breakdown brief. If you're enjoying the AI breakdown, do me a favor and tell one friend. Whatever one friend you think would get the most value out of this sort of daily news around artificial intelligence, go let them know about the show. I love growing this community organically, and I appreciate all of your guys' contributions to that. Thanks as always for listening or watching, and I'll be back soon with the main AI breakdown.

Welcome back to the AI breakdown. One of the big themes for this year has been the growing importance of the AI safety conversation. Now, this has manifest in part as a regulatory discussion, of course. We have a growing number of hearings, including one just yesterday that featured among other witnesses, Turing Award winner, Yoshua Benjio. But another dimension of this safety conversation has been companies themselves who are in the AI space, stepping up and starting to articulate a set of principles or norms or relationship exchanges through which they might develop AI more safely and responsibly.

Today, we are discussing the latest of those efforts. It's something that's called the Frontier Model Forum and is a collaboration between OpenAI, Anthropic, Google, and Microsoft. In their announcement post, OpenAI says, we're forming a new industry body to promote the safe and responsible development of frontier AI systems, advancing AI safety research, identifying best practices and standards, and facilitating information sharing among policymakers and industry. What follows in the OpenAI blog post, they say, is a joint announcement coordinated between them and their partners, Anthropic Google and Microsoft. The combined teams write that the core objectives of the forum are, one, advancing AI safety research to promote responsible development of frontier models, minimize risks, and enable independent standardized evaluations of capabilities and safety, two, identifying best practices for the responsible development and deployment of frontier models, helping the public understand the nature, capabilities, limitations, and impact of the technology.

Three, collaborating with policymakers, academics, civil society, and companies to share knowledge about trust and safety risks. for supporting efforts to develop applications that can help meet society's greatest challenges, such as climate change mitigation and adaptation, early cancer detection and prevention, and combating cyber threats. Now, this obviously comes just after last week's slate of voluntary commitments from seven leading AI companies that were secured by the White House. To recap, those commitments include, in the category of ensuring products are safe before introducing them to the public, companies are committing to internal and external security testing before their release. Companies are committing to sharing information across the industry with government, civil society, and academia.

In the category of building systems that put security first, companies commit to investing in cybersecurity and insider threat safeguards. The companies commit to facilitating third-party discovery and reporting of vulnerabilities. And finally, in the category of earning the public's trust, the companies commit to developing robust technical mechanisms to ensure that users know when content is AI generated, i.e. some sort of watermarking system. companies commit to publicly reporting their AI systems capabilities, limitations in areas of appropriate and inappropriate use. Companies commit to prioritizing research on the societal risks that AI systems can pose.

And finally, the companies commit to develop and deploy advanced AI systems to help address society's greatest challenges. You'll note that a lot of the language here is very similar. Again, let's go back to the core objectives of the Frontier Model Forum and see what they say now that we've just had a chance to recap the White House commitments. advancing AI safety research to promote responsible development. That includes, they say, enabling independent standardized evaluations of capability and safety, directly from and in line with those White House commitments, collaborating with policymakers, academic civil societies, and companies. Again, built into the commitments as something where knowledge doesn't just stay in the black boxes of companies, but is shared openly.

Supporting efforts to develop applications that can meet society's greatest challenges. Even the two that they mention here, climate change mitigation and early cancer detection, are the same that are referenced by the White House. Identifying best practices for the responsible development and deployment of frontier models. Again, an echo of things like companies committing to publicly reporting their AI systems capabilities, limitations, and areas of appropriate and inappropriate use.

So what is really different about this, given that we already saw those commitments? I think the big thing here is that instead of just a set of commitments one time that are made sort of one-to-one between the White House and these companies, this is now a membership organization, organized by and maintained by the companies involved that theoretically creates a context

for not only holding themselves and their peers to those commitments, but also for inviting new companies as they grow into that broader alignment. Now, interestingly, this is not just for any old AI company. This is for companies that are deploying large-scale models that, quote, exceed the capabilities currently present in the most advanced existing models. And what's more, these are for organizations deploying models

that can, quote, perform a wide variety of tasks. I think in some ways one of the things that they're trying to do is draw a clear line between things that are safe enough to be fine, i.e. stuff that we have today, and places where we might reasonably as a society or as a government, want to get worried, which is stuff that is more advanced than today. None of these announcements ever quite go that far, but it's been a subtext for basically everything that OpenAI has said about this since those commitments came out last week. So what are they actually going to do? Luckily for us, that is a section of this announcement, what the Frontier Model Forum will do. They write, governments and industry agree that while AI offers tremendous promise to benefit the world, appropriate guardrails are required to mitigate risks. Important contributions to these efforts have already been made by the U.S. and U.K. governments,

the European Union, the OECD, the G7, via the Hiroshima AI process, and others. To build on these efforts, further work is needed. The forum, they say, will focus on three key areas over the coming year. One, identifying best practices. basically if people learn about the best ways to prevent harm, they need to share them, not just keep it secret, to advancing AI safety research, i.e. coordinating cross-organizational efforts to progress on areas they identify like adversarial robustness, mechanistic interpretability, scalable oversight, independent research access, emergent behaviors, and anomaly detection.

One meat-on-the-bones kind of detail is that they say that there will be a, quote, strong focus initially on developing and sharing a public library of technical evaluations and benchmarks, And their third key focus is facilitating information sharing among companies and governments. Basically, create regular communication channels that can be used to actually get this information flowing between these parties. Now, in terms of how this will actually run, they say that they will establish an advisory board to guide strategies and priorities. And as part of that, they will establish key institutional arrangements, including a charter governance and funding, along with a working group and an executive board to lead those efforts. The piece concludes with some feel-good quotes from the people involved, none of which really say much of anything at all.

So what does the community think about this? Well, one first thing to note is that when you go search Frontier Model Forum on Twitter slash X, the organizations involved have done a very good job of making sure that everyone is on message. Basically, every company and every party to this Frontier Model Forum has shared the initial announcement post, as well as a tweet version of the quote that they contributed. Now, to the extent that there is chatter outside the people who have directly contributed to this initiative, it is, as you might imagine, skeptical. One person tweets,

A forum for companies that have failed in responsible development of AI systems of non-frontier models will now be responsible for frontier models. Seems to me that these companies don't fulfill the membership criteria that they have formulated. Another account, this won an organization of high school and college students that are, quote, mobilizing their peers for human-centered artificial intelligence rights. While a promising follow-up to last week's commitments, big tech company's announcement today of the Frontier Model Forum means little without concrete steps and new norms for AI safety.

Self-regulation is no substitute for government action. Now, a reasonable question to ask is, is this just an example of industry trying to capture the mantle of some sort of window-dressing self-regulatory efforts so that they're not subjected to stricter government controls? Or is there something more here? I think one thing that's worth noting on that front is that in many ways, this phase of the AI safety conversation was kicked off when Jeffrey Hinton left Google. He has subsequently done about a million interviews, but the first one that really got

things going was the piece that appeared to the New York Times on May 1st called The Godfather of AI leaves Google and warns of dangers ahead. Now, when it came to why Hinton felt like this was the time to actually go make this big change and try to engage people and raise the profile of these issues, he seemed to pin a lot of it on the competition between companies. From that New York Times piece, quote, until last year, Hinton said, Google acted as a proper steward for the technology, careful not to release something that might cause harm. But now that Microsoft has augmented its Bing search engine with a chatbot, challenging Google's core business, Google is racing to deploy the same kind of technology. The tech giants are locked in a competition that might be

impossible to stop. And it is not just hint in who's talking about this. All the way back in February, Time magazine published a piece called the AI arms race is changing everything, discussing the rapid rise of chat GPT, Time writes, the frenzy appeared to catch off guard even the tech companies that have invested billions of dollars into AI and has spurred an intense arms race. race in Silicon Valley. In a matter of weeks, Microsoft and Alphabet on Google have shifted their entire corporate strategies in order to seize control of what they believe will become a new infrastructure layer of the economy. Google declared a code-red corporate emergency in response to the success of ChatGPT and rushed its own search-oriented chatbot barred to market.

Microsoft CEO Satya Nadella said February 7th, a race starts today. We're going to move and move fast. The great concern that many have is that the pressures of market competition are going to win out against the forces of responsibility. The great prize of having the winning models and the winning platforms in this new AI-powered world, in other words, is going to justify in corporate boardrooms all across the tech world in Silicon Valley being just a little bit more risky than they might otherwise have been, releasing a technology that they don't fully understand and can't quite fully control, because if they don't, someone else will, or even worse, maybe someone else already has. To the extent that there is a non-cynical take on the Frontier Model Forum, it is frankly

its existence at all. It creates a context to at least provide lip service to a counterweight to the vicious competition between companies with some amount of cooperation between them. I find it unlikely to think that the Frontier Model Forum will at the beginning be anywhere near as powerful as those market incentives when it comes to the competition in the AI arms race, but at least it exist to provide some amount of counterweight. And indeed, in some ways, give governments and regulators a path into these organizations to try to either nudge them or frankly force them away from competition to some extent and towards collaboration. Now, whether that's overly optimistic remains to be seen. But given the choice between the Frontier Model Forum and not the Frontier

Model Forum, I'm taking the Frontier Model Forum. You guys can rest assured that we will be keeping a close eye on this. So until tomorrow, peace.