The AI Daily Brief: Artificial Intelligence News and Analysis - Regulatory Capture? Meta AI Chief Accuses OpenAI, Anthropic of Stoking Fears

Episode Date: November 3, 2023

The war of words between the AI safety advocates on the one side and the AI open source advocates on the other has gotten increasingly contentious. This week, Meta's Chief AI Scientist Yan LeCun accused the leaders of Google DeepMind, OpenAI and Anthropic of stoking fears around AI in order to provoke a response from regulators that would be beneficial to their businesses and bad for open source competitors. Today's Sponsors: Listen to the chart-topping podcast 'web3 with a16z crypto' wherever you get your podcasts or here: https://link.chtbl.com/xz5kFVEK?sid=AIBreakdown  Interested in the opportunity mentioned in today's show? [e-mail] ABOUT THE AI BREAKDOWN The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI breakdown, we're looking at the increasingly contentious debate between the pro-open source AI community on the one hand and the AI safety and extinction risk community on the other. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown. Not Network for more information about our YouTube channel, our Discord, and our newsletter. Welcome back to the AI breakdown. A quick reminder that I am currently off on a very short vacation for my 10th wedding anniversary, so this is pre-recorded. it may not have the most up-to-date information, and that's also why you're not getting a brief as normal. However, today's topic is still very topical and very much of this week. The week kicked off on Monday with President Biden's executive order on artificial intelligence,

and then in the middle of the week we had the AI Safety Summit in the UK, which was meant to really put the issues of AI safety, and in particular, the biggest concerns around the most dangerous scenarios of AI, front and center in the global political discourse. Now, as that was happening, tensions between people with different views of AI safety, the role of open source in the industry, and what's going on in those policy spheres, spilled out into the open in a major, major way. As Nathan Binatch summed up, it's a big week for AI safety and the future course of our industry. So we're going to look at two instances this week of high-profile members

of the AI community, basically arguing that the major tech companies in AI labs are as Lior at Alpha Signal AI put it, spreading fear of AI to gain control over the industry. So the first of these conversations, and a person who's very much at the center of them, is chief AI scientist at META and NYU professor Jan Lacoon. Jan is, of course, a 2018 Turing Award winner alongside Jeffrey Hinton and Joshua Benjillo, but has basically the opposite perspective as them when it comes to AI safety and AI risks. Back at the end of September, as news came out that the AI Safety Summit was meant to focus almost entirely on these existential risks, Jan tweeted,

the UK Prime Minister has caught the existential fatalistic risk from AI delusion disease or afraid. Let's hope he doesn't give it to other heads of state before they get the vaccine. Now, at the beginning of the week, Max Tegmark from the Future of Life Institute, who is one of the main organizers of that six-month pause letter, quote tweeted that late September post and said, Jan, I'd love to hear you make arguments rather than acronyms. Thanks to UK Prime Minister Rishi Sunak and EU Commission President, Ersela von der Leyen for realizing that AIX risk arguments from Turing, Hinton, Benjillo, Russell,

Altman, Hassabas, and Amode can't be refuted with Snark and corporate lobbying alone. Now, that statement absolutely set Jan off and led to an extremely notable post and frankly accusation. Jan responds, Altman, Hasabas, and Amode are the ones doing massive corporate lobbying at the moment. They are the ones who are attempting to perform a regulatory capture of the AI industry. You, Jeff, and Yahshua are giving ammunition to those who are lobbying for a ban on OpenAI R&D. If your fearmongering campaign succeed, they will inevitably result in what you and I would identify as a catastrophe. A small number of companies will control AI. The vast majority of our academic colleagues are massively in favor of

OpenAI R&D. Very few believe in the doomsday scenarios you have promoted. You, Joshua, Jeff, and Stewart are the singular but vocal exceptions. Like many, I very much support OpenAI platforms because I believe in a of forces, people's creativity, democracy, market forces, and product regulations. I also know that producing AI systems that are safe and under our control is possible. I've made concrete proposals to that effect. This will all drive people to do the right thing. You write as if AI is just happening, as if it were some natural phenomenon beyond our control, but it's not. It's making progress because of individual people that you and I know. We and they have agency in building the right things. Asking for regulation of R&D as opposed to product deployment implicitly assumes that these

people and the organization they work for are incompetent, reckless, self-destructive, or evil. They are not. I have made lots of arguments that the doomsday scenarios you are so afraid of are preposterous. I'm not going to repeat them here. But the main point is that if powerful AI systems are driven by objectives, which include guardrails, they will be safe and controllable because we set those guardrails and objectives. Now about open source, your campaign is going to have the exact opposite effect of what you seek. In a future where AI systems are poised to constitute the repository of all human knowledge and culture, we need the platforms to be open source and freely available so that everyone can contribute to them. Openness is the only way to make AI platforms reflect the entirety

of human knowledge and culture. This requires that contributions to those platforms be crowdsourced, a bit like Wikipedia. That won't work unless the platforms are open. The alternative, which will inevitably happen if open source AI is regulated out of existence, is that a small number of companies from the west coast of the U.S. and China will control AI platform and hence control people's entire digital diet. What does that mean for democracy? What does this mean for cultural diversity? This is what keeps me up at night. Tegmark responded, thanks, Jan, for the long and thoughtful reply. You're conflating two separate questions. One is superintelligence and ex-risk? Two is open-sourcing good. You criticized Rishi Sunak over one. I replied, but you still gave no arguments for why it's not

an X-risk, just repeated the trivially falsifiable claim that almost no AI researchers are worried about this. Instead, you change the topic to two. It's obviously great to have lots of AI open-sourced. We love open-source here at MIT, and it's obviously stupid to open-source absolutely everything, like bioweapon optimizing code. So there's an interesting discussion to be had about where to draw the line. If you support open-sourcing superintelligence, you presumably have an argument for one about why it's completely safe, and I'm still waiting to hear this argument of yours. And now a word from today's sponsor. Are you interested in how two top-of-mind trends AI and crypto can work together?

If so, I have the perfect podcast recommendation for you. Web3 with A16Z Crypto, the chart-topping show brought to you by venture firm Andresen Horowitz. Web3 with A16Z Crypto is your definitive resource for the future of the internet. Whether you're already building in these spaces or simply curious about what's next. If you need a place to start, they recently released an excellent episode with Stanford Cryptography Professor Dan Bonay and former Google Xer Aliya in conversation with host Sonal Choxi about the intersection of AI and crypto. From fighting deepfakes and proving humanity to large language models like ChatchipT, they cover it all. I highly

recommend checking it out, especially if you'd like to learn more about how AI and crypto will impact our everyday lives. Beyond Crypto and AI, this show is for creators seeking more ways to truly own their work, for business leaders trying to prepare for the future today, and for innovators exploring trending tech topics. So go ahead, listen to Web3 with A16Z Crypto, wherever you get your podcasts. Now, I will say for folks who have read this before, I apologize for basically just reading this discussion, but A, a lot of you won't have read it yet. And B, given that this is Twitter, it's dense and hard to piece through and surrounded by other comments, and so hopefully this is useful because I do think that it is a marker of

this particular platform that we actually get to see these conversations out in the open. Anyways, Jan responds to Max again. He says the reason that number two, which by the way was Max's question is open source and good, is in question and open source AIR&D is threatened, is one, the question of superintelligence. Those questions are not disconnected. If you ask, give me arguments for why turbojets and rocket engines won't kill people, I could respond by pointing to a dozen treatises on how to design and build reliable turbojets and rocket engines. Those treatises don't exist yet for superhuman AI because superhuman AI has not been invented yet. and people who think scaling up auto-rogressive LLMs will get us there are just wrong.

So the question becomes, what will superhuman AI systems look like? What is the blueprint? My answer to this, which you've heard many times, is that future AI systems, just like intelligent animals, will be objective-driven. Whatever outputs or actions they produce will have to satisfy a set of objectives, some of which will be guardrails that guarantee their safety and controllability. We will set those objectives and guardrails, and AI systems will not be able to violate them by construction. These systems will stay under our control because their primary objective will be to be subservient to us. Now we are not going to build super intelligent, objective-driven AI systems right away.

We are going to build very simple systems first, and make sure the guardrails we put in place guarantee their safety. Then we'll work our way up, building smarter and smarter systems, adjusting the objectives and guardrails as we go. It will be like turbojet and rocket engine reliability, an iterative process of refinement and careful engineering. Much of the X-risk hyperventilating is due to two fallacies. One, just because a system is intelligent, it will want to take over. Two, the minute we turn on a slightly flawed super-intelligent system, it will take over and destroy humanity. Number one is simply false.

The desire to dominate is not linked to intelligence. That's not even true within the human species. The people who see most motivated to dominate are not the smartest among us. To want to dominate, you need a hardwired drive to do so. Only a few social species have such a thing, including humans, baboons, chimps, wolves, etc. Non-social species like orangutans don't. But if you are still scared, we can hardwire objectives into AI systems to make them submissive to humans within limits. Number two reveals a complete misunderstanding of how new technologies are developed and deployed.

The same way we don't build a giant turbojet airliner and put 1,000 passengers in it in the first trial, we won't build a super-intelligent AI prototype and let it loose on the first try. We'll build cat-level AI first and figure out what guardrails are necessary as we work our way up the intelligence level. Now, there are so many places that we could take this conversation at this point. One of the really interesting dimensions that I think has to be included as part of this is the impact of market forces on the whole conversation. I think one of the things that makes the smartest among the AI safety and ex-risk folks that I know most concerned is that we're not just operating in a scientific vacuum.

Instead, we are operating in a market context, which rewards speed over absolutely everything else. But that's not actually the biggest point of the conversation here, and there is so much more to get into, so let's continue. Now, of course, the big place that this started was that Jan was arguing very vociferously that the big AI labs are lobbying the U.S. political establishment against open systems and for regulation in order to perform a sort of regulatory capture where their leads are preserved and they have an advantage because of the cost of compliance.

Epic's Tim Sweeney agreed with this argument. He quote tweeted Jan and said, took Apple and Google 20 years to stoop to asking regulators to endorse their monopolies, quote unquote, for safety. Now, when Google DeepMind's CEO Demis Hasabas was asked about this on CNBC, he clapped back. Hasabas said, I pretty much disagree with most of these comments from Yan. I think the way that we think about it is there's probably three buckets or risks that we need to worry about. There's sort of near-term harms, things like misinformation, deep fakes, these kinds of things, bias and fairness in the systems that we need to deal with.

Then there's sort of the misuse of AI by bad actors repurposing technology, general purpose technology for bad ends that they were not intended for. That's a question about proliferation of these systems and access to these systems. So we have to think about that. And then finally, I think about the more longer term risk, which is technically AGI or artificial general intelligence risk. So the risk of themselves making sure they're controllable. What value do you want to put in them to have these goals and make sure they stick to them?

Now, Hasabas said that it was important to start the conversation about how to regulate AGI now, even though we're not there, because as he put it, I don't think we want to be doing this on the eve of some of these dangerous things happening. Now, as I told you right up front, it wasn't just Jan making these comments this week either. Insider ran a piece about Andrew Ng called Google Brain co-founder says big tech companies are inflating fears about the risks of AI wiping out humanity because they want to dominate the market. Basically, in a conversation with the Australian Financial Review, In said that the biggest tech companies were hoping to trigger strict regulation with the, quote,

bad idea that AI could make us go extinct. He continued, There are definitely large tech companies that would rather not have to try to compete with open source, so they're creating fear of AI leading to human extinction. It's been a weapon for lobbyists to argue for legislation that would be very damaging to the open source community. He expanded upon these points in a tweet and a blog post. He wrote,

My greatest fear for the future of AI is if overhyped risks such as human extinction, lets tech lobbyists get enacted stifling regulations that suppress open source and crush innovation. In a blog post, he wrote, in recent months, I sought out people concerned about the risk that AI might cause human extinction. I wanted to find out how they thought it could happen. They worried about things like a bad actor using AI to create a bioweapon, or an AI system inadvertently driving humans to extinction, just as humans have driven other species to extinction through lack

of awareness that our actions could have that effect. When I try to evaluate how realistic these arguments are, I find them frustratingly vague and nonspecific. They boil down to it could happen. Trying to prove it couldn't is akin to proving a negative. I can't prove that AI won't drive humans to extinction any more than I can prove that radio waves emitted from Earth won't lead space aliens to find us and wipe us out. Such overblown fears are already causing harm.

High school students who take courses designed by Kira Learning, an AI fund portfolio company that focuses on grade school education, have said that they are apprehensive about AI because they've heard it might lead to human extinction, and they don't want to be a part of that. Are we scaring students away from careers that would be great for them and great for society? I don't doubt that many people who share such worries are sincere, but others have a significant financial incentive to spread fear. Individuals can gain attention, which can lead to speaking fees or other revenue.

Non-profit organizations can raise funds to combat the phantoms that they've conjured. Legislators can boost campaign contributions by acting tough on tech companies. As he sums up, I firmly believe that AI has the potential to help people lead longer, healthier, more fulfilling lives. One of the few things that can stop it is regulators passing ill-advised laws that impede progress. Some lobbyists for large companies, some of which would prefer not to have to compete with open source, are trying to convince policymakers that AI is so dangerous, government should require licenses for large AI models. If enacted, such regulation would impede open open source development and dramatically slow down innovation. Now this got its own set of responses,

including from Jeffrey Hinton. Hinton wrote, Andrew Ng is claiming that the idea that AI could make us extinct is a big tech conspiracy. A data point that does not fit this conspiracy theory is that I left Google so I could speak freely about the existential threat. Andrew responded, I didn't say it's a conspiracy, but I think overhyped fears about AI leading to human extinction are causing real harm. He then pointed to the same examples he used in his letter, young students being discouraged, and hype about harm being used to promote bad regulation. He concludes, I know you're sincere in your concerns about AI and human extinction.

I just respectfully disagree with you on extinction risk, and I also think these arguments, sincere though they may be in your case, do more harm than good. Now, as we try to wrap up, there are a couple things that I think are worth noting. First of all, I think that these conversations are a lot harder in the abstract than they are in the specific. What I mean by that is that when all of these debates started, we didn't yet have President Biden's executive order.

Once we did get that order, we could get specific about different points of disagreement. For example, it actually seems like the executive order is going to be a lot worse for the big AI labs, who have been accused of fearmongering to get regulatory capture, than it will be for open source developers. But at the same time, there is still a concern among the open source community that the very core approach to regulation, which is based on how much compute is used or the size of the models, is not the right way to go about things.

Clem from Hugging Face wrote, In my opinion, compute or model-sized thresholds for AI building would be like counting the lines of code for software building. Regulation based on this will most likely be easily fooled, create hurdles and worries for companies to compete on bigger models, so concentration of power, and slow down innovation without solving any of the safety challenges.

What about focusing on use cases and industry-specific evaluations of risks like we do for software? Jan, for example, agreed with that, saying yes, regulate end product deployments. don't regulate R&D with an arbitrary computer or model-sized threshold. Now, my point is that one still might disagree with this, but there's a lot more profitable conversation to be had with that specific conversation than the much more general one.

Anyways, it is a fascinating time in the evolution of this space and in the evolution of public attitudes around it. As I mentioned in my first episode about the Biden executive order a few days ago, it is very clear that this was not the end point but the starting gun when it comes to these debates rising to the biggest fears of the public conversation. And so, of course, what that means for you guys is now is a good time to get involved. If you have strong feelings about this, it's worth sharing them because a heck of a lot of people on both sides of this issue are going to claim to be doing things on your behalf, whether you

express those feelings or not. So if you've got them, you might as well actually share them. For us, that is going to wrap it for today's episode. Until next time, peace.