The AI Daily Brief: Artificial Intelligence News and Analysis - A Critique of Marc Andreessen's "Why AI Will Save The World"

Episode Date: June 17, 2023

Last week's Long Reads was Marc Andreessen's "Why AI Will Save the World." This week we read Dwarkesh Patel's "Contra Marc Andreessen on AI."   The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI breakdown, we're reading a pretty definitive response to Mark Andreessen's seminal essay that we shared last week, why AI will save the world. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Like, subscribe and share and go to Breakdown.network for more information. Hello, friends. Welcome back to another AI long read, in this case, an AI long read Saturday. This week, I have not lost my voice and so, as exciting as the AI experiment was, you've actually got to be a real read. the real me this time. That said, we are staying on the themes from last week. You can probably tell at this point that it's my goal to give you lots of different perspectives when it comes to questions of AI safety and risk. Given where he sits in the technology space and society more broadly,

anything that Mark penned about this particular issue, and especially something that he is so clearly trying to make a strong and definitive statement, was going to get picked up, it was going to get tons and tons of attention. There have been a few different responses, but the one of the one I've seen recommended most to me as a good follow-up to read comes from Dwarkesh Patel, the host of the Lunar Society. The piece is called Contra Mark Andresen on AI and was published on Dworkesh's blog. Dwarkeh writes, Mark Andresen published a new essay about why AI will save the world. I had Mark on my podcast a few months ago and he was, as he is usually, very thoughtful and interesting. But in the case of AI, he fails to engage with the worries about AI misalignment. Instead, he

substitutes aphorisms for arguments. He calls safety warriors cultists, questions their motives, and conflates their concerns with those of the woke trust and safety people. I agree with his essay on a lot. People grossly overstate the risks AI poses via misinformation and inequality. Regulation is often counterproductive and naively regulating AI is more likely to cause harm than good. It would be really bad if China outpaces America in AI. And technological progress throughout history has dramatically improved our quality of life. If we solve alignment, we can look forward to material and cultural abundance. But Mark dismisses the concern that we may fail to control models,

especially as they reach human level and beyond. And that's where I disagree. Section, it's just code. Mark writes, quote, My view is that the idea that AI will decide to literally kill humanity is a profound category error. AI is not a living being that has been primed by billions of years of evolution to participate in the battle for the survival of

the fittest, as animals are, and as we are. It is math, code, computers, built by people, owned by people, used by people, controlled by people. The idea that it will at some point develop a mind of its own and decide that it has motivations that lead it to try to kill us is a superstitious handwave. End quote. Dwar Keshe goes on. The claim that you will completely control any system you build is obviously false, and a hacker like Marx should know that. The Russian nuclear scientists who built the Chernobyl nuclear power plant did not want it to melt down. The biologists at the Wuhan Institute of Virology didn't want to release a deadly pandemic, and Robert Morris didn't want to take down the entire internet.

The difference this time is that the system under question is an intelligence capable, according to Mark's own blog post, of advising CEOs and government officials, helping military commanders make better strategic and tactical decisions, and solving technical and scientific problems beyond our current grasp. What could go wrong? I just want to take a step back and ask Mark, or those who agree with him, What do you think happens as artificial neural networks get smarter and smarter? In the blog post, Mark says that these models will soon become loving tutors and coaches,

frontier scientists and creative artists, that they will, quote, take on new challenges that have been impossible to tackle without AI, from curing all diseases to achieving interstellar travel? How does it do all this without developing something like a mind? Why do you think something so smart that it can solve problems beyond the grasp of human civilization will somehow totally be in your control? Why do you think creating a general intelligence just goes, well by default. Saying that AI can't be dangerous because it's just math and code is like saying

tigers can't hurt you because they're just a clump of biochemical reactions. Of course it's just math. What else could it be but math? Mark complains later that AI warriors don't have a falsifiable hypothesis. And I'll address that directly in a second. But what is his falsifiable hypothesis? What would convince him AI can be dangerous? Would some wizard have to create a shape-shifting intelligent goo in a cauldron? Because short of magic, any intelligence we build will be made. of math and code. As a side note, I wonder if Mark has tried to think through why the pioneers of deep learning, including multiple Turing Award winners, the CEOs of all the big AI labs, and the researchers working most closely with advanced models think AI is an existential risk. Surely he doesn't

think he could just hand this essay to them and they'd say, oh, it's just code. Why didn't I think of that? What was I even worried about? AI doesn't even need to have bad goals in order to be disastrously dangerous. It can simply pursue a goal we give it in a way we never intended. Even within human societies optimizing on a seemingly banal goal has led to awful consequences. Maximize equality leads to communism causing uniform suffering and death, expanding your nation's fear of control, leads to leaders starting a war that kills millions of people. With advanced AI, the situation is much worse. We're talking about something that is potentially more powerful than any human. It doesn't start off with the hundreds of nuanced and inarticulable

values embedded in human psychology as common sense. We laugh at toy examples like when tasked with solving depression, the AI develops an advanced opioid and hooks everyone up to an IV. The AI is not doing this because it's stupid, but because it doesn't automatically have all of our contradictory and often counterbalancing values. We live in a world where the only intelligent optimizers are other humans that basically think like us and want similar kinds of things. So it becomes hard for us to imagine what it would be like to have a different type of mind come online, and how strange and destructive its methods for executing our commands might be. Section. The Testable Hypothesis

from Marx essay, quote, My response is that their position is non-scientific. What is the testable hypothesis? What would falsify the hypothesis? How do we know when we are getting into a danger zone? These questions go mainly unanswered apart from you can't prove it won't happen. End quote.

And back to Dorcas. We're already in a situation where Sidney Bing threatened to blackmail, bribe and kill people, albeit in acute and endearing way. And GPT4 told the user how to manufacture biological weapons, and we're still far from human-level intelligence. Obviously, Sidney Bing can't hurt you given its current capabilities, but do you feel confident that whatever caused it to go rogue won't happen on a much larger and more destructive scale, with a model that is much smarter?

If so, why? A technologist and venture capitalists like Mark should be the most open to the possibility that there will be big, unprecedented changes whose possibility we can't immediately verify. When Mark created the first web browser in 1993, what was his immediately falsifiable thesis about the internet? Of course, you could just build GPT-8 and see if it does exactly what we want it to do in exactly the way we want it. We can then hope that gradient descent on predicting the next token somehow, by default, creates the drive to act in the best interest of humanity. Though evolution optimizes our ancestors over billions of years to leave behind the most possible copies of our genes,

and here we are using condoms and pulling out. We're hastily putting together a new airplane. Its smaller prototypes sometimes fall out of the sky in weird ways that we don't really understand. Would you like to be the test pilot? I wonder if Mark actually has asked a bunch of people working on technical alignment, whether they have a falsifiable hypothesis, or if he's just assuming that the answer must be no. I posted Mark's question in a group chat, and just off the cuff, Tristan Hume, who works on interpretability alignment research at Anthropic, supplied the following list, edited for clarity.

One, I'd feel much better if we solved hallucinations and made models follow arbitrary rules in a way that nobody succeeded in red-teaming, in a way that wasn't just confusing the model into not understanding what it was doing. Two, I'd feel pretty good if we then further came up with and implemented a really good supervision setup that could also identify and disincentivized model misbehavior to the extent where me playing as the AI couldn't get anything past the supervision. Plus evaluations that were really good at eliciting capabilities and showed smooth progress and only mildly superhuman abilities. And our data centers were secure enough I didn't believe that I could personally hack any of the major AI companies if I tried. Three, I'd feel great if we

solve interpretability to the extent where we can be confident there's no deception happening, or develop really good and clever deception evals or come up with a strong theory of the training process and how it prevents deceptive solutions. Section. The only thing that can stop a bad guy with an LLM. From Mark's essay, quote, In short, AI doesn't want, it doesn't have goals,

it doesn't want to kill you because it's not alive. An AI is a machine. It's not going to come alive anymore than your toaster will. End quote. Back to Dorcasch. This argument that AI is just like a toaster, contradicts the entire rest of the essay, which argues that AI will be so powerful that it will save the world. The very reason why AI will be capable of creating stupendous benefits for humanity, if we get it right,

is that it is much more powerful than any other technology we have built to date. If it was just like a toaster, it couldn't, quote, make everything we care about better. Even if it does exactly what a user wants it to do, how do you ensure that GPT7 doesn't teach a terrorist how to synthesize and transport a few dozen new deadly infectious diseases? Are you saying we could prevent such output? If so, what technique for reliably controlling GPT-7's behavior have you discovered which OpenAI will surely pay you a fortune to reveal? And even if you had such a technique, I'm sure they exist in principle,

otherwise alignment would be hopeless, how do you make sure everyone who builds advanced models implements them? After all, Mark advocated against mandated evaluations, standards, and other regulations at the end of the essay. Later, he writes, quote, I said we should focus first on preventing AI-assisted crimes before they happen. Wouldn't such prevention mean banning AI? Well, there's another way to prevent such actions, and that's by using AI as a defensive tool. The same capabilities that make AI dangerous in the

hands of bad guys with bad goals make it powerful in the hands of good guys with good goals, specifically the good guys whose job it is to prevent bad things from happening. End quote. Back to Dwar Keshe. The tech tree of synthetic biology or cyber warfare isn't necessarily organized according to catchy aphorisms. How exactly does a good guy with a bioweapon stop a bad guy, the bioweapon. While we might hope that new technology must always help defense more than offense, there's no metaphysical principle that says it must be so. In fact, even in theory, offense would be more advantaged. Rockets are much easier to build than missile defense systems. Synthesizing and transporting a single new virus into the U.S. is much easier than implementing

metagenomic sequencing across every single airport, harbor and border crossing, official and unofficial. The reason a terrorist hasn't done something 100x worse than 9-11 is not because the tech has in principle ruled out such attacks, in that our defense has progressed faster than their offense, rather there just aren't that many very smart people who want to cause mass destruction. I don't know what happens when only a modestly smart person can ask a chatbot for the recipe to cause one million deaths, but I see no reason to be confident that the outcome will be positive. Section. Regulation. Quoting Mark. This causes some people to propose, well, in that case, let's not take the risk. Let's ban AI now before this can happen. Unfortunately, AI is not some

esoteric physical material that is hard to come by, like plutonium. It's the opposite. It's the easiest material in the world to come by, math and code. The AI cat is obviously already out of the bag. You can learn how to build AI from thousands of free online courses, books, papers, and videos. And there are outstanding open source implementations proliferating by the day. AI is like air. It will be everywhere. The level of totalitarian oppression that would be required to arrest that would be so draconian, world government monitoring and controlling all computers, jackbooted thugs and black helicopters seizing rogue GPUs, that we would not have a society left to protect. End quote. Back to Dwar Keshe. Regulation may not be the answer and may indeed

make things worse, as in my view it has done in many other areas. I also worry that regulations will ban naughty thoughts in the name of AI safety in a way that does nothing about the larger alignment problem. But dismissing the legitimate worries about AI risk makes this more like, not less. After all, if people think that making GPD 4 woke and making sure GPT7 doesn't release a bio weapon are the same thing, and we want to make sure GPD 7 doesn't release a bio weapon, then I guess we got to make GPT4 woke. The best LLMs today require hundreds of millions of dollars to train, and something close to human level will probably take billions of dollars using the current approach. The idea that we'll need to seize the 3070s in every teenager's gaming rig

in order to evaluate and monitor the most advanced models is obviously ridiculous. Identifying and monitoring training runs that require tens of thousands of the newest server GPUs will require no more totalitarian oppression than the International Atomic Energy Agency requires to keep track of all the world's nukes. You cannot argue, as Mark has repeatedly done, that regulations are so destructive that they have paralyzed entire sectors of the economy, like energy, medicine, education, law, and finance, and at the same time argue that powerful technologies are so inevitable that trying to constrain them would be hopeless anyways. Regulations either have effects good or bad, or they don't.

Section. Motives. From Mark Andreessen's essay. Quote, In fact, these Baptists' position is so non-scientific and so extreme, a conspiracy theory about math and code, and is already calling for physical violence, that I will do something I would not normally do

and question their motives as well. Specifically, I think three things are going on. First, recall that John von Newman responded to Robert Oppenheimer's famous hand-wringing about his role creating nuclear weapons, which helped end World War II and prevent World War III with, some people confess guilt to claim credit for the sin. What is the most dramatic way one can claim credit

for the importance of one's work without sounding overly boastful? Dot, dot, dot. Second, some of the Baptists are actually bootleggers. There is a whole profession of AI safety expert, AI ethicist, AI risk researcher. They are paid to be doomers, and their statements should be processed appropriately. Third, California is justifiably famous for our many thousands of cults.

And the reality, which is obvious to everyone in the Bay Area, but probably not outside it, is that AI Risk has developed into a cult, which has suddenly emerged into the daylight of global press attention and the public conversation. This cult has pulled in not just fringe characters, but also some actual industry experts and not a small number of wealthy donors, including, until recently, Sam Bankman-Fried. And it's developed a full panoply of cult behaviors and beliefs. It turns out that this type of cult isn't new.

There's a long-standing Western tradition of millinarianism, which generates apocalypse cults. The AI Risk cult has all the hallmarks of a millenarian, Apocalypse cult. End quote. Back to Dwar Keshe. Mark is right here. The original AI safety thinkers are weirdos.

The AI ethics people are mostly grifters. And hyping up AI risk may plausibly be a way to give your AI work a larger significance in the grand scheme of things. So what? This is the obvious ad homonym. The idiosyncrasies of the people who develop a set of ideas are irrelevant to the debate around those ideas themselves. Every group of people that has done something important in history has been weird.

Normal people usually don't do cool shit. Are you throwing away your math textbooks because many of history greatest mathematicians were mentally ill? Will you smash your iPhone because the people who started the IT revolution were eccentric, autistic, and culty? It would be intellectually dishonest of me to write a section about how Mark is arguing against AI warriors because he wants to discourage regulations and industry standards that would harm his investments. I'm confident Mark genuinely believes what he says, but even if I'm wrong, who cares? The debate must always be about the ideas, not the people. Oppenheimer was correct that he was involved in a project that changed the course of human history.

Maybe Mark is right in that Oppenheimer was trying to aggrandize himself. Again, so what? If we're going to play a game of guilt by affiliation with Sam Bangman-Fried, might I point out that one of the world's largest crypto investors probably has fewer degrees of separation between themselves and Sam than a Turing award winner? To be clear, I think it would be ridiculous to say that SBF is a stand on Mark. But it's even more bizarre to name-drop SBF to dismiss AI Risk Warriors. Section, China from Mark's essay. China has a vastly different vision for AI than we do. They view it as a mechanism for authoritarian population control full stop. They're not even being secretive about this. They are very clear about it. And they are already pursuing their agenda. And they do not intend to limit their AI strategy to China. They intend to proliferate it all across the world.

Everywhere they are powering 5G networks, everywhere they are loaning built-in-road money, everywhere they are providing friendly consumer apps like TikTok that serve as front-ends to their centralized command and control AI. The single greatest risk of AI is that China wins global AI dominance, and we, the United States and the West, do not. not. I propose a simple strategy for what to do about this. In fact, the same strategy President Ronald Reagan used to win the first Cold War with the Soviet Union. We win, they lose. End quote. Back to Dwar Keshe. A world in which the CCP develops AGI first is a very bad world. But a lot of

the harm from China developing AI first comes from the fact they probably will give no consideration to alignment issues. So if we developed AI in the same cavalier attitude in an effort to beat them, and humanity's future is thwarted, what have we gained? Thankfully, we have a lead on China right now. We should maintain that lead and use it to dramatically increase investment in technical alignment. I don't understand how we get from develop AI tech faster to China is prevented from abusing AI. Do we use AI to fight a war against the CCP that takes them out of power and prevents an internal Chinese panopticon? If not, how exactly does the pace of AI progress in America change about what the CCP will do in China? At the end of the essay, Mark

advocates for free proliferation of open source AI. If the real threat from AI is the real threat from AI is that China catches up to American companies, then surely this is a bad idea. When we were competing with the Soviet Union, we wouldn't have allowed our advanced weapons technology to be leaked to the enemy. If AI is the most powerful tool in a conflict with China, then shouldn't we be similarly worried about sharing AI secrets with China through open source? Note that in terms of concrete ideas, there's a lot of overlap between China Hawks and AI safety warriors. They both want to prevent China from hacking into the AI labs and stealing weights. And I'm sure AI safety warriors were quite pleased by Biden's restrictions on China's

access to advanced semiconductors and the equipment used to make them. Export controls can harm China's ability to compete with us on AI without increasing the risk from more advanced models. Final section. AI isn't social media. People like Mark are still pretty hung up on the social media culture war. He had a front row seat to this battle for over a decade. Mark is on the board of meta and his firm A16Z was both an early investor in the original Twitter and now in Elon's takeover. Especially since 2016, there's been a huge moral panaceous. around misinformation, hate speech, and algorithmic bias. Social media companies did not stand their ground

and explain why these concerns were exaggerated or misplaced, at least until recently. Instead, they repeatedly conceded to the demands of activists, governments, and the media. It seems Mark wants to draw a line in the sand this time. We will not give in to your hysterical demands to control our technology. But a website where you post life updates, memes, and news articles, is just not the same thing, as a steadily improving general intelligence capable of doing science and tech.

And you're missing the story, you insist on thinking about the development of this current infant alien intelligence using your decade old culture war lens. Imagine the first hunter gatherers to plant some seeds across a patch of land. In the grand scale of human history, the creation of AI is as significant as the agricultural revolution. Forcing your perspective on AI through the current trust and safety wars is like the proto-agriculturalists killing each other over whether they dedicated their first harvest to the tree spirit or the fire deity. All right, back to NLW here. For just a quick round, up. Now, I'm not going to go point by point on whether I think Mark is more correct or Dwar

Keshe is more correct. Again, this is part of a longer exploration of the questions of AI safety and AI risk, and my goal is to be a forum that really gives lots of perspectives as long as they are engaged with in good faith and thoughtful consideration. What I will say, for a little bit of contextualization, because it's something I've been thinking about a lot this week, is I think that it may be important in this particular set of discourses to understand or to at least ask the question of what the goal of Mark's piece might have been. The more that I've spent time with it, the less convinced I am that it was meant to be his definitive statement about AI safety concerns. And instead, I get more convinced that it was meant as a very, very specific response to the way in which

AI risk has found its way into the media discourse specifically over the last month or so. We've talked a lot about how the folks who care about AI safety have been effectively screaming into a void for years, only to now find themselves at the center of an extremely mainstream topic. Every day there is some shocking new headline about how 42% of CEOs think that AI could end the world within five to 10 years or take your pick, right? There is always something like that in the news now. And I think it's reasonable to ask, even if you are an AI safety advocate, to what extent those headlines are driven by the media establishment thinking that this is an important discourse, which in many cases it will have been, versus them just being absolutely

sensational attention-grabbing, click-baiting, engaging headlines. And I think that that question does matter. On the one hand, you could argue that who cares? As long as the conversation is happening, it doesn't matter why the media has decided to take it up as a topic. And to some extent, that may be true. However, the risk, and I do think it is a profound risk, is that people become naturally skeptical or wary of doom-laden discourse. There have been a lot of things over the last few years that have been presented to people as the likely or potential end of humanity.

COVID-and-pandemic dialogue falls into this, as does even more notably climate change. There has been a never-ending increase and ratcheting up of the rhetoric around climate change for again what might be completely legitimate reasons of frustration that people aren't getting with the picture. The problem is that the net effect of that is that people naturally get tired. They don't want to think about their doom all the time. And so it becomes easier to inoculate oneself against those things as sensationalism, the diatribs of people who just want something from you, then as a real concern. It's a classic

boy who cried wolf, except the problem is it's not even the AIC. safety people crying wolf, it's that they have to deal with the fact that mainstream audiences feel like others have cried wolf for a long time. So there is this real question of the media's intentions and what happens if this just becomes fodder for headlines and will they turn off it because people stop engaging. But in the short term, there's no doubt that the media narrative has shifted dramatically. And to me, Entresen's YAI will save the world reads at least in a meaningful part, like a response not to Eliezer Yudkowski and Jeffrey Hinton and Max Tagmark, but to the New York Times, to the Washington Post, to CNN.

Now, I do think that Dworkesh addresses this at least a little bit at the end of the show, where he discusses that he worries that this might be fighting the social media battle one more time. There is no doubt that Silicon Valley and particularly the New York Times, but really the media establishment, have been in an influence battle for coming on five years now, that was especially ratcheted up during COVID. It is always the case that the beginning of new battles are fought like the end of old ones, and so to some extent this is inevitable. But I do think that the AI safety conversation deserves its own discourse.

That is, as much as we can do it, reflective of its own reality, rather than just echoes of technology issues passed. Anyways, guys, I hope that this was a nice rounding out perspective on that piece that we read last week. Thanks again to Dwar Keshe for writing it. There are a lot of people in this space who maybe had disagreements with it but weren't willing to or weren't able to put them together in such a coherent, clear way. And of course, thanks to you guys, as always, for listening. I hope you are having a great weekend. And we'll be doing a version of the weekly recap tomorrow, although this week, it'll be about the most interesting research that we didn't have a chance to talk about yet.

Until next time, peace.