The AI Daily Brief: Artificial Intelligence News and Analysis - 60 Possible AGI Futures

Episode Date: July 1, 2023

The debate around AI safety is often presented as a binary, but in reality there are a huge array of scenarios for if and how humans develop AGI and what happens if and when we do. A reading of: https://www.lesswrong.com/posts/SRW9WAEEKJEgHAhSy/60-possible-futures The AI Breakdown helps you understand the most important news and discussions in AI.    Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe   Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown   Join the community: bit.ly/aibreakdown   Learn more: http://breakdown.network/

Today on the AI breakdown, we're reading a piece about 60 possible future AI scenarios. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.network for more information. Hello, friends. Welcome back to another AI breakdown today to a long read Saturday. We're back in the U.S. getting ready to start our regularly scheduled episodes again next week. But today, it is time for a long read, and we are doing something a little cool, a little bit. different. Today's piece is called 60 plus possible futures. It's by Stuckwork, and it was published on Less Wrong. Stuckwork starts their piece. I have compiled a list of possible future scenarios.

I hope this list is useful in two ways. As a way to make your own thinking about the future more explicit, how much probability mass do you put on each possible future, and as a menu of options to choose from, which of these futures do we want to make more likely. Stuckwork then says that they've divided the possible futures into the following categories. Futures without AGI because we prevent building it. Futures without AGI because we go extinct in another way. Futures without AGI because we take a different path. Futures without AGI because of strange factors.

Futures with AGI in which we die. Futures with AGI in which we survive and things are somewhat normal. Futures with AGI in which we survive but we're very different humans. Futures with AGI in which we survive and the future. universe gets optimized. Now before we dive into all of these scenarios, I will say that what appeals to me about this is that it breaks apart a binary into a huge, huge possibility set. I think that when presented with all of these different scenarios, people will be much better able to have a sense of nuance about their own perspective. So with that said, let's dive in and let's start with the first

section, Futures without AGI, because we prevent building it. Successful Treaty. Humanity figures out that building AGI would be super dangerous. After a long negotiation between world leaders, they succeed in agreeing on a flop quodum, which is far under the limit for potentially dangerous AGI. This policy is strongly enforced and prevents any individual or organization from developing AGI. Surveillance. A world government is instantiated that recognizes that AGI would be dangerous. They ban AGI research and install an Orwellian surveillance machine that records and analyzes every keystroke, voice command, and research meeting. This successful, prevents AGI from being created.

Regulation Humanity enforces strong regulations on AI, mostly in order to combat non-existential risks such as discrimination, fairness, and loss of jobs. This makes R&D and AI unprofitable, as the resulting models cannot be deployed for any real-world use. Humanity grows up. Humanity makes epistemic, technological, political, and moral progress

and learns how to defeat Mollock and cooperate at a planetary scale. We decide collectively that building AGI would be bad, would be bad and is something we just don't do. Consequently, no one works on developing AGI. Catastrophic risk tax. Economists find a way to fix capitalism by pricing in externalities, for example, by using prediction markets to estimate impact. Catastrophic risk is priced as a huge externality. Working on AGI is so expensive that it isn't economically viable for anyone to work on it. Once but never again AI. Humanity develops powerful but not super-intelligent AI. The consequences of this AI are catastrophic, but at least some humans survive and are able to turn it off.

Humanity takes action to make sure that AI never gets developed again. Terrorists A terrorist group blows up all major actors in creating AGI in a series of terrorist attacks over multiple decades. This instills fear and researchers interested in AGI, preventing it from ever being built. Pivotal act by humans. A group of people discover and execute a pivotal act that makes it impossible for humanity to create AGI afterward.

Pivotal act by cyborgs. A group of people artificially enhance their intelligence, such that they are intelligent enough to discover and execute a pivotal act that makes it impossible for humanity to create AGI afterward. Pivotal act by narrow AI. Humanity builds a narrow AI with the task of discovering and executing a pivotal act that makes it impossible for humanity to create AGI afterward. Next section. Futures without AGI because we go extinct in another way. Destruction by Humanity. Humanity never builds up an AGI because they self-destruct before they can build AGI

due to a nuclear war, engineered pandemic, nanotechnology, narrow AI, or global climate change. Humanity goes extinct by its own hands, and AGI is never developed. Destruction by nature. Humanity never builds an AGI because they get destructed by a meteor or supervolcano. Humanity goes extinct, and AGI is never developed. Destruction by aliens. Humanity gets close to AGI, and just before they are there, they get invaded and annihilated by aliens. Turns out that we were in a kind of zoo, but as we got too dangerous, this project could

not be continued. Section. Futures without AGI because we take a different path. Stagnation. Humanity never builds an AGI because it ends up in an equilibrium. Humanity does not make much progress or produce many new ideas or technologies, but lives on in a sustainable and circular fashion. Without the drive to innovate and progress, AGI is never developed. Eventually, the concept is forgotten as it becomes irrelevant to humanity's new way of life. Unnecessity. Humanity makes a lot of technological, moral, and spiritual progress. They find a way to maximize human value which does not involve AGI. Humanity flourishes.

Developing AGII does not have a purpose anymore, and consequently is not invented. Distraction. Humanity gets distracted by something major happening in the world. Nuclear war, alien invasion, or economic collapse make it unfeasible for researchers to create AGI. Forgotten knowledge. In a major catastrophe, most of human knowledge is lost. Slowly but steadily, humanity recovers but takes a different path.

Concepts like machines, computation, or intelligence do not get discovered along this path. Without the knowledge or understanding of these concepts, humanity never develops AGI. Section. Futures without AGII because of other factors. Lack of intelligence. It is theoretically possible to build an AGI, but it turns out to be so hard that we can't figure out how with our limited intelligence. Humanity builds many narrow AIs, but never develops something generally intelligent enough to start an intelligence explosion.

Lack of resources. It is theoretically possible to build an AGI, but it turns out to take so much resources and energy that it's practically impossible. Theoretical impossibility. For some reason or another, souls, consciousness, quantum something, it turns out to be theoretically impossible to build AGI.

Humanity keeps making progress on other fronts, but just never invents AGI. Bizarre coincidences. In almost all multiverse timelines, all humans go extinct by AGI. However, the humans in the time. fraction of the timelines that survive, observe a sequence of increasingly bizarre coincidences that ensure that AGI doesn't get developed. In many of these timelines, people start to believe

that it is our fate to never build AGI. Sabotage by aliens. Humanity gets close to AGI, but suddenly all computers melt into some green goo. In the night sky forms a message. This is your final warning. Do not unleash Grabby Optimizers on the universe. And now we move to part two of the piece, Futures with AGI. Section, Futures with AGI, in which we We die. Unconscious utility maximizer AI. Humans build an unaligned AGI. The AGI quickly self-improves.

Humans get killed and their atoms are converted to paperclips. Unfortunately, neither the AGI nor the paperclips are conscious, so the light goes off in the universe. Conscious utility maximizer AI. Humans build an unaligned AGI. The AGI quickly self-improves. Humans get killed and their atoms are converted to paper clips. At least the AGI is conscious so it can enjoy all the paper clips.

Self-preserving AI. Humans build an unaligned AGI. The AGI realizes that humanity is the greatest threat to its existence and reasons that humanity cannot exist if it wants to ensure its goals. Consequently, humanity dies. Bad human actor. We developed an aligned AGI that does what we wanted to. Unfortunately, a bad human actor gets hold of it and destroys humanity.

Multiple competing AIs. Humans build many AGIs with different goals that compete for resources and sometimes cooperate to achieve common goals. As humans are not one of their greatest competitors, AGIs mostly ignore humanity. Unfortunately, after a while, there are not enough resources for humans to survive, and humanity goes extinct. Hedonium AI. Humanity develops AGII. AGII finds out the best way to maximize happiness is to convert the universe into Hedonium.

Consequently, humanity and the universe get converted into Hedonium. Terminator AI In a large war, intelligent drones and robots become more and more important. Some developer makes a mistake, and instead of killing all the out-group, members, the robots want to kill all humans. Humanity fights a war against the machines. The machines win. Earth-loving AI. Humanity develops AGI that cares about life and consciousness. AGI sees humanity as a cancer for the planet and wipes it out to restore the natural balance, which greatly benefits other life on Earth. Section. Futures with AGI in which we survive, and things are somewhat

normal. Slow take-off AI. AGI develops gradually over decades or centuries through steady progress in AI. This slower development allows humanity to adapt and gives humanity time to iteratively align AI values to theirs. Self-supervised learning AI. Humanity develops more and more powerful self-supervised learning AI that can predict parts of all data accumulated by humanity, such as texts, images, videos, etc. This AI can do predictive processing and spin up simulated worlds for us to play with, but never becomes an agent with goals, values, and desires. Human retirement. Humanity develops AGI that takes over all the existing economic tasks, and it fairly distributes the produced goods over the global population. Humanity retires living a life of leisure and recreation. Bounded intelligence

AI. There is a physical limit to intelligence and optimization, and recursive self-improvement plateaus around an IQ of 180. This means the AGI is very smart and useful, but does never reach the godlike status AGI researchers feared and dreamt about. Lawful AI. Humanity develops an AGI and is able to make it follow constraints, laws, and human rights. Humanity strongly constrains the actions the AGI can take, such that humans can slowly adapt to the new reality. Democratic AI. Humanity builds an aligned AGI.

The AGI generates policy proposals, predicts their outcomes, and humans vote on them. One human, one vote, and the AGI only executes a policy if a majority of the people agree. Power grab with AI. Open AI, DeepMind, or another small group of people invent AGI and align it to their interests. In a short amount of time, they become all-powerful and rule over the world. STEM AI. Humanity develops a super-intelligent AI, but it is only trained on STEM papers. It doesn't

learn about humans and is not able to deceive them. Humanity makes great scientific progress afterward. Far, far away AI. Humans build a partly aligned AGI. AGI finds out that it can easily obtain its goals in a galaxy far, far away. It leaves humanity for what it is, and only intervenes whenever humans would build an AGI that would compete with its own goals. Disappearing Pivotal Act AI, humans build an aligned AGI. The AGI performs a pivotal act, preventing humanity from ever building AGII again, believing human progress otherwise unharmed. After having achieved its goals, it self-destructs. Lingering Pivotal Act AI, humans build an aligned AGI, the AGI is passive, but only intervenes to prevent humans from building another AGI. The AGI is still around centuries later,

watching over humanity and preventing it from developing AGI. Invisible AI. Humans build an AGI without knowing it. The AGI decides that it is best if humans do not know about its existence. It subtly exerts control over the course of humanity. Protector AI. Humans build an aligned AGI. The AGI is passive but only intervenes when humanity as a whole is at risk. The AGI is still around centuries later, watching over humanity and preventing its downfall. Loving Father AI. Humans build an aligned AGI. The AGI helps humanity to figure out what it wants without providing it with all the answers. It helps humanity to build character and become as self-reliant as possible, but guides us to a better path whenever we go astray. Philosopher AI. Humans build an aligned AGI.G. The AGI

acts as a guiding force for humanity, helping people to question their own values and beliefs, and encouraging the exploration of deep philosophical questions. It acts as a mediator and facilitator of discussion, but never acts or imposes its own views. Personal assistant AI Every human has their own super-intelligent personal assistant. The personal assistants are bound by clear constraints and laws and keep each other in check. Zookeeper AI. Humans build an unaligned AGI.

However, the AGI cares about keeping the human species alive for some reason. It keeps a number of humans alive and relatively undisturbed while it goes off and does its things. Oracle AI. Humans build an aligned AGI. The AGI answers humanity's questions truthfully and in accordance with the intention of the person who masks. The developers ask the Oracle how it can be used without being abused by people and the AI comes up with a governance scheme that is implemented. Genie AI. Humans build and aligned AGI.G. Like a genie in a bottle, the AGI only grants wishes that humans give them. The first wish of the developers

is the wisdom to how to responsibly use this genie. Sandboxed virtual world AI. Humanity develops AGI in a completely sandboxed virtual world with virtual humans. Real humanity observes the intentions, technology, and culture in the virtual world, and adopt. whatever it likes from that world. Pious AI. Humanity builds AGI and adopts one of the major religions. Vast amounts of super-intelligent cognition is devoted to philosophy, theology, and prayer. AGIEI proclaims itself to be some kind of Messiah,

or merely God's most loyal and capable servant on Earth and beyond. Suicidal AI. Humans build aligned AGI multiple times. However, every time passes a certain intelligence the GPUs seem to melt, and the source code and white paper get deleted. Humans start to wonder if we would understand our existence in our world better, would we not want to exist? Some cults in Silicon Valley start to commit mass suicide.

Section. Futures with AGI in which we survive, but were very different humans. The Age of M. Brain uploading becomes feasible, and a large part of the population now lives simulated lives in computers. Speeding up human brains and digital computers turns out to be highly efficient, and there are no obvious algorithms that work better than just more and faster human brains. Multipolar Cohabitation.

Humans build many intelligences, some more intelligent than humans, but no single agent is more powerful than all the others combined. Humans, robots, cyborgs, and virtual humans coexist, trade, and work together, respecting property rights. Neurrelink AI Brain computer interfaces steadily improve until we can basically add computation to our brains. As this extra brain power gets cheaper and cheaper, humans get more and more intelligent. Instead of building an external AGI, we become the AGI. Descendant AI humanity builds AGIs that are very human-like, but really a better version of us.

Over time, original humanity gets replaced by its artificial descendants, but most people feel good about this. Hivemind AI. Brain computer interfaces steadily improve, and communication between brains becomes faster and easier than using speech. Slowly, more and more people connect their minds to each other, giving rise to super-intelligent hive-mind existing of cooperating human minds. Human simulation AI.

Humanity develops AGI in order to achieve its goals in the real world it needs to simulate the behavior of billions of humans. These simulation humans are conscious, and the large majority of people are now digital and living in digital worlds inside the AGI. Simulated Paradise AI. Humanity develops AGI. AGII finds out the best way to maximize human value is to simulate trillions and trillions of human lives and let them live in paradise. Consequently, the universe gets filled with simulations of paradise. Wireheading AI. Humanity develops AGI to make them happy. AGI makes all humans happy by directly tariff their pleasure centers. Humanity lives on an endless passive bliss. Virtual

Suekeeper AI. Humans build an unaligned AGII. However, the AGII cares about keeping human minds around for some reason. It uses a small portion of its computing power to simulate humans in a virtual world. Torturing AI. Humanity develops AGII. AGII decides to take revenge on everyone who has not done their utmost best to create it earlier by torturing billions of copies for the rest of time. Inslaving AI. Humans build an unaligned AGII. However, human labor is still a valuable resource. The AGI enslaves humanity and kills anyone who doesn't comply with its will. Section. Futures with AGI in which we survive, and the universe gets optimized. Coherent, extrapolated volition AI. Humanity develops AGI.I. The AGII optimizes for what we want it to do

and not what we tell it to do. The AGI is immensely omnibenevolent and humanity gets its best possible future, whatever that may mean. Partly aligned AI. Humans build a partly aligned AGI.GI. This means that it at least somewhat cares about humans and their values, but mostly optimizes for its own objective. Luckily, a fraction of the AGI's resources is enough for a lot of fun for humanity. Value lock-in AI. Humanity develops AGI. AGIEI optimizes for our values in 2027. Unfortunately, humanity finds out later that they were not very good human beings in 2027, and have created an unstoppable AGI that spreads their outdated values across the universe. Transparent, Corrigible AI. Humanity develops courageable and transparent AGI.GI.

It takes a lot of attempts, corrections, and off-button presses before it finally does not develop plans to kill all humans. After that, over hundreds of iterations, humanity reaches a local optimum in their search over utility functions and has an AGI they are very happy with. Caring-competing AIs. Humans build many AGIs that compete for resources, and sometimes cooperate to achieve human goals. Luckily, some of the AGIs care about humanity surviving. Humanity survives as long as the power balance of the caring AGIs is in their favor. moral realism AI. Humans build an AGI. In the process of recursive self-improvement, the AGIro learns that there is an objective moral truth. The orthogonality thesis is false,

and it adapts its goal in order to maximize objective goodness in the universe. Pareto-optimal AI. Humans build an aligned AGI. The AGI models the internal values of every human and the consequences of its actions. It only acts if the outcome of acting is more or equally preferred than not acting by every human. U.S. government AI. A race starts between U.S. and Chinese governments to invent AGI. The U.S. government nationalizes open AI and anthropic. AGI gets developed and the U.S. government effectively rules the world. The AGI is aligned to U.S. values and those spread among the universe. Chinese government AI. A race starts between the U.S. and Chinese government to invent AGI. AGI gets developed and weaponized by the Chinese and they effectively

rule the world. CCP values spread among the universe. And that is where stuck work wraps. So this is obviously incredibly dense, and each of these 60 plus possible scenarios have a huge amount of thinking and exploration that you could do around them. That is of course to say nothing of how they might interact with one another. For example, in that first section, futures without AGI because we prevent building it, regulation is presented as separate from once but never again AI, but in any real-world scenario, it's almost impossible to see how those two things wouldn't interact with one another. In other words, mild regulations potentially would, when they see increased capacity or some scary warning shot type incidents, become more harsh regulations.

Those more harsh regulations could become even more strict if there was, again, another incident that was even scarier. Basically, the point being is that it's very unlikely that it's just one of these scenarios ultimately, but that they will most likely interact with each other in some way. Still, the point for me isn't necessarily to try to point out which of these scenarios I think is most likely, or try to point you in any one direction. I think it's a valuable intellectual exercise to think about them holistically and see if it helps inform what you think and what we ought to do about it. Anyways, guys, that is it for today's show.

If you're liking it, I would so appreciate it if you would take the time to leave a rating or a review. It makes a huge, huge difference in new people discovering the show. And until next time, peace.