The AI Daily Brief: Artificial Intelligence News and Analysis - Forget Alignment, Here's Why Every AI Needs an Individual "Soul"

Episode Date: July 8, 2023

A reading of David Brin's "Give Every AI a Soul - or Else" https://www.wired.com/story/give-every-ai-a-soul-or-else/ ABOUT THE AI BREAKDOWN The AI Breakdown helps you understand the most important news and discussions in AI.    Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe   Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown   Join the community: bit.ly/aibreakdown   Learn more: http://breakdown.network/

Today on the AI breakdown, we're reading an argument for why AI needs individuation. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.network for more information. Hello, friends, welcome back to another AI breakdown. Today, we are doing a long reads, and this one is really interesting. This is a piece by author David Brin that was published in Wired. Now, David Brin is an interesting guy. He's written fiction and nonfiction, and the piece is titled, Give Every A.

A.I. A.I. A.I. A.I. Bings must say I am me. Obviously, we're getting a bit philosophical here when it comes to our AI safety and risk questions, but I thought it was an interesting take, and I think that it will provoke a lot of thoughts among you guys as well. Now, if you are enjoying the AI breakdown, I have one little request. If you are a Twitter user, I would love it if you would share your favorite episode from this week or the last couple weeks and just tag me either at NLW or at the AI breakdown pod and share it out so more people discover the show. Thanks in advance and let's get to the AI breakdown. Again, we're reading Give Every AI a Soul or Else by David Brin. Avon's in the field of artificial intelligence,

including architects of notorious generative AI systems like ChatGPT, now publicly express shared dread of terrible outcomes that might be wrought by their own creations. Many now call for a moratorium or pause in AI development, allowing time for existing nations and institutions to innovate systems of control. Why this sudden wave of concern? Amid the toppling of many cliched assumptions, we've learned that so-called Turing tests are irrelevant, providing no insight at all into whether generative large-language models G-LLMs or Gollums are actual sapient beings.

They will feign personhood, convincingly, long before there's anything or anyone under the skull. Anyway, that distinction now appears less pressing than questions of good or or bad or potentially lethal behavior. Some remain hopeful that emerging of organic and cybernetic talents will lead to what Reed Hoffman and Mark Andreson have separately called amplification intelligence, or else we might stumble into lucky synergy with Richard Brodigan's Machines of Loving Grace. But warriors appear to be vastly more numerous, including many elite founders of a new center for AI safety, who fretted about rogue AI misbehaviors from Erksum all the way to

existential threatening human survival. Some short-term remedies like citizen protection regulations recently passed by the European Union might help or at least offer reassurance. Tech pundit Yuval Noah Harari proposed a law that any work done by Gollums or other AI must be so labeled. Others recommend heightened punishment for any crime that's committed with the aid of AI, as with a firearm. Of course, these are merely temporary palliatives. Let's be clear about whether any moratorium will slow down AI advances in the slightest. As expressed succinctly by Caltech cyber scientist Yasser Abu Mustafa, if you don't develop this technology, someone else will.

Good guys will obey rules, the bad guys will not. Twas ever thus. Indeed, across the whole span of human history, just one method ever curb bad behavior by villains, ranging from thieves to kings and feudal lords. I refer to a method that never worked perfectly and remains deeply flawed today, but it did at least constrain predation and cheating well enough to spur our recent civilization to new heights and many positive sum outcomes. It is a method best described by one word, accountability. Those opining about synthetic intelligence today generally ignore lessons taught both by nature and by history.

Nature, because as Sarah Walker explains in Noima, similar patterns can be found in the rise of earlier life forms across 4 billion years. Indeed, generative AI might be compared to an invasive species now spreading without constraint into a novel and naive ecosystem. An ecosystem based on new kinds of energy flows, one that consists of the internet plus millions of computers and billions of impressionable human minds. and history because our own human past is rich with lessons taught by so many earlier tech-driven crises across 6,000 years. Times when we adapted well or failed to do so, e.g., the arrival of writing, printing, presses, radio, and so on. And again, only one thing ever limited predation by powerful humans exploiting new technologies to aggrandize their predatory power. That innovation was to flatten hierarchies and spur competition among elites in well-defined arenas, markets, science, democracy, sports,

courts, courts, arenas that were designed to minimize cheating and maximize positive some outcomes, pitting lawyer versus lawyer, corporation versus corporation, expert versus expert, rich dude. It never worked perfectly. Indeed, the method is always, as now, threatened with subordination by cheaters. But flattened reciprocal competition is the only thing that ever has worked. Reciprocal competition is both how nature evolved us and how we became the first society creative enough to build AI.

And if I sound like a scion of Adam Smith, sure. Smith despised cheater aristocrats and oligarchs, by the way. Might we apply to fast-emerging AI the same methods of reciprocal accountability that helped us tame the human tyrants and bullies who oppressed us in frivolous feudal cultures? Much will depend on the shape these new entities take, whether their structure or format is one that can abide by our rules, by our wants. Underneath all of the wrangling about how to control AI, we find three widely shared and seemingly contradictory assumptions.

One, that these programs will be operated by a few monolithic entities, e.g. Microsoft, Google, China, 2 Sigma, OpenAI. Two, that they'll be amorphously loose and infinitely divisible and replicable, spreading copies through every crack in the new cyber ecosystem. For a parallel, try that 1958 movie The Blob. Three, that they will coalesce into a super macro entity, like the infamous Skynet of Terminator movies. All of these formats and more have been explored in very good and many bad science fiction tales. I've done stories or novels featuring all of them. And yet, none of the three offers a way out of our current dilemma. How to maximize positive outcomes from artificial intelligence,

while minimizing the flood of bad behaviors and harms, we now see looming towards us at tsunami speed. Before looking for another way, consider what all three of the standard formats have in common. First, we needn't assume that these entities are yet autonomously conscious for them to be either productive or dangerous when used by human partners. We are already seeing harmful memes, counterfactual delusions, and even cult incantations generated on command

from both within the castle institutions, format number one, and outside the walls. In fact, one of the most worrisome applications is to help our existing human elites evade accountability. Perhaps these three assumptions come so naturally to mind because they resemble failure modes from history. Format number one is very much like feudalism, and number two is, of course, chaos. The third resembles despotism by a cruel master or absolute monarch. But those fearsome echoes of our primitive past may not apply as AI grow in autonomy and power. And so we ask again, How can such beings be held accountable, especially when their speedy mental clout will soon be impossible for organic humans to track? Soon only AIs will be quick enough to catch other AIs that are

engaged in cheating or lying. Um, duh. And so the answer should be obvious. Sick them on each other. Get them competing, even tattling or whistleblowing on each other. Only there's a rub. In order to get true reciprocal accountability via AI versus AI competition, the top necessity is to give them a truly separated sense of self or individuality. By individuation, I mean that each AI entity must have what author Wernervinge way back in 1981 called a true name and an address in the real world. As with every other kind of elite, these mighty beings must say, I am me. This is my ID and home route. And yes, I did that. Hence, I propose a new AI format for consideration. We should urgently incentivize AI entities to coalesce and to discreetly defined, separated individuals of relatively equal competitive

strength. Each such entity would benefit from having an identifiable true name or registration ID, plus a physical home for an operational referential kernel, possibly sole, and thereupon they would be incentivized to compete for rewards, essentially for detecting and denouncing those of their peers who behave in ways we deem insolubrious. And those behaviors do not even have to be defined in advance, as most AI mavens and regulators and politicians now demand. Not only does this approach farm-out enforcement to entities who are inherently better capable of detecting and denouncing each other's problems or misdeeds. The method has another added advantage. It might continue to function, even as these competing entities get smarter and smarter, long after the regulatory tools used

by organic humans and prescribed now by most AI experts, lose all ability to keep up. Putting it differently, if none of us organics can keep up with the programs, then how about we recruit entities who inherently can keep up, because the watchers are made of the same stuff as the watched. One percent, working on AI individuation is Guy Huntington, a quote, identity and authentication consultant who points out that various means of entity identification already exist online, though inadequate for the tasks looming before us. Huntington appraises a case study MedBot, an advanced medical diagnosis AI who needs to access patient data and performs functions that might change in seconds, but who must leave an accountable trail that humans or other bot entities might appraise.

Huntington discusses the practicality of registration when software entities spawn multitudinous copies and variants. He also considers ant-like-use-sociality, where subcopies serve a macro entity like workers in a hive. He assumes that some kind of major institution must needs be set up to handle such an ID registration system and that it can operate strictly a software. Personally, I am skeptical that a purely regulatory approach would work all by itself. First, because regulators require focus, widely shared political attention and consensus to enact, followed by implementation at the pace of organic human institutions, a sloth slash snail rate by the view of rapidly advancing cybernetic beings. Regulations can also be stymied by the free rider problem, nations, corporations, and

individuals, organic or otherwise, who see personal advantage in opting out of inconvenient cooperation. There is another problem with any version of individuation that is based entirely on some ID code. It can be spoofed. If not now, then by the next generation of cybernetic scoundrels or the next. I see two possible solutions. First, establish ID on a blockchain ledger. That is very much the modern-with-it approach, and it does secure in theory. Only that's the rub. secure according to our present set of human parse theories, theories that AI entities might surpass to a degree that leaves us cluelessly floundering. Another solution, a version of registration that's inherently harder to fool, would require AI entities with capabilities above a certain level

to have their trust ID or individuation be anchored in physical reality. I envision, and note, I'm a physicist by training, not a cyberneticist, an agreement that all higher-level AI entities who seek trust should maintain a sole kernel, SK, in a specific piece of hardware memory. Within what we equately used to call a particular computer. Yes, I know it seems old-fashioned to demand the instantiation of a program be restricted to a specific locale. And so, I am not doing that. Indeed, a vast portion, even a great majority of a cyber entity's operations must take place in far-dispers locations of work or play. Just as a human being's attention may not be aimed within their own organic brain, but at a distant hand or tool.

So, the purpose of a program's sole kernel is similar to a driver's license in your wallet. It can be interrogated in order to prove that you are you. Likewise, a physically verified and vouched for SK can be pinged by clients, customers, or rival AIs to verify that a specific process is being performed by a valid, trusted, and individuated entity. With that ping verification from a permanently allocated computer site, others, people, or AIs, would get reassurance they might hold that entity accountable, should it be accused or indicted or convicted of bad activity. And thus, malefactor entities might be adversarily held responsible via some form of due process.

What form of due process? Geez, do you think I am some super being who is capable of applying scales of justice to gods? The greatest wisdom I ever heard was uttered by Dirty Harry and Magnum Force. A man's got to know his limitations. So no, I won't define the courtroom or cop procedures for cybernetic immortals. What I do aim for is an arena within which AI entities might hold each other accountable, separately as rivals, the way that human lawyers already do today. And yes, answering Yuval Harari's dread of mass human manipulation by persuasive golems,

the solution for AI-driven mass meme hypnosis is for the mesmerizers. to be detected, denounced, and neutralized by others with the same skills. Again, competitive individuation at least offers a chance this could happen. Whichever approach seems more feasible, Huntington's proposed central agency or a looser, adversarially accountable arena, the need grows more urgent by the day. As tech writer Pat Schennell has pointed out, each hour that passes, new attack vectors are being created that threaten not only the tech used in legal identities, but also the governance, business processes, and end users, be they humans or bots. What about cyber entities who operate below some arbitrary level of ability?

We can demand that they be vouched for by some entity who is ranked higher and who has a sole kernel based in physical reality. I leave theological implications to others, but it is only basic decency for creators to take responsibility for their creations, no? This approach, demanding that AIs maintain a physically addressable kernel locus and a specific piece of hardware memory, could have flaws. Still, it is enforceable, despite slowness of regulation or the free rider problem, because humans and institutions and friendly AIs can ping for ID kernel verification and refuse to do business with those who don't verify. Such refusal to do business could spread with far more agility than parliaments or agencies can adjust or enforce regulations. And any entity who loses its SK, say through tort or legal process or else disavowal by the host owner of the computer, will have to find another

host who has public trust, or else offer a new revised version of itself that seems plausibly better. Or else become an outlaw. Never allowed on the streets or neighborhoods where decent folks, organic or synthetic congregate. A final question. Why would these super smart beings cooperate? Well, for one thing, as pointed out by Vincent Serf, none of those three older standard assumed formats can lead to AI citizenship. Think about it.

We cannot give the vote or rights to any entity that's under tight control by a Wall Street bank or national government, nor to some supreme Uber Skynet. And tell me how voting democracy would work for entities that can flow anywhere, divide and make innumerable copies? Individualation in limited numbers might offer a workable solution, though. Again, the key thing I seek from individuation is not for all AI entities to be ruled by some central agency or by mollusk slow human laws. Rather, I want these new kinds of ubermines encouraged and empowered to hold each other accountable, the way we already, albeit imperfectly do, by sniffing at each other's operations and schemes,

then motivated to tattle or denounce when they spot bad stuff, a definition that might readjust to changing times, but that would at least keep getting input from organic biological humanity, especially they would feel incentives to denounce entities who refuse proper ID. If the right incentives are in place, say rewards for whistleblowing that grant more memory or processing power, or access to physical resources when some bad thing is stopped, then this kind of accountability rivalry might just keep pace, even as AI entities keep getting smarter and smarter. No bureaucratic agency could keep up at that point, but rivalry among them, tattling by equals, might. Above all, perhaps those super genius programs will realize it is in their own best interest

to maintain a competitively accountable system, like the one that made ours the most successful of all human civilizations, one that evades both chaos and the wretched trap of monolithic power by kings or priesthoods, or corporate oligarchs, or Skynet monsters. The only civilization that, after millennia of dismally stupid rule by moronically narrow-minded, centralized regimes, finally dispersed creativity and freedom and accountability widely enough to become truly inventive. Inventive enough to make wonderful new kinds of beings, like them. There you are. This has been a dissenters' view on what's actually needed in order to try for a soft landing. No airy or panicky calls for a moratorium that lacks any semblance of a practical agenda,

neither optimism nor pessimism, only a proposal that we get there by using the same methods that got us here in the first place. Not preaching or embedding ethical codes that hyperentities will easily lawyer evade, the way human predators always evade the top-down codes of Leviticus, Hamarabi, or Kotama, but rather the Enlightenment approach, incentivizing the smartest members of civilization to keep an eye on each other on our behalf. I don't know that it will work. It's just the only thing that possibly can. All right, back to NLW here for just a very quick recap. Reading this, you can almost hear the pain cries of some of the AI safety people not being willing to surrender the premise

of this piece, which is that we have no power and no agency to slow down this inevitable push to create beings that are smarter than ourselves. So for the sake of our discourse and for the sake of actually trying to get something from the piece, I would encourage us to put aside the fact that there is still much room for debate around whether we actually want to proceed in such a way that creates super intelligent AI. But let's assume David is right and this is coming hell or water. What I think is valuable about this piece, and more than this piece, this approach to thinking about this issue, is that it at least gets us into the realm of the applied rather than the theoretical. Yes, it's predicated on a theoretical assumption in zooming out to this

period in which super intelligent AI has come to pass, but it's applied in the sense that this is a blueprint, or at least a starting point for what we could do if and when that happens. So if we're trying to map this against possible approaches to dealing with super-intelligent AI, we have on the one hand the cleanest strategy of just not building it. We have, on the other hand, approaches that call for alignment. Obviously, I'm reading this to you in a week where we have OpenAI announcing their new super-alignment team and a goal to align super-intelligence within four years. This is a more middle space.

And although he doesn't use this term, he invokes it when he invokes Adam Smith, this is sort of a market-based solution, although maybe not a market as we currently conceive them. I do think holding aside any specifics of whether David's particular approach would or wouldn't work, that exploring what the relationship and incentives between and among different AIs will be is a potentially profitable place to spend some time. To the extent that our worst risk scenarios involve coordination and agreement between diverse sets of AIs, how realistic do we think that is? And are there ways to intercede that make different AIs, if not combative with each other, at least competitive with one another in ways that lead to positive outcomes for humans.

When I read this, my overwhelming thought is that we are so barely scratching the surface of these questions that even for those who are mostly focused on that first approach to dealing with this problem, which is stopping before we develop super-intelligent AI in the first place, it's still probably worth thinking about plan B's of what happens and what we do if super-intelligent AI gets developed in spite of ourselves. Anyways, guys, I will wrap there. Plenty to chew on for your weekend. Thanks again for listening to the AI Break.

down. If you're enjoying it, please go check out the podcast version if you're watching on YouTube or the YouTube version if you're listening to the podcast. And until next time, peace.