The AI Daily Brief: Artificial Intelligence News and Analysis - An AI Drone Killed Its Operator In A USAF Simulation -- Except That's Not Actually What Happened

Episode Date: June 2, 2023

A bogus story about an AI simulation gone wrong made headlines around the world, until the progentior of the story said it was actually just a thought experiment. Oops. On the Brief, a look at Neuralangelo, a new NeRF/Photogrammetry type technology making 3D geometry from 2D videos. Check out The Cognitive Revolution The perfect AI interview complement to The AI Breakdown  https://link.chtbl.com/TheCognitiveRevolution   The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI breakdown, an Air Force simulation sees an AI drone kill its own operator. Or does it? Before that, on the brief, an incredible new technology that can turn your 2D videos from your iPhone into 3D geometry. The Breakdown is a daily podcast and video about the most important news and stories in AI. Like, subscribe and share, and go to Breakdown.network for more. This amazing new technology can take iPhone videos or any other 2D video and turn it into 3D structures. and geometry. Welcome back to the AI breakdown brief, all the AI headline news you need in five minutes or less. We kick off today with an incredible new advance in AI for 3D modeling called Neurolangelo. Dr. Jim Fan tweeted yesterday, to give you a sense of how fast AI for 3D modeling

is advancing, the field went from the left, original Nerf reconstructed mesh to write Neuralangelo in three years. Transporting reality into high fidelity simulation is no longer a pipe dream. Now, you might have heard me talk about Nerfs or Neural Radiance Fields before. This is a technology that allows you to convert two-dimensional video into a three-dimensional representation. The video that's on the screen came from Nick St. Pierre shooting a 2D video with his iPhone and then turning it into a Nerf with Luma Labs. Neuralangelo is a new approach in that space from Nvidia that takes two-dimensional video representations of a space or an object and turns them into incredibly detailed 3D models.

Bilowal Seedhu explains it, calling it photogrammetry on steroids. Now, Bilowal gives a little bit of background around the technology that this is descended from as well. He writes, What the heck is this photogrammetry thing? NVIDIA is supercharging with AI. TLDR, photogrammetry is the art and science of measuring stuff in the real world using images and other sensors, e.g. LiD.R. Invidia's new AI model is basically like photogrammetry on steroids.

Why? Traditional photogrammetry can't handle repetitive. structures, textureless surfaces, or strong color variations. But Neurolangelo blends the tech behind instant neural radiance fields to capture every detail imaginable. Thus far, Nerfs and photogrammetry have served different purposes. Nerfs are for stunning visualizations, think fly-throughs, but lacks surface detail when turned into 3D meshes.

Photogrammetry is great for surface reconstruction, think measuring stuff, but not always visually appealing. Neuroalangello is the game changer we've been waiting for. Its approach delivers the best of both worlds by bridging the gap between the gap between visuals and surface reconstruction. In what I think is a great analogy, he compares it to sculpting in 3D. Basically, first, this rough 3D scene emerges, and just like a sculptor chisels a block, refining it bit by bit, the details slowly start to come out. Now, that famous Michelangelo invocation that he doesn't actually sculpt things per se, but just discovers what's already in

the marble, might be part of the inspiration to call this Neuroalangelo. Bellowal concludes, In summary, Neurolandgelo represents a significant advance, achieving both realistic vision and finely detailed 3D models that stay true to surfaces. So let's talk applications. Obviously, there is a huge potential for gaming and metaverse and 3D world applications, which I think is going to do nothing but increase in importance over the next couple years. But then there's also the ability to create digital twins of real world objects that has a number of different applications as well.

All in all, this is definitely today's example of Arthur C. Clark's famous quote that's efficiently advanced technology is indistinguishable from magic. Next up, let's check in on AI in China, a couple announcements on that front. First of all, Chinese tech giant Baidu is rolling out a $1 billion yuan or around $145 million AI fund. In addition to that, the company is launching a competition for developers to build applications on its custom LLM, which is called Ernie. Speaking of custom Chinese LLMs, Chinese giant Alibaba is also putting chat GPT-like AI experiences built on its custom LLM into its meeting and messaging apps.

But of course, it's not just Chinese companies who are racing to integrate AI into basically every enterprise experience. Asana, the team software, has now introduced Asana intelligence. Asana's goal is to use AI to help teams make decisions faster, such as getting automatic recommendations that can remove bottlenecks or how to reallocate resources, as well as think about resource management in a different and more automated kind of way. Now, more interesting to me than just the fact that Asana had launched these tools was that alongside it, they also published their five, guiding principles for human-centered AI. These are the principles they say will guide everything that they do with AI, and I wonder to what extent this becomes a trend for other companies who introduce AI features as well. Asana's five principles include one, AI should help people achieve their goals.

Two, we design for human and AI teams. Three, people are accountable for their decisions. Four, we're committed to safety in the short and long run. And five, we promote transparency in practice and in product. Now, to me, the thing that's interesting is less the specifics, although they seem like fine principles, and more the idea that as the policy and regulatory discussion around AI grows, will we see companies articulate these sort of voluntary ethical principles when it comes to their approaches to AI? Microsoft made yet another AI announcement. They're now offering their designer service as part of their team suite, and in so doing, they are going directly after tools like Canva and Figma by putting AI-generated design directly into their core feature set.

Now, all that said, while most companies are racing to figure out how, they could integrate AI features, some companies are availing themselves of the courts to stop other people from using AI features. Getty has filed another lawsuit against Stability AI. The first one came in the U.S. and basically accused Stability AI's stable diffusion of training its model on Getty's 12 million proprietary images without any sort of permission or payment. Stability AI asked a court to throw that suit out last month, and now Getty has filed a second suit this time in the UK. They're asking a UK judge to prohibit stability AI. from offering products in the country because of their use of Getty's images for training.

You'll remember yesterday we talked about how Japan has said that they will not enforce copyright around this type of AI training, but that's not necessarily the policy in places like the U.S. and the UK. Finally, Times' forthcoming issue's cover story is all about AI risk. The big blaring red cover reads the end of humanity, how real is the risk? If you listen on Sunday, I'll be reading a couple of the pieces from that issue, but for now it's an interesting marker of how the public discourse has evolved. That's it for today's AI breakdown brief. If this was helpful, please like, subscribe, and share,

as well as turn on those notifications so you don't miss an episode. And I'll be back with the main AI breakdown in just a little bit. Hey guys, before the main AI breakdown, I wanted to tell you about another AI podcast that I've been absolutely loving recently, and that is the Cognitive Revolution. The Cognitive Revolution features interviews with the people who are actually pushing the bleeding edge of AI forward. It's entrepreneurs, researchers, thinkers, basically the people who are building critical tools and infrastructure that are going to shape technology, the economy, and the broader collective human experience.

Now, as you know, the AI breakdown is a daily news show, and for me, I love having TCR to go to to get my big think interview fix. The show is hosted by my friend Eric Torrenberg, who I've known for years and years, and his co-host, Nathan LeBenz, who spends a ton of time actually exploring and testing and just generally finding out about the product so that he's, you know, he can go to a much more significant level of conversations with their guests. Just for example, Nathan last year was invited to be a red teamer for OpenAI for GPT4 for two months. The AI space moves incredibly fast. If you're looking for great interviews with the most interesting people at the frontiers of AI, I can't recommend the cognitive revolution highly enough. A report that an AI-powered drone killed its operator in a military simulation set off an incredible firestorm yesterday,

Although now the story has shown a very different type of problem than it first appeared. Yesterday I got a number of text messages like the one on your screen right now. Yo, yo, did you do an episode on this today with a link to this AI drone article? Because WTF, like, it is literally the exact thing that every single person has been saying is going to happen. So what is this story and why was it such a concern? Throughout the day yesterday, a story started to get traction of a military simulation in which the Air Force had trained an AI drone to destroy Sam sites. The problem was that when human operators told the drone to stop, the AI started attacking the human operators, seeing them as obstacles to completing its goal.

And when humans said that that wasn't the right approach, it started attacking the communications towers, so humans couldn't tell it to stop. Now, this came from a blog summary called highlights from the RAES Future Combat Air and Space Capabilities Summit. On the 23rd and 24th of May, the Royal Aeronautical Society hosted a landmark defense conference, the Future Combat Air and space capabilities summit at its headquarters in London, bringing together just under 70 speakers and 200 plus delegates from the armed services industry, academia, and the media from around the world to discuss and debate the future size and shape of tomorrow's combat air and space capabilities. Now, the section that got all the attention was this one, AI. Is SkyNet already here?

Could an AI enabled UCAV turn on its creators to accomplish its mission? It's not long, and the details very much matter in this case, so I'm going to read the section in question. As might be expected, artificial intelligence and its exponential growth was a major theme at the conference, from secure data clouds to quantum computing to chat GPT. However, perhaps one of the most fascinating presentations came from Colonel Tucker Sinko Hamilton, the chief of AI tests and operations, U.S. Air Force, who provided an insight into the benefits and hazards in more autonomous weapons systems. Having been involved in the development of the life-saving auto-G-CAS system for F-16s, which he noted was resisted by pilots as it took over control of the aircraft,

Hamilton is now involved in cutting-edge flight test of autonomous systems, including robot F-16s that are able to dogfight. However, he cautioned against relying too much on AI, noting how easy it is to trick and deceive. It also creates highly unexpected strategies to achieve its goal. He notes that one simulated tests saw an AI-enabled drone, tasked with an SEAD mission to identify and destroy SAM sites, with the final go-no-go given by the human.

However, having been reinforced in training that destruction of the SAM was the preferred option, the AI then decided that no-go decisions from the human were interfering with its higher mission, killing Sam's, and then attacked the operator in the simulation. Said Hamilton, quote, We were training it in simulation to identify and target a Sam threat, and then the operator would say, yes, kill that threat. The system started realizing that while they did identify the threat, at times the human operator would tell it not to kill the threat,

but it got points by killing that threat. So what did it do? It killed the operator because that person was keeping it from accomplishing its objective. He went on. We train the system, hey, don't kill the operator, that's bad. You're going to lose points if you do that. So what does it start doing? It starts destroying the communications tower that the operator used to communicate with the drone to stop it from killing the target. Now, this basically couldn't be a more perfect articulation of exactly what AI safety and AI risk people have been discussing. When people ask, how would rogue AI start to cause problems for humanity? This is the type of scenario they envision.

The concern is not that AI somehow becomes evil or develops malice. The concern is that the AI, trained on a specific objective, figures out that humans are, in fact, in the way in some way, of achieving that objective. Musician Grimes, who has been deep in the AI space, responded to the initial viral thread sharing this blog post and wrote, Very scary, but good we're testing in a simulation now. Eliezer Yudkowski writes, good they tested in SIM, bad they didn't see it coming given how much the entire alignment field, plus the previous 50 years of SF were warning in advance about this exact case. Now, pretty soon, major news outlets around the world were picking up the story. Newsweek writes, military drone attacks human operator during simulation.

Sky News, AI drone kills human operator during simulation. South China Morning Post, AI-powered drone tried to kill a human operator in U.S. military simulation. Hindustan Times, AI-controlled drone turns on operator in shocking simulated test. Now, much to their credit, a number of the loudest voices when it comes to AI alignment issues had a bit more hesitation before blasting it around as complete evidence of everything they've been saying for years. Eliezer Yudkowski again says, dot, dot, dot, can we get confirmation on this being real?

Talking about a recent post, V. Mousiewicz says, I didn't include this because I want to take the time to confirm the details first. I mean, come on, that's too perfect an illustration of the problem, right? Despite the fact that I am in no way a reporter, I even took the time to try to ask the original source of this whether it happened exactly as the blog post represented it. By this morning, there were some holes being punched in the story. The U.S. Air Force denied that a simulation like this had ever happened. And then finally, a little later in the morning, we got confirmation from Colonel Hamilton

that it wasn't exactly as he had put it at the conference. Aerospace added to their blog post saying, In communication with aerospace, Colonel Hamilton admits he misspoke in his presentation at the Royal Aeronautical Society, FCIS Summit, and the rogue AI drone simulation was a hypothetical thought experiment from outside the military based on plausible scenarios and likely outcomes, rather than an actual U.S. Air Force real-world simulation. Hamilton said, We've never run that experiment, nor would we need to, in order to realize that this is a plausible outcome.

He clarifies that the U.S. Air Force has not tested any weaponized AI in this way, real or simulated, and says, despite this being a hypothetical example, this illustrates the real-world challenges posed by AI-powered capability, and this is why the Air Force is committed to the ethical development of AI. The reaction to this was really fast. Armand Domaluski, who had had the original viral tweet, said, I deleted this tweet because the AI-powered drone turns on its operator story was total nonsense. The colonel who described it as a simulation now says it was just a thought experiment.

Benedict Evans took the media to press saying, This story is bullshit and never happened, but we're running it anyway. Maybe we need to stop all AI reporting for six months to make sure it's aligned. V.C. Nick Carter writes, A drone literally went rogue and killed its operator to, in a simulated AI environment, too, there was no computation and it was a tabletop simulation to, okay, it was a hypothetical thought experiment by one guy. He follows up, the demand for anecdotes supporting AI Dumerism far exceeds the supply. Now, I have pretty harsh feelings about this.

There is just about nothing worse for real discussions about these issues than this sort of made-up BS. This has an incredibly anesthetizing effect to anyone who might otherwise actually care about what is a really important discussion. People are hypersensitive right now, and rightly so to being manipulated. This Air Force colonel, who is not just some random guy, but actually has the credentials and the experience to back up what he's saying, passing this hypothetical off as a real-world example, does amount to manipulation, even if he didn't mean it to be. The fact that it was amplified by media is incredibly predictable, because it's a big juicy story, and were it true, it would be a big deal. Now, there is also very much a media culpability part here, as Nirajagrawal points out,

how many people think an actual real human drone operator was killed in that simulated test. That's what the text messages that I got thought, because the headlines didn't necessarily go out of their way to make it seem like this was just a war game. I think it's a pretty telling example on the back end of a week that saw so many CEOs and academics sign a letter saying that this AI risk and safety conversation was one we needed to have. If we burn people out with AI Dumer's Cried Wolf scenarios, we are never going to be able to have that conversation in the way that it needs to happen. I try not to get too righteous on this show, but I think this colonel should be absolutely ashamed of himself. And to the extent he was trying to fudge a hypothetical into something

that would get people's attention, he has very much done more to hurt the cause that he was trying to support than to help it. Anyways, guys, that is it for today's AI breakdown. A bit of a fiery one, perhaps, but this is too important a discussion to have based around fake examples. Anywho, that is it for today's AI breakdown. If you're enjoying this, this show. If you're finding it useful, please like, subscribe, and share it. You can get it as a newsletter that comes out every morning or listen to it as a podcast every day. Click the notification button so you don't miss any episodes. And until next time, peace.