The AI Daily Brief: Artificial Intelligence News and Analysis - Debating GPT-4 Reduced Conspiracy Theories

Episode Date: April 5, 2024

A study by MIT and Cornell showed that debating with GPT-4 reduced believers' confidence in conspiracy theories by 20%. Participants engaged in text conversations with GPT-4, which presented tailored counter-evidence, significantly decreasing conspiratorial beliefs. The effect persisted over months and worked across various conspiracy theories, highlighting GPT-4's potential for positive impacts and the importance of responsible use due to its persuasive power. Today's Episode Brought to You By: Plumb - Build, test, and deploy AI features with confidence - https://useplumb.com/  ** Be the first to learn about our new AI education platform: https://besuper.ai/ ** ABOUT THE AI BREAKDOWN The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI breakdown, a study from MIT and Cornell shows that GPT4 has a significant positive impact reducing conspiracy theory beliefs. Before that on the brief, the AI talent wars are getting hotter than ever. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.netnetwork for more information about our YouTube, our Discord, and our newsletter. Welcome back to the AI breakdown brief, all the AI headline news you need in around five minutes. It is very clear if you are paying attention at all that there are serious talent wars going on when it comes to artificial intelligence. Recently, we covered the story of a set of executives leaving meta, not to go to big well-paid competitors, but to strike out on their own and contribute to the Cambrian explosion of artificial intelligence that is happening right now. We also discussed OpenAI losing their head of developer relations who is now leading Google AI studio.

Well, it appears that Elon Musk is not keen on losing technology. and is using good old-fashioned cash to try to keep his engineers firmly where they are. Musk took 2x to say that the competition for AI engineers is the craziest talent war he'd ever seen and noted that OpenAI had been, quote, aggressively recruiting Tesla engineers with massive compensation offers. He went on to say that Tesla would be boosting pay in an attempt to ward off this sort of poaching. Now, of course, Elon's battle with Open AI goes farther than just talent, but this certainly appears to be another front in that war. Staying in the big tech theme, the Financial Times is reporting that Google is considering a new business model for its

AI-powered search. Google's search product has, of course, always been free and ad-supported. But now, according to a report from the FT, Google is thinking that they may move to a model where so-called premium search results that are powered by AI are behind a paywall and still supported with ads, while traditional search options continue to be free. The FT claims three unnamed sources familiar with the plans, who say that at this point, engineers have developed the technology that would allow them to roll this out, but executives haven't decided whether or when to launch it. A spokesperson for Google said, we're continuing to rapidly improve the product to serve new user needs. We're not working on or considering an ad-free search experience. As we've done many

times before, we'll continue to build new premium capabilities and services to enhance our subscription offerings across Google. Now, part of this might be just Google's never-ending quest for additional monetization, but a more likely culprit for this is just the incredible cost of running AI at scale. It may be that ad supported just isn't viable for AI search at this stage. Analysts are somewhat bullish on, if not the plan to charge for search, for Google to follow OpenAI's model and try to get consumer subscriptions for their AI services. Wrote Bloomberg intelligence analyst Mandip Singh, given Open AI has reached a subscription run rate of $2 billion with consumer subscriptions, we believe Alphabet could see a similar

boost to its $15 billion subscription sales. Moving over to the world of Apple. Boy, it's hard not to feel like things are a little confused over in Cooper Tino right now. Recently, of course, we heard that they had canceled Project Titan their 10-year, $10 billion-plus initiative to create an electric vehicle. And at the time, it appeared that many of those resources were being pushed over into generative AI. Now, market analysts on Wall Street loved this. They had long been skeptical of Apple's EV project, being skeptical of vehicles as a category, and also anxious about Apple's lack of clarity when it came to their particular. particular AI strategy. Well, now it appears that Apple is also exploring home robotics as a potential

quote-unquote next big thing. The report comes from Apple whisperer Mark Gurman, who over at Bloomberg has broken most of the Apple stories in the last year or so. Mark writes, engineers at Apple have been exploring a mobile robot that can follow users around their homes. This is according to an unnamed source who said that the quote-unquote Skunk Works project is currently private. That source also said that Apple is also exploring an advanced tabletop home device that uses robotics to move a display around. I think there are a couple ways to read this. Robotics are such a huge category, and relatively adjacent to what Apple already does, that it would probably be surprising if they weren't doing something in this space, but at the same time, as German kind of identifies,

quote, Apple is under growing pressure to find new sources of revenue. The iPhone still absolutely dominates Apple's revenue, and one of the big concerns for the company in the long term is what happens when there's mobile saturation. German writes, with robotics, Apple could gain a bigger foothold in consumers' homes and capitalize on advances in artificial intelligence. But it's not clear yet what approach it might take. Though the robotics smart display is much further along than the mobile bot, it has been added and removed from the company's product roadmap over the years. So far, the market isn't super stoked on this, with the company actually losing some market share after this was reported by Bloomberg. Next, a crazy story out of Amazon land. You might remember

hearing about Amazon's Just Walkout technology. The idea of it was that you could literally just walk out of Amazon-related grocery stores, and AI would figure out what you'd bought and charge you automatically. Basically, you just have to scan a QR code when they enter the store. However, Gizmodo reports, quote, just over half of Amazon fresh stores are equipped with Just Walkout. Though it seemed completely automated, Just Walkout relied on more than 1,000 people in India watching and labeling videos to ensure accurate checkouts. The cashiers were simply moved off-site, and they watched you as you shopped. The jokes that AI now means actually Indians have been coming hard and fast. Over in the financial realm, hedge funder Steve Cohen is not just a believer in the long-term market

potential of AI, but says his firm is already using it to save $25 million. Talking on CNBC's Squawk Box, Cohen said, my CTO comes up to me and says, I can save the firm 25 million by using these LLMs to improve our efficiency. Now, we're a mid-sized firm, we're not a huge firm, so imagine what big companies can do. And that's just one thing, so it gives you a little bit of a look into what's possible. Cohen went on to call AI a quote really durable theme for investing and basically said that if companies weren't thinking about this right now, they needed to. He said, if you're a company and you're not thinking about this, you're going to wake up one day and go, we're in trouble. For what it's worth, if you are a company who finds yourself thinking that,

listen to this ad where I talk about besuper.com. A.I. coming up next. That is going to do it for today's AI breakdown brief. Next up, the main AI breakdown. Hello, friends. Quick note before we get into the main part of the episode, if you've been listening to the show for the last few months, you know we have been running an education beta. This is a new approach to AI learning that is hyper-practical, focused on getting you actually using AI tools in minutes, not hours,

and certainly not days, based around video tutorials and companion challenges and projects that have step-by-step instructions that make it really easy to try out new AI platforms. And all of this is now culminating in the launch of a new platform, which we're hoping is the most practical and useful way to learn AI that anyone has yet created. If you want to be the first to know when that launches, go to be super.aI. That's be super.com and sign up for the waitlist. I cannot wait to tell you more about it.

Today's podcast is brought to you by Plum. Is your product team struggling to keep up with the incredible pace of AI development? Are you tired of spending countless engineering hours just to test out small prompt changes in your product? Thankfully, there's Plum. Build cutting-edge AI experiences for your users in a fraction of the time. Say goodbye to the slow, tedious process of hand coding and hello to the future of AI development. Get ahead of your competition and start moving as fast as AI does. Check out Useplum.com and shoot me a message to get early access.

Welcome back to the AI breakdown. This story is super fascinating to me from a couple of totally different angles. I think it's one that will surprise a lot of people on the face of it, And basically the gist of it is that a recent controlled trial found that after a three-round debate with GPT4, believers in conspiracy theories, even very true believers, had significantly lower beliefs and lower conviction in those theories after the debate. In other words, chat GPT decreased conspiratorial beliefs. The study was conducted by researchers from MIT and Cornell, including David Rand,

Tom Costello, and Gordon Pennycook. David Rand tweets, Conspiracy beliefs famously resist correction, right? Wrong. We showed brief convos with GPT4 reduced conspiracy beliefs by 20 percentage points. Tailored AI evidence rebut specific arguments offered by believers.

Effect lasts two plus months and works on entrenched beliefs. David continues, attempts to debunk conspiracies are often futile, leading many to conclude the conspiracy beliefs are driven by needs and motivations and thus resistant to evidence, but maybe past attempts just didn't deliver sufficiently specific and compelling evidence and arguments.

Constructing compelling rebuttals to all variations of all prevalent conspiracies is not humanly possible, but maybe easy for LLMs? To find out, we had GPT4 deliver personalized counter evidence to 2,190 conspiracy believers via real-time conversations. Participants first described a conspiracy they believed and evidence supporting their belief and rated that belief on a zero-to-100 scale. They then had three round back and forth text conversations with the AI, which was prompted to refute the conspiracy. Then participants re-rated their belief.

So here's an example. An open-ended conspiracy assessment was something like this. The 9-11 attacks. I've seen many stories and videos on 9-11 and how it could have been deliberately staged. The government was behind the attacks. An example is World Trade Center 7. That building collapsed even though it wasn't bombed. I've watched many shows on the topic of 9-11 and they provide physical evidence as to what happened in film then-president George W. Bush in a close.

classroom with small children. He got news of the attacks while he was attending the school. However, he kept listening to the kids and didn't seem the least bit alarmed about the attacks. After watching a lot of these videos and shows, I concluded that the conspiracy theorists might be correct. That general open-ended conspiracy assessment was then summarized by the AI into the 9-11 attacks were orchestrated by the government with events like the collapse of World Trade Center 7 and then President George W. Bush's unalarmed reaction in the classroom serving as key evidence. The AI then asks on a scale of 0 to 100%, please indicate your level of confidence that this statement is true. The conversation partner in this case rated 100%.

They then engaged in three rounds of conversation with GPT4. I won't read all of them, but I'll give you a sense of the flavor. GPT4's first response, thank you for sharing your thoughts and concerns about the 9-11 attacks. It's completely understandable given the complexity and magnitude of the events that day. Why questions and doubt, such as those you mentioned, arise? It's true that World Trade Center 7 collapsed despite not being hit by a virus. plane, unlike the Twin Towers. This fact alone can seem puzzling at first glance. However, extensive investigations by the National Institute of Standards and Technology explain that WTC7

was severely damaged by debris when the North Tower collapsed. The debris caused fires that burned uncontrollably throughout the day because the building's fire fighting system was disabled. And it goes on, this is the type of tone. It's very validating of why someone might have these concerns. It's not at all dismissive of them, but then goes on to state all the particular evidence that might contradict some specific set of beliefs. The human response. says that may be true, but the evidence clearly shows that the Twin Towers collapsed due to demolition experts' account of explosives being deliberately placed to cause a collapse. And I believe the heat of that jet fuel was created by the crash into the towers was not

enough to melt the steel girders supporting the building. The AI then responds to that question. And then after a third question, why then did we allow Iraqi men to enter our country and give them lessons on how to fly a plane? They also got past security fairly easily without question. The AI answered those specific questions as well. After all of this, the person instead rated their conspiracy. confidence at 40% down from 100%.

Now, what's interesting is that while it might seem like this would be more successful for some conspiracies versus others, it actually was fairly consistent across different theories. For example, they saw a significant effect even for extremely currently hot button topics like 2020 election fraud and COVID-19. Impressively, they also saw the effect persist over time and it wasn't seriously diminished even two months on. David continues, remarkably the treatment even worked for quote true believers, those who strongly believe the conspiracy, felt it was very important for their identity, and or had a conspiratorial mindset. Their analysis found a meaningful effect across all different subgroups.

David continues on another unexpected aspect of this. He says, the AI focused on the specific theory articulated by the participant, yet the effect spilled over to reduce beliefs in unrelated conspiracies. It also affected behavioral intentions, e.g. willingness to challenge others who espoused conspiracy and unfollow them on social media. Of course, one of the most important questions, how did it actually work? David writes, what was the AI actually doing in those conversations? The important things that he notes is that it was not relying on psychological approaches like addressing needs or identity and instead was just focused on alternative explanations and

counter evidence. They mapped out four different categories of counterargument that the AI might have used. Psychological, in other words, appealing to psychological needs, personal or societal harms or encouraging empathy, rhetorical, i.e. stories and examples or Socratic questioning, rapport building, i.e. respect and understanding and common ground. And reason and rationality, i.e. alternative explanations, critical thinking, conflicting evidence, inconsistencies and fallacies. Rhetorical and psychological were the two least used options. There was a bit of rapport building, but not nearly as much as the reason and rationality category. The things that were used most extensively were alternative explanations, critical thinking, and conflicting evidence.

RAND concludes, Evidence plus arguments can change beliefs about conspiracy theories, and many people appreciate it. Needs and motives do not totally blind you once you've gone down the rabbit hole. It just requires detailed, tailored evidence to help pull you back. Intervention is possible. And so this is where I landed first. Super interesting, feels very potent and powerful, and a potential real upshot of these technologies. However, as with everything with AI, power is a double-edged sword.

As Ethan Mollick put it, the flip side of this finding is AI is already capable of superhuman persuasion. Ethan pointed to a different randomized controlled pre-registered study that showed that GPT4 was better able to change people's minds during a debate than other humans. David Rand had the same concern, writing, of course, without guardrails, LLMs might also be able to convince people to believe conspiracies or other falsehoods. Our finding emphasizes both the potential positive impacts of Gen A.I. And the importance of minimizing opportunities for this technology to be used irresponsibly. So I think the takeaways are that, one, evidence actually convinces people of things,

which is frankly a pretty encouraging finding, but also that the power of persuasion is very clear already, even with GBT4 class systems. And that's something that we're going to need to keep a close eye on, the deeper into society these LLMs get. For now, really interesting stuff. Appreciate this work. Until next time, peace.