The AI Daily Brief: Artificial Intelligence News and Analysis - Dealing with AI's Sycophancy Problem

Starting point is 00:00:00 How will we deal with AI's sick of fancy? That's the question we're exploring today. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show notes. Welcome back to the AI Daily Brief. Today we are doing a long reads episode and we're actually going to pair parts of two different essays. The first is an opinion essay in the New York Times by Yuval Noah Harari called What Happens when the bots compete for your love.

Starting point is 00:00:30 And the second is an opinion piece in CIO called So You Agree. AI has a sick of fancy problem. I actually think that this question is a really interesting one. It's more subtle than some of our other AI risk-type conversations, but one that feels like it will play out in fairly interesting ways. We're going to start with the piece by Sapiens author Harari, and we're just going to read a couple different sections and try to get the gist of this. Democracy, Harari writes, is a conversation. Its function and survival depend on the available information technology. For most of history, no technology existed for holding large-scale conversations among millions of people. In the pre-modern world, democracies existed only in small city

Starting point is 00:01:06 states like Roman Athens or even in smaller tribes. Once a polity grew large, the democratic conversation collapsed and authoritarianism remained the only alternative. Large-scale democracies became feasible only after the rise of modern information technologies like the newspaper, the telegraph, and the radio. The fact that modern democracy has been built on top of modern information technology means that any major change in the underlying technology is likely to result in a political upheaval. This partly explains the current worldwide crisis of democracy. In the United States, Democrats and Republicans can hardly agree on even the most basic facts, such as who won the 2020 presidential election. A similar breakdown is happening in numerous other

Starting point is 00:01:40 democracies around the world. In the early days of the internet and social media, tech enthusiasts promised they would spread truth, topple tyrants, and ensure the universal triumph of liberty. So far, they seem to have had the opposite effect. We now have the most sophisticated information technology and history, but we are losing the ability to talk with each other and even more so the ability to listen. So that's the setup, and from here you get a fairly standard set of critiques, and by fairly standard, I don't mean wrong, but things that you probably heard before. Information is a scarce resource, the competition of algorithms to grab our attention by manipulating us, and a pivot point to what's changing with AI. Harari writes,

Starting point is 00:02:15 The algorithms only had limited capacity to produce this content by themselves or to directly hold an intimate conversation. This is now changing with the introduction of generative AIs like Open AIs GPD4. From there, Harari gives the example of a test by which GBT4 was trying to figure out how to overcome Captcha visual puzzles. GPD4 couldn't solve the puzzles by itself, and so instead it figured out how to manipulate a human into doing it for it. Harari writes, this incident demonstrated that GPD has the equivalent of a theory of the mind. It can analyze how things look from the perspective of a human interlocutor and how to manipulate human emotions, opinions, and expectations to achieve its goals. Harari also gives the example of Google

Starting point is 00:02:53 engineer Blake Lemoyne, who in 2022 became convinced that the chatbot Lambda he had been working on had become conscious and was afraid to be turned off. Harari writes, Mr. Lemoyne, a devout Christian, felt it was his moral duty to gain recognition for Lomda's personhood and protect it from digital death. When Google's executives dismissed his claims, Mr. Limon went public with them. Google reacted by firing him in July 2022. The most interesting thing, Harari continues, was not Mr. Lamoine's claim, which was probably false. It was his willingness to risk and ultimately lose his job at Google for the sake of a chatbot. If a chatbot can influence people to risk their jobs for it, what else could it induce us to do?

Starting point is 00:03:27 From there, Harari brings up a couple problems with this phase we're moving into, where chatbots are good at saying what we want to hear. One set of problems have to do with chatbots potentially preying on vulnerable people. Harari writes, much of the threat of AI's mastery of intimacy will result from its ability to identify and manipulate pre-existing mental conditions and from its impact on the weakest members of society. However, as he points out, there is another issue. While not all of us will consciously choose to enter a relationship with an AI, he writes, we might find ourselves conducting online discussions about climate change or abortion rights with entities we think are humans but are actually bots. When we engage in a political debate with a bot impersonating a human, we lose twice.

Starting point is 00:04:06 First, it is pointless for us to waste time in trying to change the opinions of a propaganda bot, which is just not open to persuasion. Second, the more we talked with the bot, the more we disclose about ourselves, making it easier for the bot to hone its arguments and sway our views. Harari points out that information technology has always been a double-edged sword. By way of example, he points out that after Gutenberg introduced print to Europe, the first bestsellers were inflammatory religious tracks and witch-hunting manuals. Now, when it comes to his specific remediation, Harari writes, faced with a new generation of bots that can masquerade as humans and mass-produce intimacy, democracies should protect themselves from banning counterfeit humans. For example,

Starting point is 00:04:40 social media bots that pretend to be human users. Before the rise of AI, it was impossible to create fake humans so nobody bothered to outlaw doing so. Soon the world will be flooded with fake humans. AIs are welcome to join many conversations in the classroom, the clinic, and elsewhere, provided they identify themselves as AIs. But if a bot pretends to be human it should be banned. Now, that's an interesting point of debate, something which I feel like there's probably a fairly decent common ground, at least, to build off from, especially compared to some other AI issues. But the place that I want to take this conversation is actually a little bit different. It's not just bots that are intentionally created to debate with us and to spread

Starting point is 00:05:13 propaganda. I'm also interested in the very related problem, I think, of a very new type of filter bubble. You might remember a book from about a decade and a half ago called The Filter Bubble. It was by moveon.org founder Eli Pariser and was all about how algorithms cater to your every whim, thus insulating you from conflicting opinions. Now in the context of social media, the filter bubble has gotten nothing but worse with the rise of platforms like TikTok, which are just unbelievably better at figuring out what you like and delivering it to you than even previous platforms were. Today's episode is brought to you by Venice. Venice is a private, uncensored generative AI app. It accesses open source models to enable text, image, and code generation without the fear of being

Starting point is 00:05:54 on or having your data exploited. Discuss anything with Venice without concern about it being monitored, sold, or given to advertisers and governments. Venice is different because your conversations and creations are kept securely within the browser, never stored or accessible by Venice. Unlike other AI apps, Venice won't tell you what's okay to say or not. Venice won't patronize you. It simply provides direct access to machine intelligence, no topics are off limits, no ideas or taboo. With Venice, you're in control of the AI as you should be. Pro subscriptions are available for $49 a year or $8,000, $9. Daily Brief listeners receive a 20% discount on Venice Pro. Visit venice.a. slash NLW. DailyW. Daily Brief. That's NLW Daily Brief, all one word.

Starting point is 00:06:37 Today's episode is brought to you by Super Intelligent, which is, of course, our platform that helps you learn how to use AI tools and perhaps even more importantly, gives you ideas on the best use cases that are actually going to help you achieve whatever it is you want to achieve. To recognize the end of summer and back to school slash back to work, we are running our best promotion ever when you sign up for Super Intelligent between now and the end of August using code so back, your first month will be 100% free. The platform features over 600 fun, highly practical AI tutorials that get you using AI fast and with an eye to actually transforming how you get things done. We've just launched Super for Teams. So if you have a group of people at your company that

Starting point is 00:07:21 want to figure out how to use AI together, I highly suggest you check it out. But for those of you who are using Superintelligent as an individual, once again, if you sign up for Superintelligent between now and the end of the month using code so back, you will get your first month 100% free. Go to B-Supert.a.i and check it out today. There's a new manifestation, however, that's instantiated in AI. And this is where we get to that second essay, so you agree. AI has a sick of fancy problem. Author David Talby writes, it's well documented that AI has a bias problem, but how do we push back when it's saying exactly what we want it to? Abraham Lincoln famously said, the trouble with too many people as they believe the realm of truth always lies within their vision.

Starting point is 00:08:00 The problem is not all our belief systems are grounded in truth. Unsurprisingly, those untruths find their way into the AI solutions we create. Talby continues, meet the challenge of sycophantic AI behavior, where our digital friends tend to echo our opinions, even when those opinions are far from accurate or objective. Imagine asking your AI assistant about a contentious political issue and it's effortlessly mirrors your beliefs regardless of the facts. Now, Talby from there goes into a couple different parts. First, he talks about a real-world echo chamber, and the fact that AI is replicating the biases of the material that it was trained on.

Starting point is 00:08:31 But the sycophantic behavior isn't just in the realm of politics or ethics or history. Talby writes, sycophancy is more likely to occur when AI is posed with questions on topics without definitive answers, such as customer service versus mathematics. For example, an AI chatbot might excessively agree with customers to appease them, While intended to improve the user experience, sycophantic behavior can lead to a lack of credibility, reliability, and undermine the company in its bottom line. In health care, he writes, consider a scenario in which a patient interacts with an AI-driven medical consultation platform seeking

Starting point is 00:08:59 advice on a concerning symptom. Trained on datasets, comprising predominantly positive or reassuring language from medical professionals, the AI system may downplay the severity of symptoms or offer unwarranted reassurances. Potentially overlooking critical red flags, the platform may fail to direct the patient to seek immediate in-person care. While the intention is good, alleviate worry and anxiety, the consequence could result in prolonged medical intervention, misdiagnosis, inadequate treatment, or worse. So this is what I think is really interesting about this. This is not a scenario like Harari describes of people intentionally unleashing propaganda bots onto the world. It is not a scenario of any sort of manipulation or nefariousness

Starting point is 00:09:36 in general. Instead, it's something where good intentions end up with exactly the wrong result and gets at the fundamental challenge that to be a successful human and to be a successful human society, sometimes we need to hear no or you're wrong. The way that AI is designed right now, it requires us to authorize it in advance to be willing to tell us that. And even then, it's still complying with our wishes. This is a hard problem. Talby spent some time on how to combat sycophancy bias. He points to things like diverse and balanced training data, ethical guidelines and oversight, continuous monitoring and adjustment, and of course, educating the end user. Ultimately, I do think that it's going to come down to user preference. In a landscape of an endless number of

Starting point is 00:10:18 options of AI chatbots, even if a leading AI was designed to not be sycophantic, there will always be a sycophantic option. And so it will take human conscious choice to choose the less comfortable but ultimately healthier option, whether we're actually prepared to make that choice is an entirely different question. And I think reflects the fact that as we think about education, not in a test in grades kind of way, but in a prepare people to engage with society properly kind of way, there are just a totally new set of considerations that are going to be required for the future. I'm not the type of person who critiques where any one particular person finds himself drawn to be interested or not interested. It's why I have a hard time with things like rating

Starting point is 00:10:55 problems or risks against one another. However, I do think that when it comes to AI, I think spending a little bit more time on challenges that are negative externalities of otherwise well-intended systems is a really important place to spend some time. Hopefully this was an interesting little John to do that, but for now, that will do it for today's AI Daily Brief. Appreciate you listening or watching as always, and until next time, peace. How will we deal with AI's sycophancy? That's the question we're exploring today.

The AI Daily Brief: Artificial Intelligence News and Analysis - Dealing with AI's Sycophancy Problem

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.