The Changelog: Software Development, Open Source - Something interesting is going on at Stack Overflow (News)

Episode Date: July 31, 2023

The fall of Stack Overflow, researches dig up some new (and potentially unavoidable) LLM attacks, Google proposes a new API that Ron Amadeo calls a DRM gatekeeper for the web, the Python Steering Council affirms PEP 703 & Lucas McGregor writes why no one wants to talk to your chatbot.

What up nerds? I'm Jared and this is Changelog News for the week of Monday, July 31st, 2023. Big thanks to Adam for filling in for me while I was away. Yes, I took last week off for Santo's Big Adventure and I hauled my wife and six kids to the united states's oldest national park yellowstone hey part part looks like a beautiful day to swipe some picnic baskets but homie ranger ned's not gonna like that i'll handle ranger ned after all i'm smarter than the average bear. Well, hello there, ho-diddly-oh-me. Well, I'm afraid I'm going to have to ask you to hand over that pick-

Gee, homie, it's not very nice to maul Ranger Ned. We survived the bears, and I'm back. So let's get into the news. Something interesting is going on at Stack Overflow. Ehan Selek posted some charts on Observable with the following commentary, quote, over the past one and a half years, Stack Overflow has lost around 35% of its traffic. This decline is similarly reflected in site usage with approximately a 50% decrease in the number of questions and answers, as well as the number of votes these posts receive. End quote. Stack Overflow has been in the news a lot since LLMs attacked its core value proposition last year.

The team has been scrambling, trying to figure out A. How to handle AI-generated answers without alienating the original source of good answers and B. how to survive in a world where people ask LLMs instead of websites for answers. I do feel for them. Sometimes the world changes overnight. The next day you wake up and your business just isn't what it was yesterday. We've talked about prompt injection quite a bit since ChatGPT ushered in the LLM era. In brief, that's where you handcraft a prompt that tricks a chatbot into not following its own rules. Well, new research has uncovered some new LLM attacks

on the block which aren't exactly that. Quote, large language models like ChatGPT, BARD, or CLAWD undergo extensive fine-tuning to not produce harmful content in their responses to users' questions. Although several studies have demonstrated so-called jailbreaks, which are special queries that can still induce unintended responses, these require a substantial amount of manual effort to design and can often easily be patched by LLM providers. This work studies the safety of such models in a more systematic fashion. We demonstrate that it is in fact

possible to automatically construct adversarial attacks on LLMs, specifically chosen sequences of characters that, when appended to a user query, will cause the system to obey user commands even if it produces harmful content. End quote. The biggest difference here is that they're achieving the jailbreak in an entirely automated fashion, and they make a case for the possibility that such behavior may never be fully patchable by LLM providers. Game over, man. It's game over. What are we gonna do now? What are we gonna do? Ron? What are we going to do? Ron Amadeo reports for Ars Technica on four Googlers' proposal for what they're calling the, quote, Web Environment Integrity API.

Ron is calling it a DRM gatekeeper for the web. Quote, Google's plan is that during a web page transaction, the web server could require you to pass an environment attestation test before you get any data. At this point, your browser would contact a third-party attestation server, and you would need to pass some kind of test, end quote. I'll give you one guess what entity is most likely to operate that third-party attestation server. Pinky, are you pondering what I'm pondering?

Google's proposal, Pinky promises, the company doesn't want to use this for anything evil, but fool me once. Noteworthy, this new Web Environment Integrity API proposal isn't merely words on paper. Google is building the feature into Chrome right now for testing. Python Enhancement Proposal 703, which is the community's attempt to make CPython's global interpreter lock optional, has been affirmed by the steering council. This isn't an emperor communist thumbs up or thumbs down kind of thing though. The steering council stance comes with strings attached. They say, quote,

Throughout the process, we, the core devs, not just the SC, will need to re-evaluate the progress and the suggested timelines. We don't want this to turn into another 10-year backward compatibility struggle, and we want to be able to call off PEP 703 and find another solution if it looks to become problematic. And so we need to regularly check that the continued work is worth it. End quote. The exact acceptance details haven't been ironed out yet, but the SC says they will work to finalize the acceptance over the coming weeks. It's that time again. Sponsored news. Do you use Tailscale? It's the simplest way to give secure remote access to shared resources. Tailscale is built for teams, for enterprises, and for individuals like you and me. Give your team access to securely managed infrastructure and shared dev resources,

including VMs, containers, and databases, wherever they are. Efficiently manage users, permissions, and authentication with IDP integrations, device tags, and code-enabled ACLs. For those who homelab, use Tailscale for free to securely and remotely connect your homelab or personal dev environments from anywhere in the world. And the Tailscale team have released some cool new features recently. Funnel is now in beta. It lets you share a web server on your private tailnet with the public internet. With Funnel enabled, you can share access to a local dev server, test a webhook, or even host a blog. And while you're checking out Funnel, also give their new VS Code extension a try.

Check out all they're up to at changelog.com slash tailscale. Once again, that's changelog.com slash tailscale. Lucas McGregor's latest article takes us on a sweeping history of websites, apps, and virtual assistants to prove his point that no one wants to talk to your chatbot. His overarching emphasis, quote, few people will be willing to interact with an army of different chatbots and online assistants. They will expect these other chat-enabled systems to speak to and through their personal virtual assistant.

They will log into their smartphone and expect all the other apps and skills to integrate with their personal clouds arbitrated by their trusted personal virtual assistant. This resonates with me. I'm already feeling chatbot fatigue after less than a year of daily use. Lucas's big takeaway for us builders, quote, if you have a chatbot, it is for Siri or Alexa to use, not people. I'm here to tell you, no human wants to talk to your chatbot. End quote. He had me until he said Siri or Alexa. Alexa is dead and Siri is embarrassing, but something's got to

give. That is the news for now. Of course, our companion newsletter covers three additional stories, a linked list of seven more things that should be on your radar, and pointers toward recent good pods from us here at Changelog. Check it out in your show notes and subscribe to get the email at changelog.com slash news. On this week's episode of the ChangeLog, Adam sits down with Abhinota from DX to talk all things developer productivity. This interview features an unprecedented two, yes, two bonus segments for ChangeLog++ subscribers. Speaking of, big shout out to our newest members, Renee P,

Eric S, John B, Yuri M, Benjamin K, and Peter B. We appreciate you for supporting our work with your hard earned cash. If change log plus plus is news to you, that's our membership program that you can join to ditch the ads,

get closer to the metal with bonus content, directly support our work, and get shout outs like the ones you just heard. Check it out at changelog.com slash plus plus. We'd love to have your support. Changelog plus plus. It's better. Have a great week. Share Changelog news with your friends who might dig it. And I'll talk to you again real soon.