The Changelog: Software Development, Open Source - AI poisoned its own well, libraries to UnsuckJS, we need more Richard Stallman & ChatGPT package hallucination (News)

Episode Date: June 26, 2023

Tracy Durnell thinks AI has already poisoned its own well, Adam Hill's microsite catalogs everything you need to UnsuckJS, Lionel Dricot thinks we need more Richard Stallman, not less & the Vulcan team proves you can't trust ChatGPT's package recommendations.

What up nerds, I'm Jared, and this is Changelog News for the week of Monday, June 26th, 2023. Hey, that sounds familiar. Hello friends, I'm Jared, and this is Changelog News for the week of Monday, June 27th, 2022. What the what? That was me one year ago this week. That's right. ChangeLog News is a one-year-old. Pretty cool, huh?

Okay, let's get into the news. Here's a quick clip of yours truly and Simon Wilson talking stable diffusion back in September of 22. Like there's just probably too many of them to exclude those. So you're going to start to train on your own stuff. And then isn't that like a multiplicity, you know, every time you make a copy of a copy, it gets dumber or whatever. This is one of the big questions, the big open questions. I don't think anyone's got a really strong idea of what's going to happen. Like are all of our images going to average out to just one beautiful Greg Rutkowski dragon or yeah, I don't know. I don't know how that's, I have no idea.

I don't. Everything's going to be brown. You know, you combine all the colors, you get brown every time. Wouldn't that be fascinating? Yeah, no, I have no idea what's going to happen with that, but it's obviously a very real, real challenge. Same thing with these language models. GPT-3 is going to start consuming its own tail as well. And what does that look like? Who knows? That conversation is, is oh so relevant today because of a new study on AI model collapse that says, quote, we find that use of model-generated content in training

causes irreversible defects in the resulting models where tales of the original content distribution disappear. We refer to this effect as model collapse and show that it can occur in variational autoencoders, Gaussian mixture models, and LLMs. End quote. Tracy Dranell writes that she believes AI has already poisoned its own well. Quote, I suspect tech companies, particularly Microsoft slash OpenAI and Google, have miscalculated and in their fear of being left behind have released their generative AI models too early and too wide. By doing so, they've essentially established a

threshold for the maximum improvement of their products due to the threat of model collapse. I don't think that the quality that generative AI will be able to reach on a poisoned data supply will be good enough to get rid of all of us plebs, end quote. Since there's no consistent system for marking up generated content online as computer-generated, the toothpaste is already being squeezed from its proverbial bottle. Here's Tracy again. Because of this approach, 2022 and 2023 will be essentially lost years of internet source content, even if they can establish a tagging system going forward and get people who are hostile or ambivalent to them to use it. End quote.

If she's right, this is a big deal. Unsuckjs.com is a cool microsite from Adam Hill that catalogs the many 20 plus JavaScript libraries that progressively enhance HTML and cost 10 kilobytes or less to deliver to your clients. No build tools, no compilers, and no hassle. I'd call it perfect, but as we all know, there's nothing perfect. Perfect is boring.

I mean, except Shawshank. That's not boring. Yep. That's true. Yeah, I mean, there are perfect films. Back to the Future, Jaws. Jaws. I would say Trent's hair is perfect.

Oh, good. Thank you. Not at all boring. It's true. You know, attributes, Grace Kelly's eyes. The other side of the pillow. Oh, yeah. That is absolutely perfect.

There are perfect analogies. Jacket potato, cheese, and beans. Oh, yeah. That sounds perfectly yummy. Perfect foods. Spaghetti bolognese. Billy Jones' The Stranger album. Yeah, that was perfect.

Yes, they are perfect works of art. Their perfection is all around us, everywhere we look. The mighty Redwood. I was talking about perfection in people. I'd love to see this resource go beyond the basic information and table format it currently has. But still, I'm a big proponent of this less JS movement, and there are some high-quality libraries featured here, and some I've never even heard of. Having them all in one place is a win.

We need more Richard Stallman, not less. That's the title of a recent article by Plume, a.k.a. Lionel Dracau. After a big fat disclaimer at the top, differentiating the man's philosophy from the man himself, Plume does not vouch for the man himself. He writes, quote, RMS was right since the man himself. Plume does not vouch for the man himself. He writes, quote, RMS was right since the very beginning. Every warning, every prophecy realized. And worst of all, he had the solution since the start. The problem is not RMS or FSF. The problem is us. The problem is that we didn't listen. End quote. The core of Stallman's beliefs were the four freedoms of software.

The right to use the software at your discretion, the right to study the software, the right to modify the software, and the right to share the software, including the modified version. These four freedoms were formalized as copyleft, but according to Plume, RMS's theory had a weakness in that copyleft itself wasn't part of the four freedoms that it secured. This allowed other non-copyleft licenses to come along and secure all four as well. There's just too much here to quote it all on the show,

but read the piece, which includes Plume's suggested amendment to RMS's four freedoms of software. Then let me know in the comments what you think about this conclusion. Was RMS right? Did we just not listen? Would Plume's amendment fix things? I'd love to hear your thoughts on the matter. It's time for some sponsored news. Listen, just because you don't record a problem doesn't mean it didn't happen. Sentry is the only developer-first application monitoring platform that shows you what's slow down to the line of code. But don't take their word for it. Matthew Egan, engineering team lead at DiviPay, has this to say about the service. Quote, Unlike past tools we've used, Sentry provides the complete picture. No more combing through logs.

Sentry makes it incredibly easy to find issues in our code to deliver a much smoother payment experience and a better overall customer experience. End quote. Check the link in the show notes and get a demo of Sentry today. Why not, right? Thanks once again to Sentry for sponsoring ChangeLog News. Can you trust JATGPT's package recommendations? Mmm, not so

much. The team at Vulcan have published a new security threat vector they're calling AI Package Hallucination. Mmm, that sounds good. I'll have that. It relies on the fact that ChatGPT sometimes answers questions with hallucinated

sources, links, blogs, and statistics. It'll even generate questionable fixes to CVEs and offer links to libraries that don't actually exist. Quote, when the attacker finds a recommendation for an unpublished package, they can publish their own malicious package in its place. The next time a user asks a similar question, they may receive a recommendation from ChatGPT to use the now-existing malicious package. End quote. These AI tools like ChatGPT are a real boost to developer productivity,

but be careful out there. That is the news for now. On Wednesday, I'm talking yak shaves, system architecture, negative 10x devs, and more with Taylor Troesch. And on Friday, Kelsey Hightower joins Adam and I on Changelog and Friends. Have a great week, share Changelog with your peers who might dig it, and I'll talk to you again

real soon.