The Changelog: Software Development, Open Source - Mojo might be huge, chatbots aren't it, big tech lacks an AI moat & monoliths are not dinosaurs (News)

Episode Date: May 8, 2023

Jeremy Howard thinks Mojo might be the biggest programming language advance in decades, Amelia Wattenberger is not impressed by AI chatbots, a leaked Google memo admits big tech has no AI moats & Werner Vogels reminds us that monoliths are not dinosaurs.

What up nerds, I'm Jared and this is changelog news for the week of Monday, May 8th, 2023. Let's get into it. Austin Powers always defeats me because he has mojo. Mojo? Mojo. The libido, the life force, the essence, the right stuff, what the French call a certain... I don't know what. The just announced Mojo is a Python superset aimed at fixing Python's performance and deployment problems. It has a great pedigree. Chris Lattner, whom you may know

from LLVM, Clang, and Swift is on the modular team. And FastAI's Jeremy Howard is very excited about it. Jeremy says, quote, I remember the first time I used the V1 of Visual Basic. Back then, it was a program for DOS. Before it, writing programs was extremely complex and I had never managed to make much progress beyond the most basic toy apps. But with

VB, I drew a button on the screen, typed in a single line of code that I wanted to run when that button was clicked, and I had a complete application I could now run. It was such an amazing experience that I'll never forget that feeling. It felt like coding would never be the same again. Writing code in Mojo is the second time in my life I've had that feeling. End quote. The Mojo team has lofty goals. Full compatibility with the Python ecosystem. Predictable low-level performance and low-level control. The ability to deploy subsets of code to accelerators all while not creating ecosystem fragmentation. They want to avoid a Python 2, Python 3-like situation. This is a brand new code base and a lot of work is left to be done, but people are excited. This could be huge.

Here is JS Party panelist Amelia Wattenberger's review of AI chatbots. Boo! Boo! Boo! Wait, no. Sorry, that was Princess Buttercup's nightmare in Princess Bride. Here's Jazz Party panelist Amelia Wattenberger on AI Chatbots. Quote, last night over wine and seafood, the inevitable happened. Someone mentioned chat GPT. I had no choice but to start into an unfiltered, no-holds-barred rant about chatbot interfaces. Unfortunately, for the countless hapless people I've talked to in the past few months, it was inexorable. Ever since ChatGPT

exploded in popularity, my inner designer has been bursting at the seams. To save future acquaintances, I come to you today. Because you've volunteered to be here with me, can we please discuss a few reasons chatbots are not the future of interfaces? End quote. Bullet points from Amelia's argument are 1. Text inputs have no affordances. 2. Prompts are just a pile of context, and three, responses are isolated. She has a lot to say about the topic. Follow the link in your chapter data or in the newsletter to read more. All right, let's do some sponsored news.

Have you tried Sentry's interactive sandbox? It's the coolest, easiest way to see if Sentry's app monitoring and error tracking services jive with the way you think. For the small price of your work email address, you have free reign to poke around at a real-world-esque Sentry dashboard and kick all the tires. Performance, profiling, replays,

crons, it's all there. Check the link in your chapter data and in the newsletter and try it out today. Thanks to Sentry for sponsoring this week's changelog news. I have been hopeful about open source large language models ever since our episode with Simon Willison last month. My hopes continue to wax strong after reading this admittedly vaguely sourced leaked memo from inside the goog. While our models still hold a slight edge in terms of quality, the gap is closing astonishingly quickly. Open source models are

faster, more customizable, more private, and pound for pound more capable. They are doing things with $100 and $13 billion per AMS that we struggle with at $10 million and $540 billion. And they're doing so in weeks, not months. The landscape of LLMs is moving crazy fast right now. On a recent episode of Practical AI, Daniel asked Hugging Face engineer Rajiv Shah to describe it to them, and his response was very good, but also necessarily lengthy. I will add that clip to the end of this episode for those interested. Amazon CTO Warner Vogels pens what looks like a defense of monolithic architectures, but is in actuality a defense of there being no silver bullet. Warner says, quote, there is no one size fits all. We always urge our engineers to find the best solution,

and no particular architectural style is mandated. If you hire the best engineers, End quote. He does speak to S3's microservice architecture and how well it has served that org, but reiterates that there isn't one architectural pattern to rule them all, and says monoliths are not dinosaurs. That is the news for now. Read the companion newsletter for additional stories about the Craigslist test, the rewrite everything in rust movement, the beginning of the end of the password and a bunch more.

I want to go to there. In honor of maintainer month, our changelog interview this week features three people who are dedicated to funding open source maintainers. Alyssa Wright, Chad Whitaker, and Dwayne O'Brien. Have a great week. Share changelog with your friends if you dig it. And we'll talk to you again real soon. At this point, there's tens of kind of large language models.

And yeah, there's a number of different ways we can kind of categorize your thinking about them. One of them is kind of the simplest, which ones are proprietary, which ones are open source. There's a spectrum when we talk about access to these. So there's some like, for example, OpenAI, where you don't have access to these. So there's some, like, for example, open AI, where you don't have access to the model, you don't know, you know, what data it was trained on, you don't know the model architecture, right, you just send your data to them, they send back the predictions. And so I think that's one model there. And then all the way at the other extreme, today, for example, Databricks released the latest version of its DALI model, which was an open source model that was then instruction tuned on a data set that Databricks created themselves that they're making available kind of open source for commercial use itself there. here too, because the models, for example, vary in size, where you have, for example,

something like Bloom that was developed by Hugging Face, which is one of the largest open source models at something like 170 billion parameters, to some of these much smaller models that are coming out that the Lama models and others that are maybe a billion parameters. And that size has implications in terms of how much reasoning ability, how much stuff is inside there. But inference, is this something that your teenager is going to run on their own GPU? Or is this something that's going to take a multi GPU cluster to be able to effectively use? There's other dimensions like what data the models were trained on. For example, with the open source models, we know what data they were trained on. One piece of this, for example, that's come up is knowing how much code a model was trained on. Because one of the things that's

often asked for is, hey, can we build a text to code type model where I want to do some type of autocomplete, some type of code generation type project? Well, if I start with a large language model that already understands code, it's a lot easier to fine tune it and make that capability. So like understanding the underlying characteristics of that data. Daniel, right. It's like an alphabet soup of different names. And like literally every week they're popping up and there's so many of these different characteristics because they also differ, for example, on the model itself and what the licensing is and the model weights, the data set that it was trained, the training code that it was done. We see this with kind of how Meta released the Lama model where they told everybody about it, but then they released the weights, but then they gated the weights. So only academic people were getting to them.

But then the weights were essentially leaked and now they're all over the Internet. So now everybody's using them. So it becomes very confusing kind of in this big thick mix of, you know, how to sort this out.