The AI Daily Brief: Artificial Intelligence News and Analysis - RUMOR: GPT-4.5 Coming Before EOY (Maybe Even This Week?)

Episode Date: December 11, 2023

NLW looks at rumors from some sources that have previously had OpenAI insider leaks that GPT-4.5 could be coming as soon as this week and likely before the end of the year. Also a look at fast moving open source developments with Mistral closing its fundraise and releasing its API. Today's Sponsors: Listen to the chart-topping podcast 'web3 with a16z crypto' wherever you get your podcasts or here: https://link.chtbl.com/xz5kFVEK?sid=AIBreakdown  ABOUT THE AI BREAKDOWN The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI breakdown, we're looking at Mistral's latest release. It appears that open source AI might have a new leader. Before that on the brief, rumors are flying that GPT 4.5 could be coming before the end of the year and maybe even this week. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.net network for more information about our YouTube, our newsletter, and our Discord. Welcome back to the AI Breakdown Brief. All the AI headline news you need in our digital. around five minutes. We start today's episode in a slightly different place than normal, which is in

the realm of speculation. However, one of the interesting quirks of the Twitter slash X space right now is that there are a couple accounts that seem to have insider information on what happens next at OpenAI. The two accounts are the Jimmy Apples account, which we've talked about on the show before, and then Flowers from the Future at Futurist Flower, which only started in October, but has already built a bit of a reputation. All the way back at the end of October, on October 26th, that account had tweeted OpenAI big announcement in December, which is a fairly generic and open-ended thing. And obviously, this was all before the Open AI drama, so who knows if that would still be true. But in terms of establishing their bona fides, let's look at another tweet from October 26th,

where that same account tweeted, just verified one of my sources, Gem and I will come in the second week of December. As Pedro responded on December 7th, I came from the future, second week of December. this is actually true. Now, of course, one correct prediction does not mean that we should believe everything that this account says, but suffice it to say that there are a lot of people taking it seriously. And so, last week, when on December 7th, that account said, some of you can look forward to a little something coming next week, a lot of people stood up and took notice. Now, they expanded on that about a day later when they wrote, okay, so there's one big thing and one small open-eye thing waiting for us. The small thing is currently delayed due to company dramas, and the big

thing seems to be progressing according to plan. December remains exciting, especially next week. That was tweeted on the 8th, so next week is, in fact, this week. That account also posted this cryptic image of a black screen with a blue chat GPT logo in the middle. The AI Safety Memes account writes, GPD 4.5 next week, Jimmy Apples and Futurist Flower, both of whom have made some accurate leaks are dropping gossip.

Futurist Flower writes, there's one big thing and one small open AI thingy waiting for us. This is the same quote that I just read you. Now, back to AI safety memes, they continue. The big thing could be GPT 4.5. The small thing? Maybe that was the GPT store that just got delayed to 2024. Now, in the same post, they also referenced a Jimmy Apples tweet. Keep an eye out on a potential end of December GPT4.5 drop and new multimodal from Anthropic.

How big of a deal with this be? I don't know. But remember, the jump from GPT3 to GPD3.5 was massive. Gemini might look quaint overnight. The AI safety memes account also shared a tweet that Jimmy Apples deleted that came out right after Gemini was launched that I had referenced on this show where he wrote, At Sam Altman, what are you waiting for? Going to take the mitts off yet and release what you are sitting on or keep drip feeding. Cat emoji. Now, AI entrepreneur Bindu Reddy also tweeted

about GPT4.5, writing, fascinating. Grock, in response to the prompt, what were the top AI tweets in the last two days? Grock has decided that OpenAI has released GPT4.5. It is hallucinating, but maybe it knows something we don't know. I'm hearing juicy rumors about 4.5 dropping before end of year. Now, what Kroc had come back with was a theoretical tweet from OpenAI that said introducing GPT 4.5 the next step in our journey towards more advanced and capable AI. Stay tuned for more updates and demos. There has been some reporting recently, although it's unverified as of yet, that GROC is actually accessing draft tweets, which obviously would be a big security problem if it's true. But anyways, all of this adds up to probably nothing, but certainly

some interesting speculation for a Monday morning. Is this important? Is this something you should plan around? Obviously not. Could be that we just wasted a few minutes on rampant speculation, but the reality is that right now, coming off of this Gemini announcement of an announcement, as well as the OpenAI power struggle a couple weeks ago, this is for better or worse, something that everyone is interested in. Meanwhile, in terms of real news, the EU has finally come to agreement over the EU AI Act. Now, if you've been paying attention to this, you're probably confused around exactly how the EU lawmaking process works, because we reported a few months ago that the EU AI Act had been passed, but then there were a few remaining steps.

But then it turned out those few remaining steps were very controversial or at least had room for some controversy. And now once again, an agreement had been reached. But as the New York Times writes, the law still needs to go through a few final steps for approval. So, TLDR, this podcast is much too short to try and actually explain the European Union political process. But suffice it to say that the AI Act has moved through yet another critical step and one which did see a fair bit of controversy and disagreement. Now, the biggest issue recently had been around what the rules would be for foundation model makers like OpenAI. A lot of the other rules, such as basically banning facial recognition AI, had been there from the beginning. Remember, the EUAI Act

has been in the works for many years, in fact, longer than ChatGPT has been around. Part of the reason that there was such controversy around things like ChatGPT is that it was kind of a new addition late in the process, and there were some who wanted to punt it off entirely and basically figure it out later. Now, when it comes to those rules for Foundation models, they did not end up exempting them entirely, but as Axios sums up, foundation model providers will need to submit detailed summaries of the training data they use to build their models. However, Axios also writes, quote, there's a big catch in the EU's landmark new AI law, which is the fact that it doesn't come into effect until 2025, which is, of course, an eternity in terms of the development of AI systems.

Again from Axios, the hiatus leaves plenty of room for the US or others to undercut the EU's plans before they go into effect by, for instance, implementing less restrictive rules before Europe's kick in. Senate Majority Leader Chuck Schumer has expressed concern that EU-style laws enacted by the U.S. would put American firms at a disadvantage competing with China. Now, EU lawmakers like Tieri Breton hail the agreement tweeting out an image that pointed out that the EU was now the only continent with comprehensive AI regulation. Many commentators, however, noted that the EU, screaming about how good at regulating it was, sort of just reinforced the perception of its place as a backwater for technology. Adam Singer tweets, you hate tech and economic growth. No one takes you

seriously, along with the graph, showing how few European tech companies there were relative to Asia and the U.S. Stephen Serges tweets, You need to understand that people want industry that works, and regulation can be required to make sure this exists, but nobody is clamoring for regulation in the absence of industry. Mike Salana writes, I am genuinely so embarrassed for you. And Roheet writes genuinely thought this was someone making fun of the EU. Now, all of that said, there are indications that regular citizens in a lot of countries, including America, are not as universally pro-AI as the people tweeting in response to the EU

might make it seem. The AI Safety Memes account again writes, a new poll dropped and wow are Americans anti-AGI-I, anti-opense source, and anti-effective accelerationism. Three to one want to prioritize AI safety over AI innovation. Five to one want to stop the AGI race by banning development of ASI, and five to one are opposed to EACC. Now, once again, these results do come from the AI Policy Institute, whose whole self-admitted role is to slow down. the development of AI, but they're still pretty interesting statistics. The question I think is to what extent these numbers reflect people's considered opinions or the incredible amount of fear-based media that has been published this year around the potential risks of AI. In either case, the numbers are

so stark that they're probably fairly hard to ignore, and so really the question becomes, I think, what U.S. policymakers do with all of that. One last note, for those of you who are sad that that Google demo video where it was identifying the duck and then the hand motions about what was going on, ended up being faked or fabricated or at least just not exactly what it appeared. Someone wired up GBT4 Vision to see if chat GPT4 could do it, and sure enough, it was able to perform a little bit better. Anyways, friends, that is going to do it for today's AI breakdown brief. Up next, the main AI breakdown.

Quickly a brief word from today's sponsor. As a listener of this show, I suspect you like to stay up to date on all things AI and tech, which is why you have to check out the chart-topping podcast Web3 with A16Z Crypto. Produced by venture firm Andresen Horowitz, Web3 with A16Z is the perfect companion podcast to the AI breakdown. Web3 with A16Z crypto is your definitive resource for the future of the internet. Whether you're interested in the convergence of AI and crypto or simply curious about what's next. If you need a place to start, they recently released an excellent episode with Stanford Cryptography Professor Dan Bonay and former Google X engineer Aliya in conversation with host Sonal Choxi about the intersection of AI and crypto.

From fighting deepfakes and proving humanity to large language models like chat GPT, they cover it all. I highly recommend checking it out, especially if you'd like to learn more about how AI and crypto will impact our everyday lives. Beyond crypto and AI, the show is for creators seeking more ways to truly own their work, for business leaders trying to prepare for the future today, and for innovators exploring trending tech topics. Don't miss out. Follow Web3 with A16Z crypto on Apple Podcasts, Spotify, or your favorite listening app.

Welcome back to the AI breakdown. Mistral first jumped into the headlines back in June when the company, which was at the time only four weeks old, announced that they had raised a $113 million seed round. Yes, a nine-figure seed round, which many at the time took as indication of just how overhyped and crazy the AI world was. However, subsequent to that, the company's first model that they released, their Mistral 7B model started to become a favorite among open source developers. Indeed, whereas Meta's Lama had commanded so much of the energy and attention throughout the year, Mistral really became another genuine

open source competitor of the big AI labs at companies like Gemini Anthropic OpenAI. Now, as we head into the end of the year, there has been a lot of speculation around what the open source side of the AI arms race will look like going into 2024. Arvin Shrinivas, the CEO of Perplexity writes, next year we will see at least two of meta, mistral, and anthropic catch up to Gemini and GPT4, going to be exciting times. Well, today we got a bunch of announcements from Mistral that suggest that Arvin may be even more right than he seemed. Now, first of all, was the confirmation of the funding round, which was first reported last week. The final tally was $385 million euros or $415 million, and the valuation, which again had been reported last week, was $2 billion. The round was led by

Andresen Horowitz, and other investors including Lightspeed, Salesforce, B&B Parabas, General Catalyst, and more also participated. Now, given that a big topic of our conversation during today's brief was the EU-AI Act, it's interesting to note the extent to which Mistral is using European pride as a selling point for this company, said co-founder and CEO Arthur Mench, since the creation of Mistral AI in May, we have been pursuing a clear trajectory, that of creating a European champion with a global vocation in generative artificial intelligence, based on an open, responsible, and decentralized approach to technology. Now, in terms of how Mistral thinks about European regulations, they are one of the companies that has been lobbying for a total exemption for foundation models, with their argument

being as TechCrunch sums up, quote, that regulation should apply to the use cases and companies working on products that are used by end users directly. Now, part of what makes Mistral interesting is that their 7B model was released under the Apache 2.0 license, which means no restrictions on user reproduction outside of attribution. Now, for those wondering how Mistral plans to make money, today also came with the announcement of a new API program. Through that, developers will be able to access their new Mixtral 8x7B model, which they call Mistral Small, and that will be for free again under the Apache 2.0 license. But a third model, Mistral Medium, is only available through the paid API platform. Devendra Trapp low from the company writes,

Excited to release Mistral Platform, three chat endpoints with competitive pricing. Mistrel Tiny, Mistral 7B instruct version 0.2, upgraded base model with higher context length 8K to 32K and better fine tuning. Mistral small. Mistral 8X7B instruct version 0.1 matches or exceeds GPT3.5 performance multilingual. Mistral Medium outperforms GPT3.5 on all metrics multilingual. All endpoints have a 32K context size.

Author Carlos E. Perez writes, Mistral Medium blows away GPD 3.5. What's Mistral Medium? Hyperite CEO Matt Schumer says, Looks like Mistral has a model that's even better than Mixtral 8x7B, and they're serving it to alpha users of their API, scoring 8.6 on MT Bench that's frighteningly close to GPT4 and beats all other models tested. This is their medium size. Large will likely beat GPT4. Carlos Z. Perez again writes, Mistral has released their report on mixtral. It's very impressive. It challenges Lama 70s. and GPT 3.5 on less hardware and at lower inference latency. Furthermore, it uses only 12 billion

active parameters to achieve this. So how does this all work? Well, if you guys are willing, let's wade into the realm of the technical for just a moment. And for this, I'm just sharing other explanations from folks who are attempting to provide a little bit of a layman's explanation of how mistral is approaching things, even if, as you'll see, it's still a little bit dense. Dmitro Zulgakov writes, Mistral model is hot with mixture of experts like GPT4. It promises faster speed and lower cost than model of the same size and quality. Fascinatingly, the speed up is unevenly distributed, running on a laptop or the biggest GPU server benefits the most. Here's why. The new Mixtral 8x7b model has around 47 billion parameters. Each token uses only a subset of those approximately 25%,

two experts out of 8. But when running multiple requests in the same batch, different requests will activate different experts. For a small batch, speed is determined by how quickly the GPU can read waits for memory. For a single request, reading two out of eight experts is four times faster. At a small concurrent load and most of the speed up evaporates, all experts have to be loaded. If one can fit a big batch with many requests in a GPU, the situation changes. Speed now mostly depends on how many multiplications, flops, are done. Mixtral MOE still does only 25% of those, and we can expect speed to approach 4x. But running a big batch comes with its own challenges. The model needs to be aggressively split into many GPUs. A mixture of experts brings more options here

like expert-level parallelism. GPT4 is rumored to use mixture of experts too. It serves a lot of traffic and thus benefits from this regime. Bindu Reddy also wrote an explainer writing, Transform a mixture of expert architectures, why they are more efficient. Mistral 8x7B M-OE model is a solid 70B GPT 3.5 class model. Instead of having every part of the model work on every task, a mixture of expert model splits up the work among many specialized submodels or experts. Each expert is good at handling specific types of tasks or data. When an MOE receives a request, it routes the request to the right expert. The good news is it is more efficient when you have a diverse number of requests as you don't

hit on all the experts at the same time. In the case of Mistral for each token, only two experts are used, and this could potentially mean a 4x speedup compared to something like Lama 2. GPT4 is rumored to be an MOWE model as well. It's likely to have a greater number of larger size experts. When it comes to training MOEs, there are complexities around dealing with training the gating network or router and load balancing, avoiding over-reliance on a few experts. Since MOEs are more parameter-efficient, you don't require quite so many computational resources. It will be really interesting

to see how much the open source community will improve Mistral MOE in the coming weeks. Already, I am seeing several fine tunes, but nothing very compelling in terms of metrics yet. Anyways, there is probably a much deeper show that could get into all of this if that's something that interests you. But for now, the other piece of the story that I want to talk about is how they launch this. Again, from Bindu, the right way to drop a model, she writes. Mistral AI just dropped a mixtral 8x7b model as a torrent link. Christmas came early for open source AI. Eric Jang writes,

Mistrel's brand is already becoming one of my favorites in the AI space, releases 87-gibite torrent containing an 8X7B M-OE model via tweet and refuses to elaborate. That was a couple days ago before this whole announcement came out. Nick Dobos pointed out the contrast to what we got from Google around Gemini last week. Mistral launching a torrent link versus Google Gemini Ultra blog post waitlist. Mistral actually understands the market it's building for, power users and hacker. Google keeps trying these disappointing consumer demos to convince the uninterested.

Don't understand, Doom, to fail. Now, the other thing that some people are discussing is how much mistral is sucking the oxygen out of the room for other open source competitors like META in particular. Farr L writes, META is losing Mineshare. Hopefully they release Lama 3 via Apache 2. Accelerate Harder, writes, If META really wants to make an impact on the open source world,

they would choose a fully open source license and they'd do it soon, before mistral steals their thunder completely. Now, bringing this conversation back to the very beginning of today's brief, the way that the flowers from the future account kicked off was actually with rumors about Lama 3. On October 23rd, that account wrote, Lama 3 will come in different sizes, but the most capable one will be around 120B.

Oh yes, and it will come February 24. We'll be multimodal on par with GPT4. Still others don't want to wait for February. Boyan Tungu says, all I want for Christmas is Lama 3 175B. Now, lastly, on this Mistral announcement, a number of people including One Little Coder, noted that in their announcement post, they had a space for a link to a safety document that they ended up not actually linking. One Little Coder writes, is Mistral actually trolling us, or maybe they forgot the link?

Professor Ethan Mollock writes the significance. For those who don't follow AI closely, one, an open source model, free anyone can download modified beats GPT 3.5. Two, it has no safety guardrails. There are good things about this release, but also regulators, IT security experts, etc., should note the genie is out of the bottle. However, to Mistral's credit, AI safety memes writes, quote, it has no safety guardrails. No open source models do, since you can just strip them off in two hours.

Mistral is just honest about it. Anyways, different people might have different takes on this, but from a technology perspective, what Mistral is putting out is pretty impressive and certainly makes these big investments and big valuations make a little bit more sense. That, however, is going to do it for today's AI breakdown. I appreciate you listening or watching wherever you are. Until next time, peace.