The AI Daily Brief: Artificial Intelligence News and Analysis - OpenAI Quietly Abandoned Arrakis Model Earlier This Year

Episode Date: October 20, 2023

In a rare miss, OpenAI had to shutter their Arrakis model earlier this year. They had hoped that the model would bring GPT-4 level performance with lower cost and more efficiency, but it simply didn't. NLW also looks at rumors of Apple AI coming to iOS 18. Today's Sponsors: Listen to the chart-topping podcast 'web3 with a16z crypto' wherever you get your podcasts or here: https://link.chtbl.com/xz5kFVEK?sid=AIBreakdown  ABOUT THE AI BREAKDOWN The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI Breakdown, we're looking at all of the latest news from OpenAI, including the fact that they had to shut down a promising model earlier this year. Before that on the brief, rumors that Apple is going to release generative AI features in iOS 18. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.network for more information about our Discord, our YouTube channel, and our newsletter. Welcome back to the AI Breakdown Brief. all the AI headline news you need in around five minutes. Today we start with some juicy rumors in one of the areas of discussion that has really been confounding for people this year, which is, of course,

what the heck Apple is going to do in the generative AI space. Now, what we know for sure is that Apple has been pretty unconclusive, even amongst itself, around what it wants its approach to be. Throughout the year, we've had reports that Apple had been involved in training their own models and even experimenting with them internally, but that they just weren't sure exactly what they wanted their approach to be. Now, part of this is probably due to the fact that Apple has very different feelings about privacy, as relates to other companies. They also like to run software directly on their devices, rather than having to rely on the cloud. Of course, the state of the art in LLMs right now doesn't really allow for that type of approach, at least not with some very different

thinking. And yet, even with all of this, it has seemed like there has been more energy recently going into figuring out what Apple's approach is going to be. Specifically, in September, we got a report from the information that Apple was now spending millions of dollars a day on training AI models. That sort of financial outlay suggests that they are moving closer towards doing something in the space. And yesterday, Mac Rumors shared an analysis from Jeff Poo, who's an analyst who covers Apple supply chain for a Hong Kong investment firm,

that generative AI on the iPhone and iPad is coming potentially in late 2024. Specifically, it sounds like there could be generative AI features as soon as iOS 18, which is, of course, the next operating system for Apple devices. From the piece, in a research note this week, Poo said his supply chain checks suggest that Apple is likely to build a few hundred AI servers in 2023 and significantly more next year. He believes Apple will offer a combination of cloud-based AI and so-called edge AI, which involves more on-device data processing. Now, back in August, another supply chain analyst Ming Chi Kuo said that given how behind Apple's generative AI efforts were, It seemed like that late 2022 time frame might not be realistic, and that Apple might not really be in the market with something until 2025 or beyond. Now, of course, whenever Apple does do something in the generative AI space, it's going to be a

huge deal, and especially if it comes natively integrated into the iPhone, even with hundreds of millions of new users between now and then, it's likely to still represent a significant mainstreaming moment. Now, while that Apple News is all still firmly in the realm of rumor or, if not rumor analysis based on things like supply chains, some other news related to Apple is a little bit more clear. Apple scored a coup when they brought John Stewart out of Daily Show retirement to create a new news show The Problem with John Stewart. Apparently the show was about to start production of its third season, and that is where they ran into challenging quote-unquote creative differences.

Writes The Verge, Stewart's intended discussions of artificial intelligence and China were a major concern for Apple. Though new episodes of the show were scheduled to begin shooting in just a few weeks, staffers learned today that production had been halted. Apparently what had happened is that Apple came to John Stewart directly and said that he and his team needed to be quote unquote aligned with the company's views on those thorny topics of AI and China. Stuart, unfortunately for Apple, said kindly no thank you and decided to leave the show. Now, it is important to note that we don't actually know the specifics of their differences in opinion. In other words, we don't know what the planned coverage of AI or China was that Apple had a problem with. So take with a grade of salt,

anyone contending to have good information around that. But still, it feels likely and is the assumption of most that whereas John Stewart would perhaps be particularly critical of China, Apple is trying to walk a tightrope of not upsetting that country who represents a huge market for them. Hopefully we get more information about what opinions were of disagreement. But for now, discussion of AI and China with John Stewart are out, at least when it comes to Apple TV. Next up, let's move over into the world of hardware.

IBM has just released research about a new chip that is specifically focused on AI, and which suggests that there are some pretty meaningful advances here. The processor is called North Pole, and the specific innovation is to take an approach that doesn't require the chip to as frequently access external memory, aka RAM, which means that it cannot only perform tasks faster, such as image recognition, but it can do so while also consuming less power. Said nanoelectrics researcher Damien Quirleaz, its energy efficiency is just mind-blowing. I feel the paper will shake the common thinking in computer architecture. Basically, this chip puts memory in each of its 256 computing cores,

which means less of having to shuttle data between chips. Wright's nature, the cores are wired together in a network inspired by the white matter connections between parts of the human cerebral cortex. This and other design principles, most of which existed but had never been combined in one chip, enabled North Pole to beat existing AI machines by a substantial margin in standard benchmark tests of image recognition. It also uses one-fifth of the energy of state-of-the-art AI chips,

despite not using the most recent and most miniaturized manufacturing processes. Now, importantly, this is just a research stage. And North Pole doesn't currently have the ability to work for large language models. As Nature writes, the chip can only run pre-program neural networks that need to be trained in advance on a separate machine. So what's the utility here? Well, one piece of it is that its architecture could be used in speed critical applications such as self-driving cars,

where the ability to pre-program those functions is there, and of course there's the possibility of continuing to evolve these new approaches to chip design. I think overall it's a reminder that alongside the rise and demand for AI software applications, it's highly likely that we're going to see a significant amount of hardware innovation as well. Last up today, another fun one from the world of science. AI has for the first time ever detected a supernova all on its own. So discovering supernovas is a really difficult and labor-intensive process. Astronomers and scientists basically have to hand go through huge, huge amounts of information

and manually identify potential candidates that could be supernovas. Well, now a team from Northwestern, who are, in fact, my alma mater, have created something called the bright transient survey bot, which does all the painful intermediary work all on its own. Gizmoto writes, researchers fed the BTS bot machine learning algorithm 1.4 million images from 16,000 astronomical sources. Those images included past evidence of supernovae, glaring galaxies, and temporarily flaring stars. Equipped with that training set, the AI model was able to identify a new supernova candidate and automatically requested spectrum reading from a robotic telescope at the Palomar Observatory in California.

The system eventually identified the supernova candidate as a stellar explosion in which a white dwarf star fully exploded, and it automatically shared its finding with the astronomical community. In other words, the AI system identified and shared the new discovery all on its own. Great news to the humans involved. Now, this team at Northwestern says that in the last few years, researchers have spent 2,200 hours doing the work that now might be able to be done by BTS bot and other technology like it, which obviously frees them up for much more advanced research and digging deeper into other astronomical mysteries. Many folks, including notably Sam Altman, think that one of the most profound impacts of artificial intelligence is likely to be the way that it increases the rate and speed of scientific discovery,

and this is certainly further evidence of how that might work. However, that is going to do it for today's AI breakdown brief. Thanks as always for listening or watching. Up next, the main AI breakdown. Are you interested in how two top-of-mind trends AI and crypto can work together? If so, I have the perfect podcast recommendation for you. Web 3 with A16Z Crypto, the chart-topping show brought to you by venture firm Andresen Horowitz. Web 3 with A16Z Crypto is your definitive resource for the future of the internet. Whether you're already building in these spaces or simply curious about what's next. If you need a place to start, they recently released an excellent episode with Stanford Cryptography

Professor Dan Bonay and former Google Xer Aliya in conversation with host Sonal Choxi about the intersection of AI and crypto. from fighting deepfakes and proving humanity to large language models like chat GPT, they cover it all. I highly recommend checking it out, especially if you'd like to learn more about how AI and crypto will impact our everyday lives. Beyond crypto and AI, this show is for creators seeking more ways to truly own their work, for business leaders trying to prepare for the future today, and for innovators exploring trending tech topics. So go ahead, listen to Web3 with A16Z Crypto, wherever you get your podcasts. Welcome back to the AI breakdown.

Today we are covering a variety of stories that have to do with OpenAI. There are some fairly significant pieces in here, but more than that, and this goes for any time basically that I cover Open AI. There are some times when a single company comes to define an entire larger industry or an entire larger movement. And right now, that is certainly the case with OpenAI and Generative AI more broadly. Now, I don't think that this will always be the case, but I do think that what is pretty unarguable is that this field leaped into general consumer consciousness with the launch of chat

GPT. Between that and the launch of GPD4, which has defined the outer barrier of models we have access to, for better or worse, open AI is just the dominant force in this space. It is a big question, an open question to many, around whether when Google releases Gemini, which potentially is even more advanced than GPT4, will that sense or sentiment change? And for that, that we'll have to wait and see. But for now, what's clear is that what happens to OpenAI and what Open AI does has a major impact and is covered as carefully as anything else by people who are trying to keep track of the AI space. Given that, it was particularly interesting to read a piece in the information earlier this week about a model that they had actually

dropped work on. Now, this is the Arrakis model which was named for Dune, and by the way, I'm not a Dune scholar, so forgive me if I'm getting the pronunciation wrong. But basically, the important thing to know here is that one, OpenAI has been developing a number of different models concurrently, and that two, Iraqis was one that was supposed to have some benefits that others did not. So basically what happened is that right around the same time that ChatchipT came out, and the whole world was racing to catch up and understand what was going on, engineers inside OpenAI began working on a new model that was codenamed internally Aracus. There were a few motivations for the project. One, it sounds like there was a business strategy motivation, as the information writes,

success with Iraqis would help OpenAI show Microsoft how fast it could create successive large language models, which would be valuable as the two firms finished negotiating a $10 billion investment in product deal. But two, and I think even more importantly, as we've seen how the LLM space has evolved, the goal of the Iraqis model was to allow them to run chat GPT and other tools with less cost. However, when push came to shove, the model underperformed what OpenAI had been hoping for, and ultimately by the middle of this year, they scrapped the project entirely. So let's talk about the technical aspect of this for just a moment before we get back into the implications. As we just discussed, the goal of the Iraqis model was to be as powerful as

GPT4, but to run much more efficiently and at lower cost. The goal was to leverage a machine learning concept known as sparsity. The idea of sparsity, very simply put, and probably reductively put, is to basically only use parts of the model that are relevant to generate responses, meaning that it's cheaper to run. Now, while the initial tests were promising, It wasn't too long before they realized that Iraqis was just not performing well enough to be considered for public release. Unfortunately, why it wasn't working as well as expected was not something the information could suss out. Apparently in late spring, after identifying that things weren't going well, OpenAI's team spent about a month trying to fix the issues before senior leadership decided to pull the plug entirely.

Now, there are, of course, a couple challenges with this. One is on the business and partnership front. The information writes the failure also disappointed some executives at Microsoft, which paid for the right to use the startup's technology in its products. according to a Microsoft employee with knowledge of the matter. More broadly, OpenAI just lost time, and time is the thing that no AI lab has. What's more? For the first time, OpenAI appeared vulnerable in some ways.

As the information again put it, the Iraqi setback could pierce OpenAI's aura of invincibility after it humbled AI pioneer Google and built one of the fastest growing software businesses in history. It shows how the frontier of AI is riddled with pitfalls that can be hard to predict. Now, of course, there was other model work going on at the same time. A multimodal model called Gobie has been in the works, And once Iraqis was shut down, the team involved pivoted to trying to work on a version of GPT4 that was specifically focused on generating responses more quickly. Now, even though Iraqis didn't work, the goal of bringing down costs and having models that run more efficiently

remains at the very top of priorities for not just OpenAI, but for all of these labs. Indeed, it appears from this article that some of the news that we've seen around Microsoft working with other LLMs might have been prompted by the Iraqis failure. What's more, even though OpenAI's first attempt at this didn't work, many still expect people to experiment with this sparse model approach, said Jeff Dean, Google's chief scientist, sparse computation is going to be an important trend in the future. Now, many commentators pointed out that there was a lot of information in this piece. In fact, a whole lot more than OpenAI was probably happy about. Former GitHub CEO and now investor Nat Friedman said, this is a whole lot of leakage from OpenAI. Now, speaking of leakage from OpenAI, another thing that everyone has been talking about for the past few weeks is the fact that there have been comprehensive.

conversations for additional fundraising. The one that has seemed the farthest along was a tender offer of employee shares at a valuation that was initially reported to be between 80 and 90 billion. Apparently, now that has been honed down to $86 billion and we're starting to get more information about who might be involved. Apparently, the lead on the round is Thrive Capital, which is of course led by Joshua Kushner, who is at this point perhaps best known as the brother of Jared Kushner, and apparently up to a billion worth of employee equity will be sold. Now, many are noting that this would be a 3x jump in the paper valuation of the company. Indeed, Thrive Capital actually bought employee shares back in April at evaluation of $27 billion. Now, apparently, reporting suggests that Sam Altman

has said privately that he expects before all is said and done that OpenAI will raise $100 billion along its path to building AGI. The company has jumped from $28 million in revenue last year to a revenue run rate of around $1.3 billion right now. A few more Open AI stories before we get out of here. One has to do with upcoming product releases. You'll remember that about a week ago, we cover news here of speculation that one of the things that OpenAI might introduce at their developer day at the beginning of November is a new set of AI agent tools. Well, according to TechCrunch, one tool that they are not sure when they will release or if they will release is a tool to detect AI generated images. Now, you may remember that OpenAI previously had an

AI detection tool that was out. It was designed to detect AI generated text not only from OpenAI's chat GPT, but also from other models. They pulled the detector because it had too low a rate of accuracy and was contributing to a sense that you actually could mechanically figure out with any level of accuracy what was written by machines as opposed to written by humans. This is actually a huge problem in the education field where there are companies out there selling schools and colleges on the idea that their detector can determine which of the students is using AI, creating a huge problem of false positives, where students have no recourse to defend that they actually were not using these tools. Now, a number of people inside OpenAI

have talked very positively about this AI image detection tool, with one researcher telling TechCrunch that the tool's accuracy is really good, and with OpenAI's CTO Mira Mirradi saying at a Wall Street Journal event this week that the classifier is, quote, 99% reliable at determining if an unmodified photo was generated using Dali 3. Now, of course, while we don't really have an answer to how we're going to deal with AI detection, there are a ton of companies that are trying to figure out the right approach. DeepMind has proposed a spec they call Synth ID, which would mark AI generated images in a way that humans can't see, but which can be identified by software. Adobe announced something similar

at their event a couple weeks ago. And ultimately, it feels to me like this is going to be a problem that is not just a technology solution, but also a social solution. In other words, I don't expect that we're going to see a tool that every single company and every single person adopts. I think it's far more likely that when it comes to important images and information, people simply will choose not to trust things of their own volition unless they have a recognized and respected validation, although who does that and how remains to be seen. Now, speaking of Dolly 3, Latent Space host Swix posted some new research this week and said, In a surprising for these times moves, OpenAI actually published a research paper detailing the improvements they made for Dolly 3,

primarily caption improvement via a fine-tuned image captioner and upsampling descriptions with GPT4. So there are two pieces that are interesting to this. One is the actual information that this paper has. Again, Swix sums it up this way. At the end, 95% of Dolly 3's dataset was synthetically generated by this captioner, illustrates the power of being a fully multimodal foundation model org. Working on text vision and image gen models can currently improve each other. There's also an answer as to why Dolly 3 can poorly generate text and images.

They simply made sure it was included in the captions. No extra work done on character level conditioning. So all of that is fascinating, especially if you are keeping track of how these image generation models are evolving. But I think just as interesting is the fact that they chose to release this research at all. This is not something that OpenAI has recently done. Ivy Zhang wrote, we are so back. Open AI is open again.

to which Swicks cautioned, eh, I didn't see that, we only see what they want us to see. Now, speaking of what they want us to see, obviously the next big event in OpenAI's world is the Dev Day at the beginning of November, so much so that Logan, who does developer relations over there, tweeted yesterday, going heads down until OpenAI Dev Day.

See y'all on the other side. I expect lots more interesting rumors and leakages before then, and of course some interesting announcements at the event itself. However, that is going to do it for today's show. Thanks for listening to the AI Breakdown. If you haven't yet, go sign up for the newsletter or come join us on the Discord. You can find all that information at breakdown.network.

And until next time, peace.