The AI Daily Brief: Artificial Intelligence News and Analysis - Game of Thrones, but for AI Chips

Episode Date: April 12, 2024

Apple's unveiling of the M4 chip aims to rejuvenate its Mac lineup with enhanced AI capabilities, following a 27% sales drop. Intel introduces the Gaudi 3 chip, challenging NVIDIA's H100 with claims of 50% faster performance. Google and Meta intensified the race with new AI chips, Axion and MTIA, for improved data center operations. Amazon is developing AI-specific chips, while the Biden administration boosts U.S. chip manufacturing with significant Chips Act grants to TSMC, emphasizing the strategic importance of AI hardware. ** CHECK OUT THE JUST-LAUNCHED SUPERINTELLIGENT PLATFORM - 300+ AI video tutorials https://besuper.ai/ ** ABOUT THE AI BREAKDOWN The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI breakdown, we're looking at a set of news in the AI chip wars. Before that on the brief, OpenAI fires some researchers for alleged leaks. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. We're going to Breakdown Network for more information about our Discord or YouTube and our newsletter. Welcome back to the AI Breakdown Brief, all the AI headline news you need in around five minutes. We kick off today with a couple pieces of Open AI news, the first being quite the T. According to the information, OpenAI has fired two researchers for leaking sensitive information. The two researchers include Leopold Aschenbrenner and Pavel Ismelov.

Both of these researchers have been on the team dedicated to AI alignment and safety research. And according to the information, Ashenbrenner was also an ally, their word, of OpenAI chief scientist, Ilya Sutskever, who of course we have no idea where he actually is or what he's actually still doing with OpenAI. We also have no information about what the two allegedly leaked. Right now it's all a big bucket of speculation. The information talks about Ashenbrenner's ties to the effective altruism movement, the most notorious proponent of, of course, being Sam-Bankman-Fried.

And the intimation is that the leaks probably had something to do with the disagreement around how OpenAI was proceeding with regard to safety. But again, that's all speculation right now. Whether these two actually talk or if they just show up at a competing lab remains to be seen, but that is the story. Next up, earlier this week, OpenAI announced a more advanced GPT4 Turbo with vision model, but it was only available initially through the API. Well, now they've brought that new advanced model to chat GPT.

The company writes, Our new GPT4 Turbo is now available to paid chat GPT users. We've improved capabilities in writing, math, logical reasoning, and coding. For example, when writing with chat GPT, responses will be more direct, less verbose, and use more conversational language. We continue to invest in making our models better and look forward to seeing what you can do. So the example they give of it being less verbose, the prompt was SMS reminding friends to RSVP to my

birthday dinner invite. The previous response was, hey, friend's name, just checking in to see if you've had a chance to RSVP for my birthday dinner. I'm finalizing the table reservation and really hope you can make it. It wouldn't be the same without you. Please let me know by RSVP deadline. Looking forward to celebrating together. The new response is, hey friend's name, just a friendly reminder to RSVP for my birthday dinner. Hope you can make it. Let me know soon. Now, of course, the proof is in the pudding. And so far, I've seen a lot of people say that this does feel like GPT4 is now once again caught up with what they were seeing with Claude 3 opus. There's some disagreement around whether that's the case with coding, but I've seen some people talk about how the coding has gotten significantly better as well. Of course, there is a larger question, which is why we're only getting these very incremental improvements rather than some big model launch, and a lot of speculation that OpenAI is still very much holding things back. Next on the brief, Amazon has added well-known computer scientist Andrew Inge to its board. Andrew is an extremely well-known figure in the AI space, having led projects at Google and Baidu, and seemingly showing that Amazon is here to compete in this area. Finally, the Humane AI Pin is getting lots of reviews, and they're not all that great.

For example, the Verges Review is called Humane AI Pin Review, not even close. For $699 and $24 a month, this wearable computer promises to free you from your smartphone. There's only one problem. It just doesn't work. The reviewer writes, I came into this review with two big questions about the AI pin. The first is the big picture one. Is this thing anything? In just shy of two weeks of testing, I've come to realize that there are, in fact, a lot of things for which my phone actually sucks. Often all I want to do is check the time or write something down or text my wife, and I end up sucked in by TikTok or my email or whatever unwanted notification is sitting there on my screen.

Plus, have you ever thought about how often your hands are occupied with groceries, clothes, leashes, children, steering wheels, and how annoying slash unsafe it is to try to balance your phone at the same time? I've learned I do lots of things on my phone that I might like to do somewhere else. So, yeah, that is something. Maybe something big. AI models aren't good enough to handle everything yet, but I've seen glimmers of what's coming and I'm optimistic about the future. That raises the second question. Should you buy this thing? That one's easy. No, no, no way. The AI pin is an interesting idea that is so thoroughly unfinished and so totally broken in so many unacceptable ways I can't think of anyone to whom I'd recommend

spending the $699 for the device and the 24 monthly subscription. Now, there really was not just one nasty reviewer. There were a bunch of people who said something similar. However, for a more generous and let's call it historical point of view, I'll read this tweet from investor Kala Jalanbo, who writes, with AI pin, Humane will go down in history as the company that created the world's first ever AI computer. Yet, as history proves, version 1.0 is just the beginning of a founder's vision. Those who understand what it takes applaud entrepreneurs that have the guts and imagination to challenge giants, rather than scrutinize them like their already trillion-dollar mega-caps. The humane team are pioneers that catalyze the conversation around how we apply AI and

consumer hardware. And that, in itself, merits recognition and respect. I think that's a great way to put it. And very clearly, we are at the beginning of a flowering of new, interesting experiments when it comes to AI hardware. Who knows where that will all land? That's going to do it for today's AI breakdown brief. Next up, the main AI breakdown. Welcome back to the AI breakdown. Today we are catching up on the AI chip wars, which if they get any more intense

are really going to start being reminiscent of Game of Thrones pretty soon. So let's start with the most recent story, although just one of many from this week. Yesterday, Bloomberg's Apple whisperer, Mark German, reported that Apple is planning to overhaul their entire Mac line with a new custom silicon chip. They're calling the M4.

It's obviously the fourth in their line M-chips, and this one is specifically focused on AI. Now, what's not surprising is that Apple is thinking about AI and leveraging their custom silicon to try to win an advantage in that space. In fact, a lot of our conversations about Apple's strategy or likely strategy when it comes to AI have been about how it is trying to, on the one hand, increase the power of chips in its devices, and on the other hand, shrink powerful models such that they can be running on device without having to touch the cloud. What is surprising about the announcement is that the M3 was first released just five months ago. That's an extremely short turnaround, even for Apple, which has a

reputation for making its old technology obsolete very quickly. According to sources, the M4 chip will come in at least three varieties, and Apple is planning on updating every single Mac model with it. Now, obviously on this show, we are focused on the AI dimension of this, but just to understand the broader business dimension for Apple, their Mac business is tough right now. Mac sales fell 27% during the last fiscal year, and even during the holiday period, revenue was flat. Between that, and they're seeming sort of strategic flailing, given their recent ending of Project Titan, their electric car project, their unclear AI strategy. Overall, it feels like Apple needs to get things back on track. Investors seem to like this report. Apple had actually been down

13% on the year, very different than the other AI-focused tech stocks, but on Thursday saw their share price go up 4.3%, which was their biggest single-day gain in 11 months. The names of the M4 chips include Donin, which is the entry-level version, a middle-powerful version called Brava, and a top-end version called Hydra. These are their codenames, so they might not be called that when everything comes to light. In terms of getting more information, sources expect us to hear more about these chips and the devices that they will run in at the Worldwide Developer Conference, which is coming up in June.

Bloomberg also writes, as part of the upgrades, Apple is considering allowing its highest-end Mac desktops to support as much as a half-terabyte of memory. The current Mac Studio and Mac Pro top out at 192 gigabytes, far less capacity than on Apple's previous Mac Pro, which used an Intel processor. The earlier machine worked with off-the-shelf memory that could be added later and handle as much as 1.5 terabytes. With Apple's in-house chips, the memory is more deeply integrated into the main processor, making it harder to add more.

So, one of our competitors in the Game of Thrones for chips is Apple, but an old player seems to be coming with some new tricks as well. Ars Technica writes, Intel's Goddy 3 AI accelerator chip may give Nvidia's H-100 a run for its money. Intel claims 50% more speed when running AI language models versus the market leader. So basically this week, Intel held its... Vision 20204 event, and their big reveal was a new AI accelerator chip that they're calling the Gowdy 3. Now, Intel is pursuing a number of different strategies simultaneously. They're

increasingly competing with TSM as a chip fabricator, i.e. a builder of other people's chips, but the Gowdy 3 suggests that they're not giving up on their own chip business either. Now, it is important to note that although Gowdy 3 is projected to have faster performance in terms of training time and inference than NVIDIA's H-100, the H-100 is no longer for Nvidia's most powerful chip. The H-200 is not out yet, but is expected basically any day now. And then there is, of course, the Blackwell B-200. One of the big questions is what the price of the Gouti 3 will be. Analysts think that it could be an attractive alternative if it can come in meaningfully under the 30 to 40,000 and H-100 costs. I think the big takeaway from this Game of Thrones

perspective is that Intel seems willing to fight on multiple fronts at once and is putting out some pretty compelling products. Moving on in our battle, Google has announced its own ARM-based CPU to support, support AI work in data centers and is also introducing a more powerful version of its TPU or tensor processing unit chips. The new CPU is called Axion and will be initially used to support Google's internal AI workloads before rolling out to business customers of Google Cloud later this year. Google said that the Axion chips are already powering YouTube ads, Google Earth, and other Google services. Said Mark Lomeyer, Google Cloud's vice president, Axion is built on open foundations, but customers using Arm anywhere can easily adopt

Axion without re-architecting or rewriting their apps. Reuters reports that Axion arm-based CPU will offer 30% better performance than general-purpose arm chips and 50% more than Intel's existing processors. Google also discussed their overall cloud strategy in terms of what chips they offer, specifically announcing that they are not offering AMD chips. Again, Mark Lomeyer said, at this time, we're deploying Nvidia GPUs and our own TPUs, and he added that they are also supporting Intel's new central processing unit, which he claimed was, quote, good for inferencing certain workloads.

Lo Meyer said, we offer those three and we feel good about that combination for our our customer base. Meta also made a chip announcement, which was perhaps understandably a little bit overshadowed by the fact that they revealed that Lama 3 was coming soon, but the company says that their new meta training and inference accelerator, MTIA, is a, quote, big piece of its long-term plan to build infrastructure around how it uses AI in its services. Meta said in a post, meeting our ambitions for our custom silicon means investing not only in compute silicon, but also in memory bandwidth, networking, and capacity, as well as other next-generation hardware systems. Meta had first announced their V1 of the MTIA in May of 2023,

and said at the time that they were focused on providing those chips to data centers. That V1 was not expected to be released until 2025, but Meta now says that both that first version and a new MTIA chip are in production. Writes the Verge, right now, MTIA mainly trains ranking and recommendation algorithms, but meta said the goal is to eventually expand the chip's capabilities to begin training generative AI like its Lama language models. Over in the world of Amazon, CEO Andy Jassy,

discussed the chip wars in his annual letter. He wrote, To date, virtually all the leading FMs have been trained on Nvidia chips, and we continue to offer the broadest collection of Nvidia instances of any provider. That said, supply has been scarce and cost remains an issue as customers scale their models and applications. Customers have asked us to push the envelope on price performance for AI chips, just as we have with Graviton for generalized CPU chips. As a result, we built custom AI training chips named Traneum and inference chips named Infercia. In 2020, we announced second versions of our Traneum and Infercia chips, which are both meaningfully more precise.

performance than their first versions and other alternatives. This past fall, leading FM maker Anthropic announced it would use Traym and Infercia to build, train, and deploy its future FMs. We already have several customers using our AI chips, including Anthropic, Airbnb, HuggingFace, Qualtricks, RICO, and Snap. So the point here is not necessarily that Amazon announced anything new, but that the battle is significant enough that it warrants mention in this annual letter. But what about the geopolitics of chips? Obviously, this is a huge global issue, and we've gotten some news recently in that respect as well. The Biden administration is continuing to distribute money through the Chips Act, with the latest recipient being TSM, who are receiving a $6.6 billion

grant to increase their U.S. manufacturing. TSM is, of course, building out their first major U.S. plant in Phoenix. It's actually two plants with this money being used to build a third. And as part of the announcement, TSMC said that they're increasing their total investments in the U.S. from $40 billion up to $65 billion. This is so far the second biggest grant under this program, with the first being the $8.5 billion in grants plus $11 billion in loans that were announced for Intel a couple of weeks ago. As an aside, TSM is doing well right now in general. They saw a 16% rise in quarterly sales, which outstripped projections and also was the fastest growth in more than a year. Another chip exporting nation, South Korea, is investing around $7 billion to stay competitive

in chips as well, said South Korean President Yun. Current competition in semiconductors is an industrial war and an all-out war between nations. Writes Reuters, Yun has set a target for South Korea to become one of the top three countries in AI technology, including chips, and take a 10% or more share of the global system semiconductor market by 2030. Yun said, Just as we have dominated the world with memory chips for the past 30 years, we will write a new semiconductor myth with AI chips in the next 30 years. So lots and lots going on in the AI chip world.

Like I said, this is a battle that is not slowing down even a little bit. That is going to do it, however, for today's AI breakdown. Until next time, peace.