The AI Daily Brief: Artificial Intelligence News and Analysis - Nvidia’s Blackwell is Radically More Powerful AI Compute

Episode Date: March 19, 2024

Nvidia's new Blackwell GPU announced at the GTC conference revolutionizes AI computing with unmatched power and innovation. As the AI industry's backbone, Nvidia remains pivotal, unveiling Blackwell: an AI super chip boasting 208 billion transistors and unprecedented speed. Plus the Department of Homeland Security starts a set of AI pilots. ABOUT THE AI BREAKDOWN The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI Breakdown, Invidia announces their new Blackwell system. Before that on the brief, the Department of Homeland Security announces its own AI pilots. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.net network for more information about our Discord, our newsletter, on our YouTube channel. Welcome back to the AI breakdown brief, all the AI headline news you need in around five minutes. One of the things that I've often noted on this show is the sort of disparity between, on the one hand, the talky side of Washington, D.C., the politicians, Congress, the Senate, etc., running a seemingly never-ending set of meetings and discussions and hearings on AI legislation without ever really doing anything

about it, as compared to the U.S. military establishment, as well as private agencies, who are just moving ahead full steam. For example, we talked about how the State Department is hosting a military conference on AI this week, and yesterday the Department of Homeland Security also announced three pilot programs around generative AI. So what are those three efforts? The first is basically to use AI to better extract information from the huge volume of investigative reports and other materials that the department creates. As Axios sums up, investigators will be able to use LLMs to more quickly summarize investigative reports and to improve the process of searching through reports. The agency hopes the pilot program will improve detection of fentanyl-related networks

and help identify perpetrators and victims of child exploitation. The second initiative is a training focused initiative that is going to attempt to use generative AI to personalize materials to help train immigration officers more effectively, and in a way that is more up to date with changing legal issues and policies. Finally, they are trying to simplify and reduce the time cost around resilience and disaster planning with FEMA, making it easier, for example, for communities to submit grants and gain funding in these areas. The New York Times characterized Homeland Security as embracing AI and says, the agency will be the first in the federal government to roll out a comprehensive plan to integrate the technology into a variety of uses. Said Alejandro Mayorkas,

the Secretary of DHS, one cannot ignore it. And if one isn't forward leaning and recognizing and being prepared to address its potential for good and its potential for harm, it will be too late. And that's why we're moving quickly. So how big is the scope of these initiatives? Well, for some reference point, DHS employs 260,000 people for these new AI pilots, they're planning to hire 50 AI experts and spend $5 million overall. Meaning, in other words, that these are relatively small efforts. However, I think the fact that they are doing a full court press strategy around this indicates where it sits in terms of a priority. As part of the initiative, they'll be working with OpenAI, Anthropic, and Meta, and will also use Microsoft, Google, and Amazon Cloud in these pilots.

Basically, it sounds to me like they're testing everyone credible to see whose tools are the most helpful. Importantly, this is not something that's meant to be drawn out. The agency has to report on the results of the pilot programs by the end of this calendar year. I think it will probably not surprise you that my very strong assumption is that we're going to see a lot more efforts like this coming from basically every area of government. One additional area of overlap between the AI industry and the U.S. government may be in combating spying and cybersecurity issues. Another piece from Axios today is called insider threats are AI developers next hurdle.

They write, AI developers hiring quickly to keep pace with market demand are struggling with a new threat, spies and employees looking to steal company secrets. USAI companies are likely already prime targets for nation-state adversaries espionage campaigns. Experts predict that AI developers could become even bigger threats than shipping. manufacturers and biotechnology companies. Of course, we recently covered the Justice Department's indictment of a former Google Software Engineer for stealing AI secrets and sharing them with two Chinese companies. Seems to me like as the U.S. government gets more interested in this area, this could be a point of overlap for these companies. One more vaguely U.S. government-related story. The United States

Securities and Exchange Commission has recently at least partially moved on from its incredulity around the cryptocurrency industry to focus on warning of AI-related scams. Reuters reports that the SEC has now find two investment advisors over their AI claims. The SEC said that Toronto-based Delphalia Inc. and San Francisco-based Global Predictions Inc. agreed to pay a combined $400,000 in fines to settle civil charges related to AI washing. Wrights Reuters, the SEC found that from 2019 to 23, Delphia made false and misleading statements in SEC filings, a press release and on its website, over its purported use of AI and machine learning. Global predictions did the same in 2023 on its website and in social media. As part of the settlement, neither company was forced to

admit or deny the SEC's charges. Given the small amount of the fines and the nature of the settlement, to me it just looks like a ratcheting up of the warning shots from the SEC around other investment advisors throwing around AI language too loosely. Over in Big Techland, YouTube has announced a new tool that will allow creators to self-label when their videos contain AI-generated material. Basically, this is a new optional checkbox that comes up as creators are uploading and posting their content, which asks them to disclose whether there is altered or synthetic content that seems realistic. Examples given include making a real person say or do something they didn't, altering footage of real events and places, or showing a, quote, realistic looking scene that didn't

actually happen. The flip side is that they are distinguishing for things like beauty filters, special effects, and clearly unrealistic content like animation. Right now, these disclosures are voluntary. Finally, today, an interesting chart on which jobs are most likely to be impacted by AI, with the source being a World Economic Forum report called Jobs of Tomorrow. At the top of the heap, that report found that 73% of IT tasks could be automated or significantly altered, 70% of finance tasks, 67% of customer sales, 65% of operations, 57% of HR, 56% of marketing, 46% of legal, and 43% of supply chain. Of course, one of the big questions, when it comes to how AI does ultimately impact jobs, is whether the disruption will be on the task level or on the actual job level.

In other words, will this change what people do, or will it straight up for place people or some combination thereof. This will obviously continue to be one of the most important questions in terms of society's relationship with AI, but for now, that is going to do it for the AI Breakdown. Up next, the main AI breakdown. Welcome back to the AI breakdown. It's hard to overstate how unique Nvidia's position in the AI industry is. It's rare, first of all, to have a company so singularly associated with a new technological revolution, particularly an infrastructure company like Nvidia. But there's no denying that when people think of AI right now, the only company that might have some more mindshare mental association is OpenAI because they were

the ones who introduced ChatGBT, BT, which got this whole party started. Still, when it comes to the world of mainstream finance and just the mainstream in general, Nvidia is probably the company most associated with the generative AI transformation. It has over the course of the last year become the third most valuable public company, and frankly it shows no signs of slowing down. This week, Nvidia is holding its GTC conference. This is an event that used to just be for hardcore computer engineers and for Nvidia customers, but now has taken on the trappings of a festival. Extending that analogy, the headline act yesterday was, of course, CEO Jensen Huang,

whose announcement of Nvidia's new GPU platform Blackwell did not disappoint. Zai Rahul sums up. Invidia just announced Blackwell, the most powerful GPU in the market. Main features? AI Superchip, 208 billion transistors. second-generation transformer engine, fifth-generation NVLink, RAS engine, 100% in-system self-test, secure AI full-performance encryption, decompression engine 800 gigabytes a second. That all sounds like Greek to you. Don't worry because the upshot comes to the next line when Si writes, analysts estimate this could

potentially be 10 to 100x faster than Nvidia's current hopper slash A100 GPUs for very large transformer model workloads requiring multi-GPU scaling. This represents a monumental leap in scale for accelerating the trillion parameter AI future Nvidia is targeting. In Jensen's presentation, he showed one chart in particular that just did a great job of showing the exponential growth of AI compute. The headline reads 1000x AI compute in eight years and goes back to 2016 when Nvidia introduced Pascal at 19 terraflops. The next year, 2017, saw Volta with 130 taraflops. 2020 ampere comes with 620. Hopper, the next great update, what we've been working on, 22, with 4,000 tarflops, and now Blackwell with 20,000 tarflops. CEO Jensen really summed it up very succinctly when he said in this keynote, Hopper is fantastic,

but we need bigger GPUs. And if the compute capacity of Blackwell is a lot of the initial focus, NVIDIA is also clearly trying to tell a different story about its future, right? CNBC. See, Nvidia executives say the company is becoming less of a mercenary chip provider and more of a platform provider, on which other companies can build software. It said Jensen Huang, Blackwell's not a chip, it's the name of a platform. Invidia's enterprise VP Manavir DOS extended this conversation. The sellable commercial product was the GPU, and the software was all to help people use the GPU in different ways.

Of course we still do that, but what's really changed is we have a new commercial software business now. The software that they introduced was called NIM, and the basic idea is to make it easier for people to deploy artificial intelligence applications. Again from CNBC, VP DOS said NVIDIA's new software will make it easier to run programs on any of NVIDIA's GPUs, even older ones that might be better suited for deploying but not building AI. Said DAS, if you're a developer, you've got an interesting model you want people to adopt. If you put it in a NIM, we'll make sure that it's runable on all our GPUs so you reach a lot of people. Now, if you are interested in

some of the more technical parts of the announcement, there's tons that you can go find on that. For example, in his presentation, Jensen explained how the Blackwell GPU is the first chip to combine two separately manufactured dyes into one chip. There were, of course, a lot of other announcements from this presentation as well. We're going to talk specifically about Omniverse, which is software that creates digital twins of real-world items, and is slated to come to Apple's Vision Pro, as well as Project Groot, which is infrastructure for humanoid robots. First, though, I want to talk about the market reaction to this, because again, part of what

has put Nvidia at the center of the conversation is the fact that it has become the standard bearer for AI stock performance. On the one hand, as Bloomberg points out, the announcement of these new trips was widely anticipated, which, quote, made it hard for the presentation's details to impress investors, who sent the shares down about 1.6% in pre-market trading on Tuesday. Bloomberg gets it part of why that might be. They write, for all its success, NVIDIA revenue has become highly dependent on a handful of cloud computing giants, Amazon, Microsoft, Google, and Meta.

Those companies are pouring cash into data centers aiming to outdo their rivals with new AI-related services. The challenge for NVIDIA is broadening its technology to more customers. And that, of course, is where we get some of this additional focus on Nvidia becoming a platform and its software business. And yet, if the immediate response of investors was not to send Nvidia stock to the moon, there is still a sense of long-term strength. For example, Goldman Sachs raised their price target to 1,000, saying we came away from the keynote with renewed appreciation of Nvidia's unique ability to innovate at data center scale, their large ecosystem in breadth of its customer and partner engagements, and ultimately compelling position is one of the key enablers and beneficiaries of the ongoing buildout of the generative AI infrastructure.

Even more than just Nvidia alone, this week we've been seeing the same back and forth that we had throughout last year where macro wobbles compete with AI enthusiasm to reign supreme in terms of how Wall Street is thinking about the markets. Indeed, Reuters wrote a piece yesterday called Wall Street ends higher. Investors juggle fed nerves with AI enthusiasm. Investors, they wrote, were torn between enthusiasm about the prospects for AI on the technology sector and worries about the Federal Reserve's policy update on Wednesday. Said Lindsay Bell, chief strategist at 248 Ventures, This is a market that really wants to hold on to the momentum trade, but what's weighing on investors' minds is what happens with the Fed this week.

Let's talk briefly, though, about some of these other parts of the announcement. Jeremy Dalton, the head of immersive technologies at PWC writes, Nvidia just announced it as bringing its Omniverse platform to the Apple Vision Pro, allowing complex 3D assets to be streamed directly to the headset for further design, assisted by generative AI. By doing this, you don't need to worry about storage or be constrained by the device's processing power. VR filmmaker and YouTuber Hugh Howe writes, Omniverse shows the capabilities of spatial computing by blending 3D photoreal environments with the real world.

Bilal Situ is at the event and writes, I got to try this demo in person today. Mind blown. Immaculate streaming quality, full-res CAD models with real-time ray tracing. Billowell shared the same demo video, which for those of you listening to the podcast, involves modifying a car. And when someone asked you were able to go in the car and still see the room around you, Billowal responded, yes, there's a mixed reality mode and it was surprisingly good. Perhaps even more exciting to many,

with something called Project Groot. Dr. Jim Fan, who I often quote on this show, writes, Today is the beginning of our moonshot to solve embodied AGI in the physical world. I'm so excited to announce Project Grute, our new initiative to create a general purpose foundational model for humanoid robot learning. The group model will enable a robot to understand multimodal instruction such as language, video, and demonstration, and perform a variety of useful tasks. We're collaborating with many leading humanoid companies around the world so that Groot may transfer across embodiments and help the ecosystem thrive. Announced in Jensen's keynote, Project Grute is a cornerstone for the Foundation Agent Roadmap of the newly founded Gear Lab. At Gear, we are building

generally capable agents that learn to act skillfully in many worlds virtual and real. Nate Barcy tried to sum up, Grut will enable a robot to understand multimodal instructions like language, video, and motion. Very soon we will see them cooking, preparing coffee, in supermarkets, changing tires, etc. So what makes something like Project Groot different than, for example, the figure O1 robot that we've talked about recently, is that this is meant to be a system for many different robots, not just one humanoid robot that Invidia is building. So it's coming at the problem in a very different way, assuming a future in which there are many different types of humanoid robots. One additional note on Grute, Invidia also announced a dedicated chip called the Jets and Thor

chip that's specifically designed for humanoid robots. Writes Venturebeat, to make sure humanoid robots can run complex multimodal models like Grut, Nvidia has launched the Jets and Thor computing platform for humanoids. Based on the company's Thor SOC, the computer includes a high-performance CPU cluster, and next generation GPU based on the Nvidia Blackwell architecture, with a transformer engine delivering 800 terraflops of 8-bit floating point AI performance. I think if you zoom out, one of the remarkable things about this really is just how unusual it is to have a hardware company like this,

not even a consumer hardware company, but an infrastructure company, be so frankly hot and at the center of a transformational technology. It's a fascinating thing to watch and something that doesn't show any signs of slowing down. If and as there are more interesting announcements from GTC, I am sure we will be covering them, but for now, that is going to do it for the AI breakdown. Until next time, peace.