The AI Daily Brief: Artificial Intelligence News and Analysis - Amazon Joins Google and Microsoft in AI Nuclear Buildout

Episode Date: October 18, 2024

Amazon joins Google and Microsoft in a major push toward nuclear energy to meet AI’s growing power needs, committing to 600 megawatts in nuclear projects. The announcement highlights how big tech is relying on small modular reactors to meet climate goals and sustain the energy demands of AI advancements. Concerned about being spied on? Tired of censored responses? AI Daily Brief listeners receive a 20% discount on Venice Pro. Visit ⁠⁠⁠https://venice.ai/nlw⁠⁠⁠ and enter the discount code NLWDAILYBRIEF. The AI Daily Brief helps you understand the most important news and discussions in AI. Subscribe to the podcast version of The AI Daily Brief wherever you listen: https://pod.link/1680633614 Subscribe to the newsletter: https://aidailybrief.beehiiv.com/ Join our Discord: https://bit.ly/aibreakdown

Today on the AI Daily Brief, Amazon joins the nuclear party, while NVIDIA continues its move to other parts of the AI stack. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show notes. I was at an AI dinner last night featuring investors, some big enterprises and a number of different startup entrepreneurs. And one of the big points of discussion was just how dominant in Vindia really is. Now, part of the conversation is, of course, how they are the company that has made the most money out of the whole AI revolution so far, but another part of this was how they are quietly moving across the entire stack. In other words, they are not content to just be the

chips underneath everything they are working on basically all the other pieces of AI as well. Lending credence to that conversation, Nvidia quietly released a new open source model that seems to be blowing away the benchmarks. Called the Lama 3.1 Nemotron 70B instruct, the model was uploaded to Hugging Face without fanfare on Tuesday. The model apparently performs better across major benchmarks than OpenAI's GPT40 and Anthropics Claude 3.5 Sonnet. As you can probably tell from the name, the model is a refinement of META's open source Lama 3.1 model. The training featured RLHF for reinforcement learning from human feedback, and overall Venturebeat writes, Nvidia's latest model release signals just how fast the AI landscape is shifting. While the long-term

impact of Lama 3.1 Nemotron 70B instruct remains uncertain, its release marks a clear inflection point in the competition to build the most advanced AI systems. These recent releases, particularly the open source NVLM project, have shown that Nvidia's AI ambitions go beyond just competing. They are challenging the dominance of proprietary systems like GPT40 in areas ranging from image interpretation to solving complex problems. Now, as the model has only been out for a couple of days, people are still in the early stages of figuring out what it can do. Still, as Venturepeat points out, the model correctly answered the question how many R's are in strawberry with a detailed and accurate response, which is a classic challenge for LLMs.

Now, not everyone is convinced. X user Centex reran the benchmarks and commented that it wasn't quite as good as claimed, but still, this brings up really interesting questions about the commoditization of models. Will companies and individuals always be willing to pay for the state of the art, or will we reach a point where cheap but good enough models are frankly good enough? Speaking of competition in areas other than the state of the art, Mistral has released a new set of models that are optimized for laptops and phones. They are calling it Le Ministrao and released two models 3B and 8B, both of which have a context

window of 128,000 tokens, and which they write were, quote, built to provide a compute efficient and low latency solution for local privacy first inference for critical applications such as on-device translation, internetless smart assistance, local analytics, and autonomous robotics. Ultimately, there's nothing huge here other than a reinforcement of the larger trend lined, which is that even as competition at the state of the art increases, so too does competition for these smaller, cheaper models as well. Moving over into the world of energy,

which, by the way, will also be the subject of our main episode, little-known data center startup Crusoe Energy has formed a $3.4 billion joint venture with Blue Owl Capital to build a 200-mawatt data center in Abilene, Texas. Once complete, the facility will house one of the largest supercomputers in the world. The press release was a little coy, stating the project is 100-per-long-term, leased to a Fortune 100-Hiperscale tenant,

with occupancy expected to begin in the first half of 2025. The design will be optimized for direct-to-chip liquid cooling and will also accommodate air cooling. At completion, the data center will be able to operate up to 100,000 GPUs on a single integrated network fabric, quote, advancing the frontier of data center design and scale for AI training and inference workloads. That description could only fit the new data center being built by Oracle for use by OpenAI, which is exactly what the information reported. For some, the curious part of the story is why a project so big would choose a relatively obscure company to design and build it. Crusoe Energy began as a specialist in building Bitcoin mining facilities.

There are a number of big differences, but one of the things Crusoe became known for was cutting-edge cooling systems. It sounds as though this new AI cluster will leverage a number of different experimental features, so perhaps Crusoe is the only company with the technical chops to pull it off. It also sounds as though the project will be highly unconventional in a number of other ways. The deadline is extremely tight with OpenAI wanting to fire up the GPUs by early next year. Crusoe described this as a, quote, record-setting construction timeline. The press release added, The notional power plan for the site includes both on-and-offsite renewable resource,

including surrounding wind developments and a potential future large-scale on-site solar installation. The goal is to optimize existing renewable power resources and incentivize new greenfield renewable power development. It sounds then as though the facility will tap into local renewable energy projects that have been constructed but are not yet connected to the grid. This was another specialty of Crusoe in the Bitcoin mining space. The firm had its origins in deploying shipping containers filled with miners adjacent to oil wells, capturing methane waste to power their generators. For all the discussion in the Web 3 world of the intersection of AI and Web 3, so far it seems like the biggest overlap is taking advantage of the data center

expertise of the mining adjacent companies to take on projects that no one else would be crazy enough to take on. One more today, Photonic Computing Startup Light Matter has raised $400 million to solve one of the biggest bottlenecks in next generation data centers. Their product is a high bandwidth switch that uses light waves rather than wires to pass communications between GPUs in a training cluster. Theoretically, this could be a much faster way to connect the units together. Currently, the top-end data centers use 100,000 GPUs, but they all need to be connected together, allowing them to function somewhat like a single machine. This scale is already banging up against the limits of current technology and performance issues

are going to start to hinder further advances. Light Matter CEO Nick Harris said, Hyper-scalers know that if they want a computer with a million nodes, they can't go do it with Cisco traditional switches. Once you leave the rack, you go from high-density interconnect to basically a cup on a string. Now, the current state of the art is NVIDIA's NVL-72 platform, which wires together 72 blackwall units in a single rack. The issue comes from the connections between racks. Harris added, for a million GPUs, you need multiple layers of switches and that adds a huge latency

burden. You have to go from electrical to optical to optical. The amount of power you use and the amount of time you wait is huge, and it gets dramatically worse and bigger clusters. Harris is convinced his company can deliver, and it seems like that conviction is hard one. He said finally, Photonics is coming way faster than people thought. People have been struggling to get it working for years, but we're there. After seven years of absolutely murderous grind. That's going to do it for today's AI Daily Brief Headlines edition. Next up, the main episode. Today's episode is brought to you by Plum. Generative AI promises to supercharge your productivity and give you superpowers, but if you're not an engineer, trying to harness AI

can be incredibly frustrating. Hours wasted wrestling with complex tools only to give up when they don't work. We all have tedious tasks we'd love to automate and challenges AI could solve, but few of us have the skills to fully leverage these game-changing technologies. That's where Plum comes in. The mission? To make automating your work feel like magic. Imagine typing out AI, read my Gmail and ping me in Slack when something critical comes in, and watching it come to life before your eyes.

No coding required. Whether you're a marketer, salesperson, or founder, Plum enables you to create custom AI workflows in minutes, not hours. Check out UsePlum.com, that's Plum with a B, for early access to the future of Workflow Automation. Today's episode is brought to you by Venice. AI companies, store your entire conversation history and attach it to your identity forever. That's every question you ask, every answer you receive, every image you generate, every thought

you share with the machine it's all being spied on. If you trust all the companies, hackers, and NSA board members that will ever have access to your AI conversations, then rejoice, for you are well served. For the rest of us, Venice is an alternative. Venice is a powerful AI app for text, image, and code generation that respects you as a sovereign individual, and believes privacy and free speech are not only human rights, but necessary for civilizational advancement. Private, permissionless, and uncensored, you can try it for free without an account. AIA Daily Brief listeners receive a 20% discount on Venice Pro. Visit venice.a. slash NLW and enter the discount code, NLW Daily Brief. That's NLW Daily Brief. All one word.

Today's episode is brought to you by Super Intelligent, which is of course our platform that helps you learn how to use AI tools and perhaps even more importantly, gives you ideas on the best use cases that are actually going to help you achieve whatever it is you want to achieve. To recognize the end of summer and back to school slash back to work, we are running our best promotion ever. When you sign up for Super Intelligent, using code so back, your first month will be 100% free. The platform features over 600 fun, highly practical AI tutorials that get you using AI fast and with an eye to actually transforming how you get things done. We've just launched super for teams. So if you have a group of people at your

company that want to figure out how to use AI together, I highly suggest you check it out. But for those of you who are using Superintelligent as an individual, once again, if you sign up for Superintelligent between now and the end of the month using code so back, you will get your first month 100% free. Go to be super.aI and check it out today. Today we are talking about something that at this point is absolutely undeniably a trend, and that is, of course, AI going nuclear. Now, one of the things that is very clear in has been clear for a while is that one of the bottlenecks and constraints on the continued buildout of AI was ultimately going to be power, especially as we push further and further into

the frontier, the need for energy for compute for data centers just goes up and up and up. Now, nuclear has long been discussed as a potential answer to that challenge, and it's finally starting to happen in a real way. In September, we got word that Microsoft was making a deal to help restart Three Mile Island, and earlier this week, Google announced a deal with Cairo's to purchase nuclear energy for multiple small modular reactors. Amazon has now joined Microsoft and Google in making a big commitment to nuclear power with the company announcing three deals on Wednesday, each focused on the construction of small modular reactors.

Two of the deals are partnerships with local energy companies in the Pacific Northwest and Virginia to build out new generation, while the third is a $500 million investment in nuclear energy startup X Energy, who are developing pebble bed SMRs. The Pacific Northwest project will serve as the test bed for this new technology. All in, Amazon is looking to partner in the development of around 600 megawatts of nuclear power across the two sites, with room for additional expansion over time. Amazon said that they expect the reactors to be up and running in the early 2030s,

and overall, X energy is seeking to bring 5 gigawatts of power online by 2039. Like Google's nuclear announcement from earlier this week, Amazon presented the news as a climate initiative. Matt Garman, CEO of Amazon Web Services said, Nuclear is a safe source of carbon-free energy that can help power out operations and meet the growing demands of our customers while helping us progress towards our climate pledge commitments to be net zero carbon across all our operations by 2040. One of the fastest ways to address climate change is by transitioning our society to carbon-free energy sources, and nuclear energy is both carbon-free and able to scale, which is why it's an important area of investment

for Amazon. Our agreements will encourage the construction of new nuclear technology that will generate energy for decades to come. Now, what's starting to become really clear is that the big tech firms believe that the only way that they can meet both their emissions targets as well as their need for increased consumption is the full embrace of nuclear power. These recent announcements have completely cast off any hesitancy around nuclear that was common even up to just a few years ago. Now, it's a little bit out of scope for this particular show. The Cliff Notes version of SMRs is that the idea is to use new designs that have less risk of meltdown. Amazon and Google are each choosing a competing design. The other big point in addition to risk is that SMRs should be

quicker and cheaper to build out than conventional nuclear power plants. But another interesting to mention of this is how big tech going full force into nuclear power might change the way that the U.S. thinks about infrastructure investment in the green transition. Nuclear is currently a little under 20% of the U.S. power supply. The last new reactor was built in Tennessee in 2016. That construction commenced all the way back in 1973 and was put on hold for 32 years until 2007. That means that even once work was restarted, it still took almost a decade to bring the reactor online. Most U.S. nuclear power plants were constructed between 1970 and 1990, and typically have a 50-year lifespan. That means that even if electricity demand remains steady, there would still

be a need to refurbish or replace the entire fleet by 2040. There's been a recognized need to move on this issue for some time, but until recently there hasn't been any urgency. Electricity demand has basically been stagnant in the U.S. for decades, and tight regulations have made the development of new reactors a challenging proposition. This goes double for new reactor designs. A new design hasn't been approved by U.S. regulators in over 50 years until recently. Contrast the situation to other nations. China has built 55 nuclear reactors over the past 10 years, bringing on more than 34 gigawatts of new power. They have another 23 under construction. India has plans to build 18 plants over the next 10 years. Basically, for some time now, there has been a nuclear renaissance happening around the rest of the

world, and it appears that AI is bringing it to the U.S. Perhaps the key to making this happen is stable demand. The hyperscalers are signing these nuclear deals committing to be anchor customers. That is, they are committing to purchase enough energy to make the new reactors viable all in their own. This, in many ways, was the missing piece for U.S.-based nuclear. New reactors are extraordinarily costly, so without guaranteed stable demand in place, they can't be built. Now, another part of the story is, of course, the larger shift in narrative. Mark Nelson on X writes, Sierra Club last month quietly reversed 50 years of anti-nuclear advocacy to endorse nuclear energy as a clean source of power.

In their new electricity policy report, they call for using nuclear to protect the climate. Now, I think one could argue that this is actually more like a tacit acknowledgement that it's necessary rather than an overt reversal of their position, but still it shows just how inevitable this shift is. Rudy Havinstein on Twitter also points out how this is a broader shift as well. He writes, it's odd how for decades anyone pro-nuclear was some sort of anti-baby seal far-right maniac, but once Microsoft and Amazon needed, it's okay. Many in the AI community are just focused on the fact that this is yet another indication of the hyperscaler's belief in what the future looks like.

Bindu Ready writes, all the cloud vendors anticipate a massive surge in AI inference and are ensuring they will have energy in the future. Key takeaway, AI training and inference will grow exponentially over the next decade, and it will require a lot of power. Ethan Malik points out, the Google deal to acquire small nuclear reactors to power data centers increases the odds that we will see AI model scale through at least three more generations in orders of magnitude post-GPT5 to 2030.

According to Epic, power was the binding constraint. Lori Mila Verta, however, points out something important about this discussion. She writes, The story on electricity demand growth from AI and data centers is ridiculous, overblown. Excellent graph from the new IEA World Energy Outlook. Data Centers projected to drive about 5% of electricity demand growth. Please focus on the real challenges and tune out pundits who peddle this story. The point here is that when it comes to real changes in energy use, the bigger story is about the industrialization of the global south, and frankly, the air conditioning that comes

along with wealth growth. However, what makes data centers interesting is that we're talking about the need to grow power supply in the West for the first time in decades. So it is a big story, just not perhaps the big story that it seems sometimes. Overall, it is a super interesting moment and yet one more indicator of just how triple quadruple quintupled down the big companies are on the AI future. That's going to do it for today's AI Daily Brief. Appreciate you listening or watching as always. And until next time, peace.