The AI Daily Brief: Artificial Intelligence News and Analysis - The Picks and Shovels Businesses Quietly Driving the AI Build Out

Episode Date: August 1, 2024

Explore the unsung heroes behind the AI revolution: the businesses providing essential infrastructure for AI’s rapid growth. Learn about the increasing role of data centers, innovative cooling solutions, and the crucial components powering AI’s expansion. Discover why Wall Street’s view on AI may not capture the whole picture, as these “picks and shovels” companies play a vital role in shaping the future of AI technology. Concerned about being spied on? Tired of censored responses? AI Daily Brief listeners receive a 20% discount on Venice Pro. Visit ⁠⁠⁠⁠⁠⁠⁠https://venice.ai/nlw ⁠⁠⁠⁠⁠⁠and enter the discount code NLWDAILYBRIEF. Learn how to use AI with the world's biggest library of fun and useful tutorials: https://besuper.ai/ Use code 'podcast' for 50% off your first month. The AI Daily Brief helps you understand the most important news and discussions in AI. Subscribe to the podcast version of The AI Daily Brief wherever you listen: https://pod.link/1680633614 Subscribe to the newsletter: https://aidailybrief.beehiiv.com/ Join our Discord: https://bit.ly/aibreakdown

Today on the AI Daily Brief, we're talking about liquid data center cooling and other interesting opportunities in the AI buildout. Before then, the headlines, META says that AI has helped its ad revenue. The AI Daily Brief is a daily podcast and video about the most important news and discussions in AI. To join the conversation, follow the Discord link in our show notes. Welcome back to the AI Daily Brief Headlines edition, all the daily AI news you need in around five minutes.

We kick off today with yet more reporting from Earnings Week, where the market is responding to meta's earnings in a more positive way than they did. earlier in the week with Microsoft. CNBC writes, Metas advertising growth is proof that hefty AI spending is already paying off. This, of course, gets to the big question, which we've been following all summer, and which is also the subject of today's main episode, around ROI and AI hype and market cycles and et cetera, et cetera, et cetera. But in this case, Metas advertising sales in the last quarter increased 22%, which was

double the growth rate at Google. Zuckerberg said that AI is, quote, improving recommendations and helping people find better content as well as making the advertising experience more effective. said Zuckerberg, those are already products that are at scale. The AI work we're doing is going to improve that. Overall, Meta's revenue was up over $39 billion, with 98% of its sales coming from advertising. CNBC writes that Zuckerberg pointed to AI as the foundation behind Metas refreshed online advertising platform, which was battered after Apple introduced an iOS privacy update in 2021. Said Mark Mahaney, internet analyst at Evercore ISI, they rebuilt their ad tech stack using AI,

and they changed their user interface and generated a lot more user engagement because of AI. It's showing up in the revenue and profits now. After the report, meta shares were up 7%. And so now with this, we have the second area after cloud where AI is actually showing up in ROI. Now, do I think that this will make much of a difference when it comes to the narrative competition around whether AI is in a bubble or not?

No, because I don't really think that this comes down to evidence that people are seeing. I think it's much more about preconceived notions in previous pattern matching. Still good to see that meta, who is investing so heavily, particularly in open source AI, is leading not only in the... their models, but also in their revenue performance coming from AI.

One additional thing, Zuckerberg repeated some of what we've heard from other CEOs like Google CEOs in Darby Chai when he said it's hard to predict how this will trend multiple generations into the future, but at this point, I'd rather risk building capacity before it's needed rather than too late. Next up, there are rumors that XAI is considering acquiring character AI, but Elon Musk has denied them. The report originally came from the information, who tends to have better sources on this sort of thing than just about anyone else. And I think that part of the reason that the story got

traction, hold aside the details, is that it does seem to reflect something that people think, broadly speaking, is coming, which is, of course, a wave of consolidation, as AI companies that have started over the last couple years and have spent tons and tons of money have to figure out either business models that actually work, or new sources of capital, or, frankly, just a new place to land. The information writes the discussion may not result in a deal, but the internal debate at XAI points to the types of pairings that are likely to become more common, as smaller AI startups such as Character face the steep costs of training in running their models while competing against deep-pocketed rivals. There is also a talent to mention of this, of course. There's only so

much AIML talent available with top engineers. And fascinatingly, it does seem like something is going to happen with Character. The information says an offer from XAI should one materialize with end months of uncertainty over Character's next steps. Although it isn't at risk of running out of cash soon, it hasn't raised a new round of venture funding in more than a year, a contrast with the rapid funding pace of many AI startups. Character in recent weeks has sought to cut costs and has instituted new processes for approving budgets and headcounts according to an employee. Ultimately, Elon used X to say that they are not considering this acquisition, but who knows? An update from a previous story, the EU's AI Act finally goes into effect today four years after it

was initially proposed. CNBC writes, the legislation applies a risk-based approach to regulating AI, which means that different applications of the technology are regulated differently depending on the level of risk they posed to society. Said Charlie Thompson, a VP at APN, this will bring much more scrutiny on tech giants when it comes to their operations in the EU market and their use of EU citizen data. Now, notably, one thing that is disallowed under the AI Act are things including social scoring systems that rank citizens based on analysis of their data and predictive policing. The predictive policing was interesting to me because the Guardian was reporting that Argentina is considering using AI to predict future crimes. They write, Argentina's security forces

have announced plans to use artificial intelligence to predict future crimes. This week, new president, Javier Miele, created the artificial intelligence applied to security unit, which the legislation says will use, quote, machine learning algorithms to analyze historical crime data to predict future crimes. It is also expected to deploy facial recognition software to identify wanted persons, patrol social media, and analyze real-time security camera footage to detect suspicious activities. Unsurprisingly, there are lots of folks who are very concerned about this. Amnesty International Argentina says, large-scale surveillance affects freedom of expression because it encourages people to self-censor or refrain from sharing their ideas or criticisms if they

suspect that everything they comment on, post, or publish is being monitored by security forces. A new study from the ILO and the World Bank said that between 2 and 5% of jobs in Latin America and the Caribbean are threatened by AI in a way that makes them at risk of being fully automated. Overall, they think that between 26 and 38% of jobs in the region could be exposed to generative AI and impacted by it in some way. The report also suggests that AI could improve the productivity between 8 and 14% of jobs. Lastly today, Taco Bell is set to begin rolling out AI drive-thru ordering in hundreds of locations by the end of the year. Taco Bell is owned by Yom Brands and apparently hopes to roll out AI to all of its drive-thru lanes in the future eventually. Now, other companies have tested similar technology, including Wendy's and White Castle. However, not all of these experiments have gone well.

McDonald's ended its trial of automated order taker, which was an AI that had tested in partnership with IBM. Worth noting, however, that it seems that the issue might not have been AI in general, but the specific partnership, as McDonald's says that they're planning to turn to other vendors instead. I fully expect that the first time you get an AI ordering experience at a Taco Bell, you come over here and tell us. For now that is going to do it for the AI Daily Brief Headlines edition. Next up, the main episode. Today's episode is brought to you by Super Intelligent. As you guys know, Super Intelligent is a platform we are building to help everyone, individuals and teams, maximize their use of AI. We help you figure out how to use AI tools, as well as what to use AI for. And this is really important. The whole goal of superintelligent is not just to give you tutorials and lessons, but to show you how other people like you are actually getting value from AI right now.

For those of you who are still out there working, learning, and grinding deep in the summer, I'm excited to share our best offer ever. If you sign up with code year 50 right now, you will get 50% off the already reduced annual price. Due to popular demand, we have extended this offer for just a couple days. We were going to close it at the end of July. now we are running it through Friday, August 2nd. But if you want this 50% off discount, hand on over to B-super.a.I and take advantage right now. Like I said, this will close on Friday, August 2nd.

The code one last time is year 50 for 50% off the already discounted annual rate for Super Intelligent. See you there. Today's episode is brought to you by Venice. Venice is a private, uncensored generative AI app. It accesses open source models to enable text, image, and code generation without the fear of being spied on or having your data exploited.

Discuss anything with Venice without concern about it being monitored, sold, or given to advertisers and governments. Venice is different because your conversations and creations are kept securely within the browser, never stored or accessible by Venice. Unlike other AI apps, Venice won't tell you what's okay to say or not. Venice won't patronize you. It simply provides direct access to machine intelligence, no topics are off limits, no ideas are taboo.

With Venice, you're in control of the AI as you should be. Pro subscriptions are available for $49 a year or $8. per month. AI Daily Brief listeners receive a 20% discount on Venice Pro. Visit venice.aI slash NLW and enter the discount code NLW Daily Brief. That's NLW Daily Brief, all one word. Welcome back to the AI Daily Brief. One of the big themes for quite some time now, certainly one of the biggest discussions of this summer, has been whether the market has gotten out ahead of itself when it comes to AI evaluations. Now, this has sometimes been translated broadly into a larger question of whether AI is overhyped, but as you've probably heard me say before,

I think it's largely about specific market dynamics and the fact that Wall Street is trying to figure out how to price the future in stocks that are available to trade today, a thing that they're very uncomfortable with. Now, this week has been kind of an interesting one on that front, as we've had sort of a mixed story. On the one hand, we saw that Microsoft's results earlier this week were sort of a mixed story. AI was increasing as a percentage of Azure revenue, but Azure revenue didn't quite hit the estimates. But then meta, as you heard in the headlines today, started to show how AI was paying off in the form of its advertising revenue. And in between all of this, we also had a big burst for Nvidia yesterday. Invita shares were actually up 13% on Wednesday, although the

exact reason for that, I think, is a little bit debatable. On the one hand, it could have been about Nvidia specifically. CNBC characterizes the jump as happening after, quote, remarks from top customer Microsoft and rival chipmaker AMD, signaled there wouldn't be a slowdown in the multi- billion dollar buildout of AI servers based around GPUs. Another way to look at this if we're putting on our skeptical hats is that even if there is a bubble in the hyperscalor buildout, in other words, companies like Microsoft and Google not being able to get the return on their investment in AI, in the meanwhile, Nvidia and other chipmakers continue to benefit because they're cashing checks all along the way. UBS analyst Carl Kyrsid wrote in a note, Microsoft earnings report may

encourage most Nvidia and semiconductor investors as Microsoft's CAPEX came in much hotter than expected at 19 billion in the quarter. Other analysts are, argued that their peers weren't thinking about growth in AI in general their correct way. Morgan Stanley analyst Joseph Moore, for example, wrote, our perception is that the market is taking a very glass-half-empty view of some of the hyperscale comments, where there is a clear desire on the part of customers to continue to commit resources to developing multimodal generative AI. In other words, this particular analyst is arguing, to some extent that other investors may be missing the forest for the trees.

Now, it wasn't just NVIDIA who had a good day. AMD reported better than expected sales and earnings, and CEO Lisa Sue said that the company expected $4.5 billion in AI chip sales this year, which was an 11% increase over what it had previously estimated. Goldman Sachs analysts said, we believe AMD's positive data center GPU business outlook, and Microsoft's comments indicating a sustained increase in CAPEX through fiscal year 25 bode well for Nvidia. Now, of course, we also had a very doveish FOMC meeting that seemed to all but confirm that rate cuts were coming in September, so that also could have had some impact on the particular pop, but I think that the larger dynamics remain.

What's more, the betting odds are that there's a lot more AI infrastructure buildout to come before any even theoretical bubble pops. One of the big stories earlier in the year was OpenAI CEO Sam Altman, flying around the world trying to convince companies to go in on a $7 trillion scheme to increase AI chip capacity. And so the question that we want to ask for the rest of this show is to the extent that there are questions around whether the hyperscaler big tech companies themselves, the Microsoft stocks, the Google stocks, etc., are the right place to bet on.

AI, are there other less obvious areas? I will also use this moment to remind you that nothing I ever say on this show or any other is financial advice, but one interesting question might be what else goes into data centers other than just the chips themselves? In May, Goldman Sachs posted a report, AI is poised to drive 160% increase in data center power demand. They write, on average, a chat GPT query needs nearly 10 times as much electricity to process as a Google search. In that difference lies a coming sea change in how the U.S., Europe and the world at large. will consume power and how much that will cost. So basically, this report estimates a few things. First of all, they're betting that there will be a significant increase in the percentage of global

power that data centers consume, from 1 to 2% today to 3% to 4% by the end of this decade. This, of course, has lots of implications. As Goldman Sachs points out, since 2020, the efficiency games appear to have dwindled and the power consumed by data centers has risen. Some AI innovations will boost computing speed faster than they ramp up their electricity use, but the widening use of AI will still imply an increase in the technology's consumption of power. By 2028, they expect that AI will represent around 19% of data center power demand. This has other implications as well. U.S. power demand, which has been around zero for the last decade, is likely to rise two and a half percent over the next decade, which, as they point out, is a type of spike that hasn't been seen in the U.S. since the early

years of the 20th century. And so, as they point out, there is going to be a lot of work to get data center infrastructure and the power grid ready for this sort of new demand. They estimate that Europe needs trillion-plus of investment to prepare its power grid for AI. And of course, if they're right, that creates lots of interesting opportunities for investors to hone in on. Now, one of these little areas that I've recently been exploring is data center cooling. A recent report called moving beyond the limitations of traditional data center cooling digs into the state of this field and how it's changing. Analyst Stephen Hill writes, in spite of the remarkable advancements we've witnessed in the world of computing over the last few decades, one of the most persistent challenges that still

remain for data center operators lies in the basic problem of removing heat. It's a simple fact that as servers and other infrastructure devices get faster and more powerful, they also consume an increasing amount of electricity. And as everyone in the IT industry already knows, the majority of the electricity that passes through a computer exits in the form of heat. For data centers, it also requires nearly the same amount of energy to remove it. This, he said, leads to the delicious irony of computing, no matter how powerful our systems get, one of the key challenges still comes down to the relatively primitive issue of heat management. He'll then go through what he calls the options. Air cooling, he identifies as ubiquitous but inefficient.

He writes, the majority of data centers have continued to depend on the tried and true raised floor model of air cooling. Much like a refrigerator, raised floor cooling depends on the circulation of cooled air throughout the entire machine room. Hot air rising from the systems is gathered from the ceiling and pass through computer room air handlers, which are devices that are located at strategic locations at the periphery of the room. CREH is based on chilled water or computer room air conditioners based on refrigerants,

cool the air and then deliver it below a tiled floor that's raised with a metal framework between 24 and 48 inches above a concrete subfloor. One of the problems he points out is that you can only push so much air through the holes in a given tile. This has become more of a problem as the design of data centers have changed. Hill continues, the majority of high-density data center operators have adopted air as the model for cooling massive cloud data centers, likely because the substantial evaporating cooling solutions they require continue to be

energy-efficient at scale for moderate workloads. But he says this also presents problems because of the extraordinary water consumption requirements of these facilities. A large cloud data center can consume billions of gallons of fresh potable water per year. Now you, the listener may have heard this start to come up as one of the critiques of the environmental impact of AI, and so again, we're left with the question of what other solutions might there be. The next that Hill points to is water-based direct liquid cooling, which he calls better but challenging. Hill points out that water is about 24 times more efficient than air when it

comes to transferring heat, but, as he points out, water-based cooling isn't without its challenges simply because water and electricity don't really mix particularly well. Hill writes, the majority of in-server water cooling is based on the direct-to-chip or D-2C model, where pressurized water is passed through a water block covering the chip, and then return to a radiator or a chilled water source for recirculation. D2C water cooling, he writes, is clearly an improvement on air-only cooling for hosting higher-density applications within an existing air-cooled data center. But he writes it's hardly common and almost never used as the primary source of cooling.

Next, he comes to dielectric liquid cooling, which he considers evolving as a viable option. He writes, one of the earliest uses for refrigerant cooling was the Kray 2 supercomputer that was introduced in 1985. An incredibly powerful system for its time, but the only way to keep the extremely high-density circuit boards cool was to bathe them in a flowing dielectric coolant that wouldn't react with energized components. The fluorineuric coolant developed by 3M was pumped through the chassis and then bubbled up

through a dramatic see-through waterfall chamber, almost as large as the computer itself, where the refrigerant reverted back from gas to liquid for recirculation. It seems that everything old is new again on a revolving basis in IT, and the same holds true for refrigerant-based cooling. Based on a similar principle as our air conditioners and refrigerant freezers, this model leverages a cycle where heat is extremely efficiently transferred through the evaporation and condensation of a fluid with a relatively low boiling point. One of the versions of this he points to is called immersion cooling.

There are two varieties of this single phase and two phase, which he say both require a substantial tank filled with liquid, with the main difference being whether or not it's based on the evaporation or condensation process of refrigeration. Now, the latest entry, he says, is something called refrigerant-based two-phase D to C cooling. Try saying that 10 times fast. He said it offers some of the best attributes of earlier options,

in particular the way it leverages the heat transfer enabled by the phase change of the refrigerant fluid from liquid to gas and back again in a closed cycle. Like a water-based D-to-C system, primary cooling is provided by a cold-plate heat sink that is attached to the primary heat sources such as CPUs, and potentially any other high-density ICs that may become major heat generators. However, unlike water-based systems, two-phase can remove heat by taking advantage of both sensible and latent heat of vaporization. It also requires four to nine times less flow rate at the chip compared to a single-phase water-based cooling. The example he points to is a company called a Celsius's new cool platform. He writes, their processes specifically optimized to remove heat from

relatively small electronic components. The cold plate known as the vaporator, in contact with the processor is supplied with a liquid refrigerant, where it converts to gas in the presence of heat. The warmer gas flows through a manifold as part of a closed loop to an integrated heat exchanger where it condenses by transferring its heat to facility water or another refrigerant. And then there's a heck of a lot more about this specific technology. Now, in January of this year, data center dynamics.com wrote a piece called 2024, the year of liquid cooling. The Puehling. Tipping point for liquid cooling has arrived, they say, with a few choices on technology. In that they quote Josh Clayman, who's the CEO of that company of Celsius, who says liquid cooling

as a sort of pragmatic solution has always been niche. But we're seeing the dynamic rapidly changing. Mainstream CPUs are becoming much hotter after years of incremental increases in wattage, so suddenly we're seeing a j-curve in terms of the heat that these chips produce. Now, there's a lot more that's interesting about this. I kind of went down a little bit of a rabbit hole. But the point relative to this show, which is obviously much more technical than we normally get is that I believe that one of the things that's going to happen, as Wall Street digs deeper into

these questions of the short, medium, and long term of AI, I think that whatever conclusions they come to when it comes to hyperscaler valuation, and even base level infrastructure valuations in the form of companies like Nvidia or AMD or Intel, they're likely quickly going to expand their field of view and horizon to other parts of the infrastructure buildout that are going to enable this whole transformation, which regardless of what Wall Street thinks, the hyperscalers don't seem to show any signs of slowing down anytime soon. I think that there's going to be a lot more focus on companies like this Ascelsius and other competitors in that space. And I think there's going to be really interesting opportunities

to start to dive deeper and figure out what that infrastructure bill that's actually going to look like. Anyways, guys, that will do it for today's episode. Like I said, a little bit more exploratory and ponderous. But since it's the summer and since we've been having this conversation over and over about bubble or not, valuations too high or not, I think digging a little bit deeper, peeling back the surface understanding what else is happening, what industry areas are coming to the four to actually enable this transformation is a really interesting place to spend some time. For now, though, that is going to do it for today's AI Daily Brief. Until next time, peace.