The AI Daily Brief: Artificial Intelligence News and Analysis - Did Google DeepMind Just Revolutionize Materials Science?

Episode Date: December 1, 2023

Deepmind's GNoME AI has 10x'd the number of known stable materials, and more than 736 of them have already been successfully created in the lab. Plus Amazon's new Titan image generation model and Microsoft sets up in the UK. Today's Sponsors: Notion - Notion AI. Knowledge, answers, ideas. One click away. - https://notion.com/aibreakdown Listen to the chart-topping podcast 'web3 with a16z crypto' wherever you get your podcasts or here: https://link.chtbl.com/xz5kFVEK?sid=AIBreakdown  ABOUT THE AI BREAKDOWN The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI breakdown, we're looking at some serious materials breakthroughs from Google DeepMind. Before that on the brief, the new open source Chinese model at GPT 3.5 levels, plus new models from Amazon, and a big investment in the UK from Microsoft. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.netnetwork for more information about our Discord, our newsletter, and our YouTube channel. Welcome back to the AI breakdown brief, all the AI headline news you need in around five minutes. One of the things we've been talking about quite a bit recently is the fact that still no one can beat GPT4. Even as we see companies like Amazon launch their own models, which, by the way, we'll be talking about one of those later today,

GPT4 remains the king. And so, of course, I took notice when AI entrepreneur Bindu Ready tweeted, this is insane. And just like that, we have open source Quinn 72B beating GPT4 on some benchmarks. China roars back at toxic late stage capitalism and corporatism plaguing the U.S. join the global open source revolution. Now, I'm not going to comment on the late stage capitalism piece, but I certainly took note of the idea that Alibaba's open source model was beating GPT4 on some benchmarks. However, upon closer inspection, the evals that Quinn beats GPT4 on are in Chinese. Now, that's still good, but it's a little bit more understandable. And basically, we have here yet another GPT3.5 style model. Bin Juan Hui from Alibaba writes, we are proud to present our sincere open source

works, Quen 72B and Quen 1.8B. Quen 72B has been trained on high-quality data consisting of three trillion tokens, boasting a larger parameter scale and more training data to achieve a comprehensive performance upgrade. Additionally, we have expanded the context window length of 32K and enhanced the system prompt capability, allowing users to customize their own AI assistant with just a single prompt. Now, all of this is over on Huggingface, Huggingface.co, slash QWEN, if you want to check it out. Now, as your heart palpitations come down, there is another interesting China-US competitive story that comes through the proxy of Saudi Arabia. Bloomberg is reporting that the U.S. has forced a Saudi fund to divest from an AI chip startup that was also backed by OpenAI Sam Altman.

Basically, a company called Prosperity 7, which is a venture capital arm of Saudi Aramco, invested in the $25 million round for rain AI last year. However, the Committee on Foreign Investment in the United States has now forced Prosperity 7 to sell its shares and unwind the deal. Now, this is not publicly announced. This is all from Bloomberg sources. This fits certainly with a larger pattern. As Bloomberg writes, the U.S. is heightening scrutiny over the activity of Middle Eastern wealth funds, part of a growing resistance towards entities regarded as having close ties with China. CFIUS is reviewing several multibillion-dollar deals this year on concerns they could pose national security risks. Now, the company in question Rain AI is a startup that, quote, designs AI chips inspired by the way the brain works. Obviously, AI chips are a central piece

of the China-U.S. tension, and apparently whatever Prosperity 7 is doing is too valuable in the eyes of the Biden administration to let the Saudis, and by extension, the Chinese have access to it. Now, adding even more complication to this is, of course, the fact that all the reports suggest that Sam Altman has been working to raise billions of dollars for another novel chip project, codename Tigris, and has been seeking out Middle Eastern funds to do so. Bloomberg also makes clear, quote, while Altman is an early investor in Rain AI, it's not clear whether he remains active in the company or how he views the technology today. Now, moving over to the world of big tech and their AI efforts for a moment, when all was said and done, we did not get the rumored Olympus model at Amazon's reinvent, but they also didn't just leave us with the Q Enterprise chatbot that we talked about earlier in the week.

We also had the announcement of the Titan Image Generator, which is akin to a Dolly 3 or a mid-jury, but has a couple notable features. First of all, Amazon claims that it has built-in guardrails against toxicity and bias. Second, every image created with Titan Image Generator has an invisible watermark that can identify it, and, as a generative AI creation. Third, Amazon is extending copyright indemnification to customers who are using Titan's foundation models, including text to image. Basically, if you're building with Amazon's tools,

they're going to protect you against copyright claims. Now, all of this is part and parcel of the bigger approach here, which is that Titan is not a standalone application or website, but something that's available to developers from within Amazon Bedrock. Now, it's quite clear between the watermarking, the guardrails, the copyright indemnity, and the fact that this is available through Bedrock,

that this is an enterprise-focused offering. This is meant to give big companies who are nervous about using something like Mid Journey the comfort to still get in the image generation game. A lot of these efforts are similar to features we've seen from Adobe's Firefly, who has at least in part a similar target in mind. Moving over to Microsoft, Microsoft is spending a huge amount of money on building out its data capacity around the world, and it appears that the UK is going to be one of the major beneficiaries of that strategy.

Announced earlier this week at a conference with UKPM Rishi-Soonak, Microsoft is investing $2.5 billion or $3.2 billion into the company, including opening up data centers that house more than 20,000 GPUs. Said Sunak in a statement, today's announcement is a turning point for the future of AI infrastructure and development in the UK. Now, the UK and Microsoft have had a little bit of a tense relationship over the last few months. Back in April, the UK's antitrust regulator went against Microsoft in its attempt to acquire Activision Blizzard, but subsequent to that, it waived through a restructured version of that deal, which apparently, as Reuters put it, put Britain back in Microsoft's favor. Said Microsoft President Brad Smith,

Microsoft is committed as a company to ensuring that the UK as a country has world-leading AI infrastructure. Now, in addition to just GPUs, this big investment in the UK will include what Microsoft called a training plan to help ensure Britons have the skills they need to build and work with AI. Still, of course, the biggest thing in Microsoft land remains their relationship with OpenAI, and one of the things that people have wondered is whether they will get access to the board after being shocked by the board's action to fire Sam Altman just a couple of weeks ago. According to reporting from the information, via statements from restored OpenAI CEO Sam Altman, Microsoft will be on the board in a way, but as a non-voting observer.

Basically, this will allow Microsoft to have visibility into the board's debates and deliberations, but it won't be able to influence decisions in a direct way. This to me suggests some amount of resilience in the OpenAI structure. I think many folks would have thought that Microsoft would have demanded a board seat in some new governance structure after this whole hullabaloo, but apparently for now at least it's content with this additional visibility. Of course, the board story is still very much up in the air. There are three members currently, Brett Taylor, the former Salesforce CEO, who is chair, Cora CEO, Adam DeAngelo, who's the loan holdover from the last board, and former Treasury Secretary Larry Summers. As part of his

compromise to come back as CEO, Altman is not currently on the board, although that may not be the case forever. Now, also in Sam's first official days back as CEO, we learned that Miram Maradi had gone back to her role as CTO, and that Greg Brockman will be returning as president. Now, one big open question is what happens to former board member and chief scientist Ilya Sutskhaver. He seemed to have been at one point an instigator of Sam's firing, but then also flipped sides and became one of the first people to sign that big letter demanding his return. In a memo to employees, Altman wrote, while Ilya will no longer serve on the board, we hope to continue our working relationship and are discussing how he can continue his work at OpenAI. Now, all of the messaging continues to be that this was

not about AI safety. Outgoing board member Helen Toner wrote on Twitter, Our decision was about the board's ability to effectively supervise the company. We were not motivated by a desire to slow down OpenAI's work. That was repeated in statements from Microsoft. So, I don't know, man, for now, it still remains a mystery. Elon must use his deal book summit interview to reiterate that he wanted to hear from Ilya about what Sam had done that was so egregious that he wanted to fire him, clearly intimating that there was something there. For now, we will just have to wait and see what happens. And that will do it for today's AI breakdown brief. Up next, the main AI breakdown.

Quickly a brief word from today's sponsor. As a listener of this show, I suspect you like to stay up to date on all things AI and tech, which is why you have to check out the chart-topping podcast Web3 with A16Crypto. Produced by venture firm, Andresen Horowitz, Web3 with A16Z is the perfect companion podcast to the AI breakdown. Web3 with A16Z crypto is your definitive resource for the future of the internet. Whether you're interested in the convergence of AI and crypto, or simply curious about what's next. If you need a place to start, they recently released an excellent episode with Stanford Cryptography Professor Dan Bonae and former Google X engineer Ali Yaya in conversation with host Sonal Choxi about the intersection

of AI and Crypto. From fighting deepfakes and proving humanity to large language models like Chatsybt, they cover it all. I highly recommend checking it out, especially if you'd like to learn more about how AI and crypto will impact our everyday lives. Beyond crypto and AI, this show is for creators seeking more ways to truly own their work, for business leaders trying to prepare for the future today, and for innovators exploring trending tech topics. Don't miss out. Follow Web3 with A16Z crypto on Apple Podcasts, Spotify, or your favorite listening app. And now a quick word from today's sponsor. I am a huge Notion user. We're talking multiple accounts for multiple projects. I use it for everything from applicant tracking to note taking to project management, to sharing public

documents to frantically capturing ideas I have while out hiking or just driving around. Given that and given the topic of the AI breakdown, I was excited to learn that they've launched a new AI tool called Q&A. It's like a personal assistant that responds in seconds with exactly what you need. Notion AI can give you instant answers to your questions using information from across your wiki, projects, docs, and meeting notes. For someone like me who makes dozens of notes per day around a huge array of topics, having a built-in AI tool to help recall that is incredibly useful. Now, beyond that use case, think about this. Have an urgent question you normally turn to a coworker to answer? Just ask Q&A instead. It'll search through thousands of documents

and seconds and answer your question in clear language no matter how large or complex your workspace is. Plus, you can trust your data is secure because Notion AI is designed to protect your information. No AI models are trained with your information, the data is encrypted, and answers will never use information from pages you don't have access to. With Notion AI, it's even easier to do your most meaningful work. Try Notion AI for free when you go to notion.com slash AI breakdown. That's all lowercase letters, notion.com slash AI breakdown to try the powerful, easy to use notion AI today.

And when you use our link, you're supporting the show. One more time, that's notion.com slash AI breakdown. Welcome back to the AI breakdown. Today we are doing some big science stuff, and I will be clear that we are way out of my depth of expertise. I am just going off of what I'm reading, but what it seems like to me is a fairly significant announcement, and I don't think I'm alone in that. So what we're talking about is a new tool from Google DeepMind that is called Nome that's basically designed to generate new potential recipes for new materials. Nome came up with 2.2 million potential compounds of which 380,000

seem theoretically stable. Now, to get a sense of why this is significant, you might remember the whole conversation earlier this year about LK99. This was the new material that was for a time this year claimed to be able to act as a room temperature semiconductor. Now, think about how much chatter there was around just one new material that might have really powerful properties, and now go back and apply it to hundreds of thousands. Now, of course, the challenge is these could be useful for building new types of batteries for that semiconductor use case. That is, of course, if they can actually be made. But let's take a step back and actually do an overview that comes from Google DeepMind themselves.

They tweet, Introducing Noam, an AI tool that helped discover 2.2 million new crystals. Crystals are found in everything from the chips powering our phones to solar cells creating clean energy. The model also better predicts the stability of new materials. Noam, which stands for Graph Network for Materials Exploration, was trained using active learning, a technique to scale up a model first trained on a small specialized dataset.

Developers can then introduce new targets, allowing machine learning to label new data with human assistance. This makes Noam well suited to the scientific. of discovering materials, which requires searching for patterns not found in the original dataset. As a result, the model has boosted the precision rate for predicting material stability from 50% to 80%. Over the last decade, scientists have only managed to unearth 28,000 stable materials using expensive and time-consuming computational methods. Based on this record, our research is equivalent to nearly 800 years of knowledge. 380,000 of our stable predictions are now publicly

available to help researchers make further breakthroughs in materials discovery. Teams at Berkeley Lab and others have already independently created 736 of Noam's new candidates in the lab. To build a more sustainable future, we need to develop greener technologies. We hope NOM's promising recipes for new materials could help improve products like batteries for electric cars or even superconductors for more efficient computing. So let's talk a little bit about how this type of process used to work, before the advent of AI and the approach taken by Google DeepMind. They write, in the past, scientists searched for novel crystal structures by tweaking known crystals or experimenting with new combinations of elements, an expensive trial and error process that could take months to

deliver even limited results. Over the last decade, computational approaches led by the Materials Project and other groups have helped discover 28,000 new materials. But up until now, new AI-guided approaches hit a fundamental limit in their ability to accurately predict materials that could be experimentally viable. You can see that human experimentation led to the discovery about 20,000 of these. Additional computational methods increased that to 48,000, and now Nome has increased the number of potential stable compounds to 421,000 overall. So how does Noam actually work? In a section of their blog post called harnessing graph networks for materials exploration, they explain. Again, DeepMine writes, Noam is a state-of-the-art graph neural network model. The input data for

GNNs takes the form of a graph that can be likened to connections between atoms, which makes GNNs particularly suited to discovering new crystalline materials. Noam was originally trained with data on crystal structures and their stability openly available through the materials project. We use Noem to generate novel crystal candidates and also to predict their stability. To assess our model's predictive power during progressive training cycles, we repeatedly checked its performance using established computational techniques known as density functional theory, DFT, used in physics, chemistry, and material science to understand structures of atoms, which is important to assess the stability of crystals. We used a training process called active learning that dramatically boosted

Nome's performance. Nome would generate predictions for the structures of novel stable crystals, which were then tested using DFT. The resulting high-quality training data was then fed back into our model training. Our research boosted the discovery rate of material stability prediction from around 50% to 80%, based on an external benchmark set by previous state-of-the-art methods. They also have a graph that visualizes this. They write, Nome uses two pipelines to discover low-energy-stable materials.

The structural pipeline creates candidates with structures similar to known crystals, while the compositional pipeline follows a more randomized approach based on chemical formulas. The outputs of both pipelines are evaluated using established density functional theory calculations, and those results are added to the Noam database in forming the next round of active learning. So this is a process that is both starting from random as well as starting from known structures, and then when promising results appear, it goes back to feeding both of those pipelines to further refine the model. Now, of course, if these are just recipes, the question is, can they actually be created? Again, from their blog post, the Noam project aims to drive down the cost of discovering new materials.

External researchers have independently created 736 of Noam's new materials in the lab, demonstrating that our model's predictions of stable crystals accurately reflect reality. In fact, a team at the Lawrence Berkeley National Laboratory also published a second related paper in nature called an autonomous laboratory for the accelerated synthesis of novel materials that is meant to, as they put it in the abstract, close the gap between the rates of computational screening and experimental realization of novel materials. To do this, they say, we introduce the A-Lab, an autonomous laboratory for the solid-state synthesis of inorganic powders. This platform uses computations, historical data from the literature, machine learning, and active learning

to plan and interpret the outcomes of experiments performed using robotics. Over 17 days of continuous operation, the A-Lab realized 41 novel compounds from a set of 58 targets, including a variety of oxides and phosphates that were identified using large-scale abanito-fac stability data from the materials project in Google Deep Mind. Synthesis recipes were proposed by natural language models trained on the literature and optimized using an active learning approach grounded in thermodynamics. Analysis of the failed synthesis provides direct and actionable suggestions to improve current techniques for material screening and synthesis design.

The high success rate demonstrates the effectiveness of artificial intelligence-driven platforms for autonomous materials discovery and motivates further integration of computations, historical knowledge, and robotics. So going back to use cases, one of the things that I think is getting people excited about this is how many more shots on goal for potential use cases this approach to discovering new materials really can be. Now, when they say that their discovery of 2.2 million materials, would be equivalent to about 800 years worth of knowledge, one of the examples they give,

quote, 52,000 new-layered compound similar to graphene that have the potential to revolutionize electronics with the development of superconductors. Previously, around 1,000 such materials had been identified. Additionally, quote, we also found 528 potential lithium ion conductors, 25 times more than a previous study, which could be used to improve the performance of rechargeable batteries. So, like I said, this is heady, advanced stuff. It is almost inevitable that there are going to be major questions around how readily creatable these things are, although the early results on that also seem promising. What's more, the identification of potentially stable materials does not guarantee that they will improve performance on any of these particular use cases. But still,

seeing such a sea change in how discovery happens is, I think, a great example of why some people identify scientific discovery as potentially one of the most revolutionary aspects of the AI field. Now, whether this process can discover an LLM that actually outperforms GPT4 remains to be seen. In all seriousness, though, very cool stuff from Google DeepMind, something I will certainly be watching closely. But for now, I appreciate you guys listening or watching as always. Until next time, peace.