The AI Daily Brief: Artificial Intelligence News and Analysis - Mark Zuckerberg Wants to Make Open Source AGI

Episode Date: January 19, 2024

Other revelations from an Instagram reel include that Meta will have around $20B worth of compute by the end of 2024; that Llama 3 is being trained, and that it intends to be state of the art -- not just the best open source model. ABOUT THE AI BREAKDOWN The AI Breakdown helps you understand the most important news and discussions in AI.  Subscribe to The AI Breakdown newsletter: https://theaibreakdown.beehiiv.com/subscribe Subscribe to The AI Breakdown on YouTube: https://www.youtube.com/@TheAIBreakdown Join the community: bit.ly/aibreakdown Learn more: http://breakdown.network/

Today on the AI breakdown, we're looking at Mark Zuckerberg's new ambition to build open source AGI. Before that on the brief, ASU and OpenAI signed its first university partnership. The AI breakdown is a daily podcast and video about the most important news and discussions in AI. Go to Breakdown.network for more information about our YouTube, our newsletter, and our Discord. Welcome back to the AI breakdown brief, all the AI headline news you need in around five minutes. The relationship between AI and education has been interesting. On the one hand, it is contentious in the sense that many professors and faculty members and teachers have been concerned that the rise of things like chat chepti just means endless cheating.

On the other hand, others have seen it as an incredibly powerful tool for changing the way that we learn. What if every student had an incredibly sophisticated tutor for any subject they needed? Throughout that process, there's been a lot of evolution. For example, New York public schools initially banned chat chaptiT, but then went the other direction after they considered it a little bit more. Well, now today, OpenAI has announced its first partnership with a university, that university being Arizona State. So the three areas that ASU is saying that they want to focus on for their use of Chad GPT enterprise are one, enhancing student success,

two, forging new avenues for innovative research, and three, streamlining organizational processes. So one of the things that I think is really interesting about this is that it reflects a phenomenon that I think we are going to see more across not just the educational world but the corporate world as well. Take this quote from Chief Information Officer Kyle Bowen. Our faculty and staff were already using ChatGBTBT, and after the launch of ChatGPT Enterprise, which for us addressed a lot of the security concerns we had, we believed it made sense to connect with OpenAI. So in other words, this was a bottom-up movement. People just naturally adopted a tool which they found really interesting and useful, and then that ratcheted up at some point to the organizational level. Now, the organization

had some concerns but those were addressed by a specific product offering, leading to the opportunity for this larger partnership. I think in general this is what we're going to see in businesses as well. We're going to see employees make the decision themselves to adopt some of these tools, be it chat GPT or something else. They're going to be productive and then show their friends and colleagues how they're using those tools who are going to then adopt them as well. And at some point, managers and bosses will decide whether it makes sense to just go big across that tool set. Now, one of the things that's valuable about when an organization does decide to invest, is that all of a sudden there's more space for cross-pollination and for best practices

from one section of the organization to get to others. For example, at ASU, the university is going to begin taking submissions from faculty and students on how and where to use chatch EBT. Now, it sort of makes sense that this is coming from ASU, as last year around all of this excitement around chatchapit, they had already launched an AI accelerator program, and they had even begun to offer classes focused on things like prompt engineering. Now, this is a story I would have covered either way, because I think it's really interesting and reflective of a trend. But I will note that I got this news from so many people at ASU yesterday. And one, the fact that they're listening to the AI breakdown suggests to me that they have

great taste and are very smart. But two, the fact that it wasn't just coming from PR people, but from professors and faculty members and lots of folks, suggests that this is something that that university is really excited about. So I think it is worth watching. Over in the K-12 world, Axios Columbus shows how Ohio is preparing to launch an AI policy for their K-12 schools. Per a new report, they are one of 11 states that are preparing policy guidance for K-12 schools on generative AI. Here once again you have a situation

where lots and lots of teachers are already using AI in their classrooms in various ways. The sort of guidance that comes from the district or state level gives them more confidence that they're not going to be punished or reprimanded for doing so and help shape the bounds of what they can do, which of course will help expand what they can do. Now moving away from education into the business world for a moment, the Financial Times reports that Cohere is in talks to raise as much as $1 billion. The company's last fundraise was in June 2023 when it raised $270 million at a valuation of $2.2 billion. That round included not only venture capitalists like Index, but also strategics like Nvidia and Oracle. Cohere is led by influential ex-Gouglers like Aidan

Gomez and is trying to go after the enterprise LLM market as opposed to the consumer market like chat GPT. Still, the capital that they are looking to raise somewhere between a half million and a billion dollars shows that even when focused on something of a niche, the cost to compete in this space are just absolutely enormous. As if to make a point on that, the information has shared a special report around how Google has developed a special class of stock as a way to fight the AI talent wars. The information writes, even as Google makes a major round of staff cuts, it is tapping a special pool of stock compensation to retain its top artificial intelligence researchers, as rival OpenAI lures them with multi-million dollar pay packages. Select researchers at Google's

DeepMind Unit have received large grants of restricted stock worth up to millions of dollars per person. Leaders of the unit have been increasingly using a little-known advanced compensation program to compete with OpenAI's eye-popping offers. Now, basically, these are accelerated vesting programs. So instead of the standard four-year vesting period for grants, some of these vest as fast as a single year. To me, there is nothing particularly surprising about this. It just shows the intensity with which these organizations are competing, and it's clearly a two-part competition with compute resources on the one hand and talent on the other. Meanwhile, over at Microsoft, the company has just announced that its AI-powered reading coach is now available at no cost to anyone who has a Microsoft

account. They wrote in a blog post, it's well known that reading is foundational to a student's academic success. Study show that fluent readers are four times more likely to graduate high school and get better jobs. With the latest AI technology, we have an opportunity to provide learners with personalized, engaging and transformative reading experiences. So once again, here we are back at the education theme. And part of the reason for that is I just think AI is going to open up so many new avenues for more personalized learning, which has been a drum that education people have been beating for years, but now we actually have the technology to make it so.

Lastly, today, a little update from the stock market. Nvidia and AMD both reached new all-time highs on Thursday. If you thought that 2024 was going to slow down the chip wars, think again, because despite 127.6% gains for AMD in 2023 and 238.8% gains for Nvidia, the slowdown doesn't appear in sight. Anyways, friends, that is going to do it for today's AI breakdown brief. I'll be back soon with the main AI breakdown. Welcome back to the breakdown. I think that everyone believed that 2024 was going to be another big year in the AI arms race. And starting off right fresh, we have some big, bold statements from Mark Zuckerberg over at Meta. That's certainly

reinforce that point. Comically summing up the situation is James Mishra who tweets, Zuckerberg in 2004. I want to look at picks of Harvard chicks. Zuckerberg in 2024. We will build AGI. We will storm into heaven and execute God for crimes against humanity. We will build a vision of the human race that encompasses a thousand sons. So what is James referring to here? Well, it is this reel that came out on Instagram yesterday, where Zuckerberg writes some updates on our AI efforts. Our long-term vision is to build general intelligence, open-source, it responsibly and make it widely available so everyone can benefit. Now, I think this is a really significant announcement, and so let's actually watch it in

its entirety. I am bringing META's two AI research efforts closer together to support our long-term goals of building general intelligence, open-sourcing it responsibly, and making it available and useful to everyone in all of our daily lives. It's become clearer that the next generation of services requires building full general intelligence. Building the best AI assistance, AI's for creators, AI's for businesses, and more. That needs advances in every area of AI, from reasoning to planning to coding to memory and other cognitive abilities. This technology is so important and the opportunities are so great that we

should open source and make it as widely available as we responsibly can so that we everyone can benefit. We are building an absolutely massive amount of infrastructure to see. support this. By the end of this year, we're going to have around 350,000 in Vdia H-100s, or around 600,000 H-100 equivalents of compute if you include other GPUs. We're currently training along with 3, and we've got an exciting map of future models that we're going to keep trading responsibly and safely to. People are also going to need new devices for AI, and this brings together AI and the Metaverse. Because over time, I think a lot of us are going to talk AI's frequently throughout the day. And I think a lot of us are going to do that using glasses,

because glasses are the ideal factor for letting an AI see what you see and hear what you hear, so it's always available to help out. Rayban meta-glasses with meta-a-i are already off to a very strong start, and overall across all this stuff, we are just getting started. All right, so tons to dig into here. Let's kind of take things one by one. First of all, right leading the pack, easily the thing that has been. being most discussed is the fact that Zuckerberg has identified creating artificial general intelligence as the company's new clear goal. Now on top of that, he says that to the extent that they can reasonably and safely, they want to open source it. These are two very big things. Other companies

like OpenAI have, of course, long had AGI as their core mission. This is a new declaration from Zuckerberg and is likely to shape the way that they design their efforts going forward. Now, when it comes to this idea of open sourcing it, that's also a fairly radical and important position relative to the rest of the field. This really cements meta on one very clear end of the spectrum of the AI safety discussion that is frankly diametrically opposed to many others on the opposite end of that spectrum. Now, alongside this reel that was released on Instagram, Zuckerberg did some interviews with The Verge and others. They tried to get a little bit more information about what his definition of AGI was, and he said, I don't have a one-sentence pithy definition.

You can quibble about if general intelligence is akin to human-level intelligence, or if it is like human plus, or if it is some far future superintelligence. But to me, the important part is actually the breadth of it, which is that intelligence has all these different capabilities where you have to be able to reason and have intuition. Zuckerberg also thinks that we will not have some singular moment, but that AGI will arrive as a gradual process. He said, I'm not actually sure that some specific threshold will feel that profound. The Verge also asked Zuckerberg more about his position on openness. He said, I tend to think that one of the bigger challenges here will be that if you build something that's really valuable, then it ends up getting very concentrated. Whereas if you make it more open,

then that addresses a large class of issues that might come about from unequal access to opportunity and value. So that's a big part of the whole open source vision. He also very clearly tried to contrast meta's approach with that of open AIs, although he didn't say the words, open AI. Quote, there were all these companies that used to be open, used to publish all their work, and used to talk about how they were going to open source all their work. I think you see the dynamic of people just realizing, hey, this is going to be a really valuable thing, let's not share it. Zuckerberg also alleged basically some amount of regulatory capture or safety capture saying, the biggest companies that started off with the biggest leads are also in a lot of cases,

the ones calling the most for saying you need to put in place all these guardrails on how everyone else builds AI. I'm sure some of them are legitimately concerned about safety, but it's a hell of a thing how much it lines up with the strategy. Now, of course, one could easily be cynical about Zuckerberg's open source turn and believe that he just shifted into this lane because this was the lane available for him. The question is practically whether the motivation actually matters or whether what matters is that the most singularly powerful person in one of the world's biggest companies has decided to walk down this path, whatever the reason. Now one thing that he would not commit to is for sure saying that if they reach AGI, they will open source it fully. He's hedging,

right? He says for as long as it makes sense in a safe and responsible thing to do, that I think we will generally want to lean towards open source. Obviously, you don't want to be locked into doing something because you said you would. So like I said, this is by far the most significant part of this announcement, the fact that meta is going after AGI and they want to do it in an open source way. But there is a lot more in here as well. One is the sheer scale of computing resources that Zuckerberg and Meta are going to bring to this goal. He told the verge that by the end of the year, Meta will own more than $340,000 of Nvidia's H-100 GPUs. Now, external research has suggested that meta's H-100 shipments for last year was at $150,000.

The only other company with shipments of that size is Microsoft, and it's at least three times larger than everyone else. The Verge writes, when it's Nvidia A100s and other AI chips are accounted for, meta will have a stockpile of almost 600,000 GPUs by the end of 2024. That's around $20 billion worth of compute. That's the scale that we're talking about when it comes to these large models, and that gets to another part that's important from this announcement, that meta is training Lama 3.

Now, in some ways, this isn't surprising at all. Of course, meta is training Lama 3. We've heard things from behind the scenes that suggest that as soon as Lama 2 came out, they were working on it. However, there seems to be a different emphasis now, said Zuckerberg. Lama 2 wasn't an industry leading model, but it was the best open source model. With Lama 3 and beyond, our ambition is to build things that are at the state of the art and eventually the leading models in the industry. Now, one of the things that I wonder in terms of that switch is how much it has to do with the rise of Mistral.

for basically all of 23, until Mistral launched, Meta's Lama was the default and leading open source model that people were building around. Its ecosystem was benefiting the most from the entire open source AI movement.

When Mistral came on the scene, they started to have some amount of competition. And Mistral was not content to just have the best open source model they were obviously going after a GPT4 level performance. Mistral has stated very clearly their goal to have a GPT4 level model

open source this year. So perhaps in some ways, this big announcement from meta reflects that type of competitive pressure. Now, the last part of his announcement has to do with the connection between AI and the metaverse. There are, of course, a million pundits and critics who have for as long as Zuckerberg has been talking about AI, saying that it was just some big pivot away from the metaverse after crypto crashed over the last couple years. And so to some extent, Zuckerberg is probably trying to put a pin in that narrative. But at the same time, I don't at all think that Zuckerberg

is just blowing a bunch of smoke up our butts when he talks about his vision of humans interacting with AIs. I think that he has a lot of him. in the team at Meta genuinely believe that a big part of human social interaction in the future is not going to be with just other humans, but with synthetic humans in the form of artificial intelligence. There's certainly some early evidence that that's the case, given how much particularly young people are using things like character AI, as well as the traction of virtual boyfriends and girlfriends. And so, on the one hand, Zuckerberg is shoring up a narrative, but on the other hand, I do think he's revealing part of the company's vision for the future. Now, the company

reorganization bringing the AI research team fair to sit inside the product organization and put it all under a shared leadership, sort of mirrors what Google went through last year bringing together their disparate AI teams as well. It also is representative of the incredible talent wars that are going on as companies compete for a very limited set of engineering resources. Chief product officer Chris Cox wrote in an internal note to employees, with this change, we elevate the importance of AI research as an essential ingredient to the long-term success of the company and our products. Alongside the major infrastructure investments, Mark mentioned today, moving fair and Gen. AI closer together will mean a more coherent AI research portfolio and roadmap, with Lama becoming

the primary launch vehicle for progress towards AGI, plus a streamlined development process for new technologies or novel AI research that navigate the legal policy and brand landscape consistently in an increasingly scrutinized space. TLDR on this reorg then is not that just two teams are coming together, but that meta is putting AI at the very center of the product. That, I think, is the big shift. Now, when it comes to takes on this, there are some folks who are absolutely incredulous. Jeffrey Laddish wrote, Mark Zuckerberg just announced that meta is going full speed developing open source AGI. This is insane. You can reasonably debate how far you can push open weight models before the risks

outweigh the benefits. I don't know where that line is, but AGI is clearly too far. One reason AGI is too far is that many AI researchers think we won't be able to main control of AI with human-level scientific R&D abilities. But even if you don't buy this, open source AGI will present enormous weaponization potential, e.g., everyone could make pandemic viruses. We've made very little progress on making models resistant to malicious fine-tuning. For the foreseeable future, if you release model weights, you give people access to the full weaponization potential of that model. Maybe let's solve that problem before trying to build open AGI.

Jeffrey Miller writes something similar. This is insane, he says, open source AGI is available to everyone? Hundreds of millions of Islamists want deaths to the West, a global caliphate, and Sharia law. Why would we want them to have all the access to human-level AI to help them plan terrorism? Others are very excited. Rengi, the Synthetic Data Maximilist, writes, It's so, so cool that Zuck wants to open source AGI, one of the coolest developments in the AI space since ChatGPT.

Aravin Shrinivas, the CEO of Perplexity, said, Open Source AGI is an amazing vision. You're building a very powerful technology and actually aligning to what makes sense for the world. More people have a say in what makes sense and doesn't. Zuck and Jan Lacoon are leading the revolution, and it's going to have an incredible short-term impact for the AI ecosystem as a whole. Meta gets a lot of shit, but there's literally no other company doing this today. Bindu Ready writes,

Open Source may win the AGI race. Just a year ago, a statement, like this would have sounded ridiculous. In fact, folks like Ilya from OpenAI have said that open source winning the AGI race would be akin to someone in a small garage winning the space race. Now a year later, open source is in the second position and it may take the lead by the end of the year. Here's how we get there. Open source techniques can improve a base LLM by 5 to 10% today. Every day, we are moving that needle, so by the end of the year we could move that up to 15 to 20%. Meta just announced they are going to open source Lama 3.

Mistral has announced they are going to open source a GPT4 class model. So assuming both meta and Mistrel open source their models and the rest of us have invented new methods to mix, match, and improve them, we may have a shot at catching up to GBT5. The race will get intense post-GPT-5 release, and open source has a very strong chance of winning. There are thousands of talented researchers and startups who are working on this 24-7. I don't know if OpenAI will be able to keep up. Jan Pelag put an image of the Mark Zuckerberg reel next to Times magazine cover The End of Humanity. How real is the risk?

He wrote, these images are six months apart. Imagine saying open source AGII six months ago. Nine months ago, Yudkowski calling for bombardment of data. data centers, six months ago, the end of humanity on the cover of Time magazine. Today, Zuckerberg saying in public open source AGI, public opinion shifts fast. Now, I'm not so sure that public opinion has really shifted. In fact, what I think is that as public opinion has gotten louder, the folks on the open source techno-optimist accelerationist side of this equation have made the determination that they

also need to get louder. Ultimately, I think who wins that debate will have less to do with who can shout the loudest and more to do with how benefits show up in real people's lives. But that is a topic for another show. For now, that's going to do it for today's AI breakdown. Appreciate you listening or watching as always. And until next time, peace.