Moonshots with Peter Diamandis - Who Will Govern the Future of AGI? w/ Emad Mostaque (Stability AI Founder) | X Spaces

Episode Date: November 22, 2023

Amidst all the recent Tech news, Peter and Emad hopped on Twitter (X) Spaces to discuss AGI governance, the future of AI, open-source models, and more.  Emad Mostaque is the CEO and Founder of Stability AI, a company funding the development of open-source music- and image-generating systems such as Dance Diffusion and Stable Diffusion. Get started on Stability AI Click here to sign up for the launch of the $101M XPRIZE – the largest in history.  ____________ I send weekly emails with the latest insights and trends on today’s and tomorrow’s exponential technologies. Stay ahead of the curve, and sign up now:  Tech Blog My new book with Salim Ismail, Exponential Organizations 2.0: The New Playbook for 10x Growth and Impact, is now available on Amazon: https://bit.ly/3P3j54J Learn more about my executive summit, Abundance360 _____________ Connect With Peter: Twitter Instagram Youtube Moonshots Learn more about your ad choices. Visit megaphone.fm/adchoices

Imagine being the first person to ever send a payment over the internet. New things can be scary, and crypto is no different. It's new, but like the internet, it's also revolutionary. Making your first crypto trade feels easy with 24-7 support when you need it. Go to Kraken.com and see what crypto can be. Not investment advice. Crypto trading involves risk of loss. See Kraken.com slash legal slash ca dash pru dash disclaimer

for info on Kraken's undertaking to register in Canada. That's the sound of unaged whiskey transforming into Jack Daniel's Tennessee whiskey in Lynchburg, Tennessee. Around 1860, Nearest Green taught Jack Daniel how to filter whiskey through charcoal for a smoother taste, one drop at a time. This is one of many sounds in Tennessee with a story to tell.

To hear them in person, plan your trip at tnvacation.com. Tennessee sounds perfect. Hey, Amad, good to hear you. It was always a pleasure. Yeah. So where are you today? I'm in London. Good. Other side of the planet.

I'm in Santa Monica. Monica. It's been quite the extraordinary game of ping pong out there these last four or five days. I didn't think the first thing that AI would disrupt would be reality TV, right?

Yeah. It's been fascinating how X has become sort of the go-to place to find out the latest of where SAM is working and what's going on with the AI industry. You found the notifications and the way it goes.

I think that's the thing. What else will move at the speed of this? I was saying something recently. AI research doesn't really move at the speed of conferences or even PDFs anymore, right? You just wake up and you're like, oh, it's 10 times faster. So I think that's why X is quite good. I actually unfollow just about everyone.

I just let the AI algorithms find the most interesting things for me. So I've got like 10 people that I follow, and it's actually working really well. It's getting better. Well, it has been. I've been enjoying the conversation. It really feels like you're inside an intimate conversation among friends

as this is going back and forth. I think this entire four or five days has been an extraordinary, up-close, intimate conversation around governance and around what's the future of AI? Because honestly, you know, as it gets faster and more powerful, the cost of missteps is going to increase exponentially. Let's begin here. I mean, you've been making the argument about open source as one of the most critical elements of governance for a while now.

Let's hop into that. Yeah, I think that open source is a difficult one because it means a few different things. Like, is it models you can download and use? Do you make all the data available and free? And then when you actually look at what all these big companies do, all their stuff is built on open source basis. It's built on the transformer paper.

It's built on the new model by Kai-Fu Lee. And Vero1.ai is basically Lama. It's actually got the same variable names and other things like that. Plus a gigantic supercomputer right the whole conversation has been you know how important is openness and transparency and and what are the governance models that are going to allow the most powerful technology on the planet uh to uh enable the most benefit for humanity and the safety. So, I mean, you've been thinking about this and speaking to transparency, openness, governance for a while. And could you, I mean, what do you think is going to be,

what do you think we need to be focused on? Where do we need to evolve to? It's a complicated topic. I think that most of the infrastructure of the internet is open source Linux, everything like that I think these models it's unlikely that our governments

will be run on GPT-7 or BARD or anything like that how are you going to have black boxes that run these things I think a lot of the governance debate has been hijacked by the AI safety debate, where people are talking about AGI killing us all, and then there's this precautionary principle

that kicks in. It's too dangerous to let out, because what if China gets it? What if someone builds an AGI that kills us all? It'd be great to have this amazing board that could pull the off switch, you know? Whereas in reality, I think that you're seeing a real social impact from this technology

and it's about who advances forward and who's left behind if we're thinking about risk because governance is always about finding as you said the best outcomes and also mitigating against the harms right and there's some very real amazingly positive outcomes that are now emerging that people can agree on but also some very real social impacts that we have to mitigate against so i mean let's begin how is how is stability governed uh stability is basically governed by me and um so i looked in foundations and DAOs and everything like that, and I thought to take it to where we are now, it needed to have very singular governance.

But now we're looking at other alternatives. And where do you think it's going to be? Where would you head in the future? I mean, let's actually jump away from this in particular. What do you recommend the most powerful technologies on the planet? How should they be governed? How should they be governed? How should they be owned? Where should we be in five years?

I think they need to be public goods that are collectively owned and then individually owned as well. So, for example, there was the tweet kind of storm, the kind of I am Spartacus, or his name is Robert Wilson in the OpenAI team, saying OpenAI is nothing without its people. Stability, we have amazing people, 190 and 65 top researchers. Without its people, we're open models used by hundreds of millions. It continues.

And if you think about where you need to go, you can never have a choke point on this technology, I think, if it becomes part of your life. The phrase I have is, not your models, not your mind. These models, again, are just such interesting things. Take

billions of images or trillions of words and you get this file out that can do magic, right? Trailer magic sand. I think that you will have pilots that gather our global knowledge on various modalities, and you'll have co-pilots that you individually own that guide you through life. And I can't see how that can be controlled

by any one organization. You've been on record talking about having models owned by the citizens of nations. Can you speak to that a little bit? Sure. So we just released some of the top Japanese models from visual language to language to Japanese SDXL as an example. So we're training for half a dozen different nations and models now.

And the plan is to figure out a way to give ownership of these data sets and models back to And the plan is to figure out a way to give ownership of these data sets and models back to the people of that nation. So you get the smartest people in Mexico to run a Stability Mexico, or maybe a different structure that then makes decisions for Mexicans with the Mexicans about the data and what goes in it. Because everyone's been focusing on the outputs, the inputs actually are the things that matter the most. The best way I've thought about thinking of these models is like very enthusiastic graduates,

so hallucinations isn't just probably too hard. A lot of the things about like, oh, what about these bad things the models can output? It's about what you input. And so what you put into that Mexican data set or the Chinese or Vietnamese one will impact the outputs. And there's a great paper in Nature Human Behavior today about that, about how foundational models are cultural technologies. So again, how can you outsource your culture and your brains to other countries, to people that are from a different place?

I think it eventually has to be localized. I think one of the points you said originally is we have to separate the issue of governance versus safety and alignment. Are they actually different?

I think that a lot of the safety discussion or this AGI risk discussion is because the future is so uncertain because it is so powerful. Right. And we didn't have a good view of where we're going. So when you go on a journey and you don't know where you're going, you'll minimize your maximum regret. You'll have the precautionary principle. And then that means you basically go towards authority you

go towards trying to control this technology when it's so difficult to control and you end up not doing much you know because everything could go wrong um when you have an idea of where we're going like you should have all the cancer knowledge in the world at your fingertips or climate knowledge or anybody should be able to create whole worlds and share them then you align your safety discussions against the goal against the location that you're going to again just like setting out on a journey i think that's a big change similarly most of the safety discussion has been on outputs not inputs if you have a high quality data set without knowledge about anthrax your language model is unlikely to tell you how to build anthrax.

So I think that's it. And transparency around that will be very useful. So let's dive into that safety alignment issue for a moment because it's an area you and I have been talking a lot about. So Mustafa wrote a book, Mustafa Suleiman wrote a book called The Coming Wave in which he talks about containment as the mechanism by which we're going to be making sure we have safe AI.

You and I have had the conversation of it's really how you educate and raise and train your AI systems in giving, making sure that there's full transparency and openness on the data sets that are utilized. Do you think containment is an option for safety? No, not at all. A number of leaders say, what if China gets open source AI? The reality is that China, Russia, everyone already has the weights for GPT-4 because they just downloaded it on a USB stick. You know? You know that it's been compromised, right?

There's no way they couldn't. The rewards are too great. And there is an absolutely false dichotomy here. And a lot of the companies want you to believe that giant models are the main thing and you need to have these gigantic ridiculous supercomputers that only they can run i mean like we run gigantic supercomputers but the reality is this the supercomputers and the giant trillion zillion data sets are just a shortcut for bad quality data

it's like using a hot pot or sous-viding a steak that's bad quality. You cook it for longer and it organizes the information. With stable diffusion, we did a study, and we showed that basically 92% of the data isn't used 99% of the time. Because now you're seeing this with, for example, Microsoft's PHY release. It's trained entirely on synthetic data. DALI 3 is trained on RVAE and entirely synthetic data.

You are what you eat. And again, we cooked it for longer to get past that. But the implications of this are that I believe within 12 to 18 months, you'll see GPT-4 level performance on a smartphone.

How do you contain that? And how do you contain it when China can do distributed training at scale and release open source models? So Google recently did a 50,000 TPU training run on their V5Es. The new V5Es, their TPUs, are very low-powered relative to what we've seen. But again, you can do distributed dynamic training. Similarly, like we funded 5Mind, and we've seen Google DeepMind just did a new paper on localization

through distributed training. The models are good, fast enough, and cheap enough that you can swarm them, and you don't need giant supercomputers anymore. And that has a lot of implications, and how are you going to contain that? So coming back to the question of do you mandate training sets, does the government set out what all companies should be utilizing and mandate if you're going to have an aligned AI, it has to be trained on on these sets how do we how do we possibly govern that look we have food standards right for ingredients

why don't you have data standards for the ingredients that make up a model it's just data compute and some algorithms right And so you should say there are the standards, and then you can make it compulsory. That will take a while. Or you can just have an ISO-type standard. This is good-quality model training, good-quality data. And people will naturally gravitate towards that, and it becomes the default.

Are you working towards that right now? Yeah. I mean, look, we spun out Al Eleuther AI as an independent 501c3 so they could look at data standards and things like that independently of us. Kind of the opposite of OpenAI. And this is something I've been talking to many people about, and we're getting national

data sets and more, so that hopefully we can implement good standards. Similar to how we offered OptOut and had a billion images opted out of our image data set, because everyone was just training on everything. Is it required? No. But is it good? Yes. And everyone will benefit from better quality data. So there's no reason that for these very large model training runs,

the data sets should not be transparent and logged. Again, we want to know what goes into that. Again, if we have the graduate analogy, what was the curriculum that the graduate was taught at? Which university did they go to? It's something that we'd want to know. But then why do we talk to GPT-4 when we don't know where it went to university or where it's been trained on?

It's a bit weird, that, isn't it? What do you think the lesson's going to be from the last four days? I'm just confused. I don't know who was against who or what.? I'm just confused. I don't know who was against who or what. I think I just posted. Were we against misalignment or Moloch? I think probably the biggest lesson is it's very hard to align humans, right?

And the stakes are very large. Like, why is this so interesting to us? Because the stakes are so high. You tweeted something that was, you know, serious and unfortunately funny, which was how can we align AI with humanity's best interests if we can't align our company's board with its employees' best interests? Well, the thing is it's not the employees' best interests.

It's like the board was set up as a lever to ensure the charter of OpenAI. So if you look at the original founding document of OpenAI from 2015, it is a beautiful document talking about open collaboration, everything. And then it kind of changed in 2019. But the charter still emphasizes cooperation, safety, and fundamentally, I posted about this back in March when I said the board and the government structure of OpenAI is weird

like what is it for what are they trying to do because if you say you're building AGI in their own road to AGI they say this will end democracy most likely there's no way democracy survives

AGI because either obviously it'll be better and you get it to do it or persuade everyone or we all die or it's utopia forever right abundance baby yeah there's no yeah well this thing there's but then regardless there's no way it survives agi there's no way capitalism survives agi the agGI will be the best trader in the world, right? And it's like, who should be making the decisions on the AGI, right? Assuming that they achieve those things. And that's in their own words. So I think that people are kind of waking up to, oh, there's no real way to do this properly.

And previously, we were scared of open and being transparent, everyone getting getting this which again was the original thing of open air and now we're scared of who are these clowns you know and put in the nicest way because this was ridiculous like you see better politics in a teenage sorority right and it's fundamentally scary that unelected people no matter how great they are and i think some of the new board members are great should have a say in something that could literally upend our entire society according to their own words i find that inherently anti-democratic and illiberal democratic and a liberal? At the end of the day, you know, capitalism has worked and it's the best system that we have thus far.

And it's a self, you know, it's, it's built on self interest and built on continuous optimization and maximization. I'm still wondering where you go in terms of governing these companies at one level, internal governance, and then governing the companies at a national and global level. Has anybody put forward a plan that you think is worth highlighting here? that you think is worth highlighting here?

Not really. I mean, organizations are a weird artificial intelligence, right? They have the status of people, and they're slow, dumb AI. And they eat our hopes and dreams. That's what they feed on, I think. But this AI can upgrade them. It can make them smarter. It can, how do you coordinate?

And from a mechanism design perspective, it's super interesting. Like in markets, I think we will have AI market makers that can tell stories. Like the story of Silicon Valley Bank went around the world in two seconds. The story of OpenAI goes around. AI can tell better stories than humans. It's inevitable. I think that gives hope for coordination, but then also it's dangers of disruption. I want to double-click one second on the two words that you use most, openness and transparency, and understand fully what those mean one moment.

The question is not only what they mean, but how fundamental they need to be. Openness right now in your definition in terms of AI means what? It means different things for different things, unfortunately. I don't think it means open source. I think for me, open means more about access and ownership of the models so that you don't have lockstep. You can hire your own graduates as opposed to relying on consultants. Transparency comes down to, i think for language models in particular i don't think this holds for media models you really need to know what it's been taught that's the only way

to safety like you should not engage with something or use something if you don't know what its credentials are and how it's been taught because i think that's inherently dangerous as these get more and more capabilities. And again, I don't know if we get to SGI. If we do, I think it'll probably be like Scarlett Johansson and her, just to give fine thanks to the GDs, but assuming we don't, you still need transparency. Again, how can any government or regulated industry not run on a transparent model? They can't run on black boxes.

I get that, and I understand the rationale for it, but now the question is, can you prove transparency? I think that, again, a model is only three things, really. It's the data, the algorithm, and the compute. And then they come and the binary file pops out. Then you can tune it with RLHF or DPO or genetic algorithms or whatever. But that's really the recipe,

right? And so the algorithms, you don't need algorithmic transparency here versus classical AI because they're very simple. One of our fellows recreated the Palm 540 billion parameter model. This is Lucid Rains on GitHub. You look at that if you're a developer and you want to cry. It's GitHub.

It's crazy. In 206 lines of PyTorch. And that's it. The algorithms are not very complicated. Running a gigantic supercomputer is complicated. And this is why they freaked out when Greg Brockman kind of stepped down, because he's one of the most talented engineers of our time.

Built these amazing, gigantic clusters. And then the data and how you structure data is complicated. So I think you can have transparency there. Because if the data is transparent and who cares about the supercomputer, who really cares about the algorithm? Now let's talk about the next term, alignment, here.

Alignment's thrown around in lots of different ways. How do you define alignment? I define alignment in terms of objective function. So YouTube was used by the extremists to serve ads for their nastiness, right? Why? Because the algorithm optimized for engagement, which then optimized for extreme content which

then optimized for the extremists did youtube mean that no but they're just trying to sell ads up right but it meant it wasn't aligned with its users interests and so for me if you have these technologies that we're going to outsource more of our mind our culture our children's futures to you that are very persuasive. We have to ensure they're aligned with our individual community and societal best interests. And I think this is where the tension with corporations will come in. Because whoever licenses Scarlett Johansson's voice will sell a lot of ads, you know? They can be very, very persuasive. But then what are the controls on that no one talks about that the bigger question of alignment is not killerism making sure the ai

doesn't kill us but again i feel that if we build ai that is transparent that we can test that people can build mitigations around we are more likely to survive and thrive. And also I think there's a final element to here, which is who's alignment. Yes. Different cultures are different. Different people are different. What we found with stable diffusion is that

when we merge together the models that different people around the world have built, the model gets so much better. I think that makes sense because a monoculture will always be less fragile than a diversity. the model gets so much better. I think that makes sense because a monocult show will always be less fragile than a diversity. Again, I'm not talking about in the DEI kind of way. I'm talking about it in the actual logical way.

So we have a paper from our reinforcement learning lab, called CARPA, called QDAIF, Quality and Diversity Through Artificial Intelligence Feedback, because you find these models do get better with high quality and diverse inputs. Just like you will get better if you have high quality and diverse experiences. You know? And I think that's something that's important that will get lost if all these models are centralized. You know, you and I have had a lot of conversations about timelines here.

We can get into a conversation of when and if we see AGI, but we're seeing more and more powerful capabilities coming online right now that are going to cause a lot of amazing progress and disruption. How much time do we have Imad and we've had, we had a conversation when we were together at FII about, um, uh, the disenfranchised youth coming off COVID. Uh, so let's talk one second about timelines of how long do we have to get our shit together? Um, both um both as as uh ai ai companies and uh and investors and governors of society um we don't i mean the speed here is is awesome um and frightening How long do we have?

Everything, everywhere, all at once, right? We don't have long. Like AGI timelines for every definition of AGI, I have no idea. It will never be less than 12 months, right? Because it's such a step change. So let's put that to the side. Okay.

Right now, everyone that's listening, are you all going to hire the same amount of graduates that you hired before? The answer is no. Some people might not hire any because this is a productivity answer and we have the data for that across any type of knowledge industry. You just had a great app that you can sketch

and it does a whole iPhone app for you, right? I've gone on record and saying there are no programmers we didn't know in five years. Why? Where would there be? What are interfaces? You had a 50% drop, I just put that on my Twitter, in hiring from Indian IITs. That's crazy.

So what you're going to have in a couple of years is around the world at the same time, these kids that have gone through the trauma of covid highly educated stem programming accountancy law simultaneously people will hire massively less of them because productivity enhances and you don't need as many of them why would you need as many paralegals and that for, is a gigantic societal issue. And the only thing I can think of is to stoke open innovation and the generative jobs of the future through open source technology.

Because I don't know how else we're going to mitigate that. Because, you know, Peter, you're a student of history. What happens when you have large amounts of intelligent, disenfranchised youth? We've had that happen a few times. We just had Arab Spring not long ago. Revolt. Civil war, if not international law.

War is a good way to soak up the excess youth. Yep. But it's not pleasant. It's not pleasant for society. And fundamentally, the cost of information-gather gathering organization has collapsed. Like, again, you look at stable video that we released yesterday, right? It's going to get so much better so quickly, just like stable diffusion. The cost of creating movies increases. The demand

for quality stuff increases. But there's a few years where demand and supply don't match. And that's such a turbulent thing to navigate. That's one of the reasons I'm creating Stabilities for Different Countries, so the best and brightest from each can help navigate. And people don't talk about this. I loved your idea that the stability models and systems will be owned by the nation. In fact, one idea that I heard you say, which I thought was fantastic,

was you graduate college in India, you're an owner in that system. You graduate from in Nigeria, you're an owner in that system. Basically to incentivize people to complete their education and to have them have ownership in what is ultimately the most important asset that nation has. And talk about it as infrastructure as well. I think that's an important analogy that people don't get. This is the knowledge infrastructure of the future. It's the biggest leap forward we have because you'll always have a co-pilot that knows everything in a few years

and that can create anything in any type. But it must be embedded to your cultural values and you can't let anyone else own that. So it is the infrastructure of the mind. And who would outsource the infrastructure to someone else? So that's why I think Nigerians should own the models of Nigerians for Nigerians. And it should be the next generation that does that. That's why you give the equity to the graduates.

That's why you list it. That's why you make national champions. Because, again, that has to be that way this is far more important than 5g and this gives you an idea of the scale we're just at the start the early adopter phase a trillion dollars was spent on 5g this is clearly more important more than a trillion dollars will be spent on this and again it flips the world and so there is huge threat for our societal balance. And again, I think open is a potential antidote to create the jobs of the future.

And there's huge opportunity on the side because no one will ever be alone. And we can use this to coordinate our systems, give everyone all the knowledge that they need at their fingertips, and help guide everyone if we build this infrastructure correctly. And again, I don't see the highlight can be closed agi um you know the conversation and the definition of agi has basically uh been all over the place uh ray kurzweil's prediction has been for 30 years that it's 2029 again that's a blurry line of what we're trying to target, but Elon's talked about anywhere

from 2025 to 2028. What are you thinking? What's your timeline for even digital superintelligence? I honestly have no idea. People are looking at the scaling laws

and applying it but as I've said data is the key and it's clear that we already have like you could build a board GPT and it'd be better than most corporate boards right so I think we're already seeing improvements over the existing one of the complications here is

swarm AI so even like it's the whole thing, like a duck-sized human or 100 human-sized ducks, right? We're just at the start of swarm intelligence and that reflects and represents how companies are organized. Andre Carpathy has some great analogies on this in terms of the new knowledge OS. And that could take off at any time,

but the function and format of that may not be this whole Western anti-compromised consciousness that we think of, but just incredibly efficient systems that displace existing human decision-making. And so there's an entire actual range of different AGI outcomes depending on your definition. And I just don't know. But I feel again like I wake up and I'm like,

oh, look, it's fed up 10 times the model. You know, like I'm just not, no one can predict this. But there is a point at which, I mean, we're heading towards an AI singularity, using the definition of a singularity as a point after which you cannot predict what's coming next. And that isn't far away. I mean, how far out is it for you?

A year, two years? I think you're heading towards it in the next few years. But like I said, every company, organization, individual has an objective function. My objective function is to allow the next generation to navigate what's coming in the optimal way and achieve their potential. So I don't want to build an AGI. I don't want to do any of this.

Amplified human intelligence is my preference. And trying to mitigate against some of the harms of these agentic things through data transparency, good standards, and making it so people don't need to build gigantic models on crap, which I think is a major danger, even if not for major. But again, we just don't understand because it's difficult for us to comprehend superhuman capabilities. But again, we're already seeing that in narrow fields.

We already know that it's a better rider than us. We already know that it can make better pictures than us. And a better physician and a better educator and a better surgeon and a better writer than us. I already know that it can make better pictures than us. And a better physician and a better educator and a better surgeon and a better everything. Yeah, and again, I think it's this mythos of these big labs being AGI-focused, whereas you can be better than us in like 5% of the stuff that humans can do,

and that's still a massive impact on the world, and it can still take over companies and things like that, right? Like if you. And that's still a massive impact on the world. And it can still take over companies and things like that, right? Like if you take over a company, then you can impact the world. And there's clearly with a GPT-4 or a thousand of them orchestrated correctly that can call up people and stuff.

You wouldn't know it's not the CEO. You know, I can make an MRGPT and then they won't have to make all these tough decisions. They're nearly there. And most of my decisions aren't that good. So it'd probably be better. So I think that we're getting to that point.

It's very difficult, and the design patterns are going fast. We're at the iPhone 2G, 3G stage. It's got copy and paste. And we've just got the first stage as well of this technology, which is the creation step. It creates stuff.

The next step is control and then composition where they're annoyed because chat GPT doesn't remember all the stuff that you've written. That won't be the case in a year. And the final bit is collaboration, where these AIs collaborate together and with humans to build the information superstructures of the future. And I don't feel

that's more than a few years away. And it's completely unpredictable what that will create. Let's talk about responsibility that AI companies have for making sure that their technology is used in a pro-human and not a disruptive fashion. Do you think that is a responsibility of a company, of a company's board, of a company's leadership? How do you think about that?

Again, with the corporate as capitalist system, it typically isn't because you're maximizing shareholder value and there aren't laws and regulations, which is why I think there's a moral, a social, and a legal slash regulatory aspect to this. Companies will just look at the legal slash regulatory. In some cases, they'll just ignore them, right? But I do think, again, we have a bigger moral and social obligation to this. This is why I don't subscribe to EA or EAC or any of these things. I think it's complicated and it's hard, given the uncertainty of how this technology proliferates. And you've got to do your best and be as straight as possible to people about doing your best.

Because none of us are qualified to understand or do this and none of us should be trusted to have the power over this technology right you should be questioned you should be challenged with that and again if you're not transparent how are you going to challenge when i think of the most linear organizations on the planet i think of governments maybe religions but governments let's leave it there. How can, you know, let's talk about Western government, at least the US, I would have said Europe, but I'll say the UK and Europe. What should they be, what steps should they be taking right now? You know, If you were given the reins to say, how would you regulate?

What would you want them to do or not do? I believe it's a complicated one. So I signed the first FLI letter. I think I was the only AI CEO to do that back before it was cool. Because I said, I don't think AGI will kill us all, but I just don't know. I think it's a conversation that deserves to be had, and it's a good way to have that conversation. back before it was cool. Because I said, I don't think AGI will kill us all, but I just don't know.

I think it's a conversation that deserves to be had, and it's a good way to have that conversation. And then we flipped the wrong way, where we went overly AI death risk and other things like that, and governments were doing that at the AI Safety Summit in the UK. And then we had the King of England come out and say, this is the biggest thing since fire.

I was like, okay, that's a big change. That's right. The King of England said it, so I must be on the right track. But I think if you look at it, regulation doesn't move fast enough. Even the executive order will take a long time. The EU things will kind of come in. Instead, I think that governments have to focus on the tangibles. AI killerism, again, it can be addressed by considering this as infrastructure

and what infrastructure we need to give our people to survive and thrive. The U.S. is in a good initial place with the CHIPS Act, but I think you need national data sets. You need to provide open models to stoke innovation and think about what the jobs of the future are because things are never the same again. You don't need all those programmers when co-pilot is so good and you're moving from co-pilot to the level above,

which is compositional co-pilot and then collaborative co-pilot. You'll be able to talk and computers can talk to computers better than humans can talk to computers. So we need to articulate the future on that side, but then the other side. One of the examples I give is a loved one had a recent misdiagnosis of pancreatic cancer. Vida, you and I talked about this. And the loss of agency you feel,

and many of you on this call will have had that diagnosis in India, is huge. And then I had 1,000 AI agents finding out every piece of information about pancreatic cancer. And then after that, I felt a bit more control. Why don't we have a global cancer model that gives you all the knowledge about cancer and helps you talk to your kids and connects you with other people like you,

not for diagnosis or research, but for humans? This is the Google MedPALM2 model, for example, that outperforms humans in diagnosis, but also empathy. And what if we armed our graduates to go out and give support to the humans that are being diagnosed in this way. That makes society better and it's valuable. And that's an example of a job of the future, I think. I don't believe in UBI. I think we believe in universal basic jobs as well. Yes, universal basic opportunity.

Universal basic opportunity, universal basic jobs. But then, Paul's making me think about it now because the graduate unemployment wave is literally a few years away. That is, I mean, when I think about what, I parse the challenges we're going to be facing in society into a few different elements. I think, you know, what we have today is amazing. And if generative AI froze here, we'd have an incredible set of tools to help humanity across all of its areas. And then we've got what's coming in the next, you know, zero to five years.

We've talked about patient zero perhaps being the U.S. elections and the, you know, I think you had said, you know, it was Cambridge Analytica that required interference. Now it's any kid in the garage that could play with the elections. That's a challenging period of time. And this graduate unemployment wave, as you mentioned, coming right on its heels. The question becomes, is the only thing that can create alignment and help us overcome this AGI at the highest level, meaning it is causing challenges, but ultimately is a tool that will allow us to enable to solve these challenges as well.

I mean, that's a crazy thought, right? All this stuff is crazy, the sheer scale and impact of it. And these discussions, we had them last year peter and now everyone's like yeah that makes sense like oh wow right it may be agi it may be these coordinating automated story makers and balances for the market right next year there's 56 elections with 4 billion people heading to the polls what could possibly go wrong okay possibly go wrong you know oh my god but again the technology isn't going to stop like even if stability puts down things if open ai puts down things

it will continue from around the world because you don't need much to train these models again this supercomputer thing is a myth you've got another year or two where you need them you don't need them after that and that is insane to think about. You just released Stability Video. Congratulations on our stable visual diffusion. Thank you. And I'm enjoying some of the clips. How far are we away from me telling a story to my kids

and saying, let's make that into a movie? One to two years away. What's two years away? Two years away. So this is a building block. It's the first creation step. And then, like I said, you have the control step, composition, and then collaboration. And self-learning systems around that.

So we have Comfy UI, which is our node-based system where you have all the logic that makes up an image, like you can take a dress and a pose and a face that combines them all, and it's all encoded in the image because you can move beyond files to intelligent workflows that you can collaborate with.

If I send you that image file and you put it into your Comfy UI, it gives you all the logic that made that up. How insane is that, right? So we're going to step up there and what's happened now is that people are looking at this ai like instant versus again the huge amount of effort it took to take this information structure it before but the value is actually in stuff that takes a bit longer like when you're shooting a movie you don don't just say, do it all in one shot, right? Unless you are a very talented director and actor, you know?

You have mise en place, you have staging, you have blocking, you have cinematography. It takes a while to composite the scenes together. It will be the same for this, but a large part of it will then be automated for creating the story that can resonate with you. And you can turn it into Korean or whatever. And there'll still be big blockbusters, like Oppenheimer and Barbie, but again, the floor will be raised overall. Similarly, we had a music video competition,

check it out on YouTube, with Peter Gabriel. He allowed us to use kindly his songs, and people from all around the world made amazing music videos to his thing, but they took weeks. So I think we're somewhere in the middle here, where, again, we're just at that early stage, because chat GPT isn't even a year old.

You know, stable diffusion is only 14, 15 months. And I think you'd agree that neither of them is the end all and be all. It's the earliest days of this field. I had the conversation with Rayiniest building yeah i had this conversation with ray kurzweil uh two weeks ago um we're just after a singularity board meeting we had we're just hanging on on a zoom and chatted and you know the realization is that unfortunately the human mind is awful at exponential projections and despite the convergence of all these technologies, we tend to project the future as a linear extrapolation

of the world we're living in right now. But the best I can say is that we're going to see in the next decade, between now and 2033, we're going to see a century worth of progress. But it's going to get very weird very fast isn't it um i mean there's there's two-way doors and there's one-way doors right like in december of last year multiple headmasters called me and said we can't set essays for our homework anymore and every headmaster in the world had to say it's that same thing it's a one-way door yes and this is the scary part the the one-way doors, right?

Like when you have an AI that can do your taxes, what does that mean for accountants? All the accountants at the same time. It's kind of crazy, right? It is. And the challenge, I mean, one of my biggest

concerns, so listen, I'm the eternal optimist. I'm not the guy whose glass is half full. It's the glass that's overflowing. And one of the challenges I think through when I think about where AI, AGI, ASI, however you want to project it to, is the innate importance of human purpose. And unfortunately, most of us derive our purpose from the work that we do. I ask you, tell me about yourself, and you jump into your work and what you do. And so when AI systems are able to do most everything we do, not just a little bit better, but orders of magnitude better,

thing we do not just a little bit better but you know orders of magnitude better um redefining purpose and redefining my role in achieving a moonshot or a transformation uh is it's the you know it's the impedance mismatch between uh human societal uh growth rates and tech growth rates and tech growth rates. What are your thoughts there? Yeah, I mean, I think, again, exponentials are hard. Like, if I say GPT-4 in 12 to 18 months on a smartphone, you'd be like,

well, that's not possible. Why? You know, like, GPT-4 isn't possible. Stable diffusion is impossible, right? Like, now they've almost become commonplace, but why would you need supercomputers in these things? I do agree there's this mismatch, and that's why we're in for five years of chaos. That's why I called it stability. I saw this coming a few years ago, and I was like, holy crap, we have to build this company. And now we have the most downloads of any models of any company,

like 50 million last month versus 700,000 from Estral, for example. And we will have the best model of every type, except for very large language models by the end of the year. So we have audio, 3D, video, code, everything. And a lovely, amazing community. Because it's just so hard, again, for us to imagine this mismatch. There's a period of chaos.

But then on the other side, there's this PDoom question, right? The probability of doom. I'm going to say something. With this technology, the probability of doom is lower than without this technology. Because we're killing ourselves. And this can be used to enhance every human

and coordinate us all. And I think what we're aiming for is that Star Trek future versus that Star Wars future, right? Yes, I'm into that. And I think that's an important point, that the level of complexity that we have in society,

we don't need AI to destroy the planet. We're doing that very well, thank you. But the ability to coordinate. So one of the things I think about is a world in which everyone has access to all the food, water, energy, healthcare, education that they want. Really, a world of true abundance, in my mind, is a more peaceful world, right? Why would you want to destroy things if you have access to everything that you need? And that kind of a world of abundance is on the backside of this kind of awesome technology. We have to navigate the next period.

I believe we'll see it within our lifetimes, particularly if we get longevity songs, right? And that's so amazing, right? But then we think about, as you said, why peace? A child in Israel is the same as a child in Gaza. And then something happens. A lie is told that you are not like others. And the other person is not human like you. All wars are based on that same lie.

And so, again, if we have AI that is aligned with the potential of each human that can help mitigate those lies, then we can get away from war because the world is not scarce. There is enough food for everyone. It's a coordination failure. And that can be addressed by this technology. I agree. One of the most interesting

and basic functions or capabilities of generative AI has been the ability to translate my ideas into concepts that someone who has a different frame of thought can understand. But that's what this generative AI is. It's a universal translator.

Yes, for sure. It does not have facts. The fact that it knows anything isative AI is. It's a universal translator. Yes, for sure. It does not have facts. The fact that it knows anything is insane. Hallucinations is a crazy thing to say. Again, it's just like a graduate trying so hard. GPT-4 with 10 trillion words and 100 gigabytes is insane. Stable diffusion has like 100,000 gigabytes in a two gigabyte file.

50,000 to one compression is something else. It's learned principles. Yes. And this is the... It's knowledge versus data. It's knowledge versus data. And you apply some experience, you get the wisdom, right? Because it's learned the principles and contexts,

and it can map them to transform data. Because that's how you navigate you don't navigate based on like logical flow we have those two parts of our brain navigate sometimes based on instinct based on the principles you've learned so tesla's new auto driving model self-driving model is entirely based on i can't say which architecture and if they said it publicly it's based on this technology it doesn't have any rules. It's just learned the principles of how to drive from massive amounts of Tesla data

that now fits on the hardware without internet. And so they went from self-driving being impossible to now, hey, it works pretty well, you know, because it's learned the principles. And so that's why this technology can help solve the problem. This is why it can help us amplify our intelligence and our innovation. Because it's the missing part, the second part of the brain. You know, next, I can't give more details yet,

but next week we're announcing the largest XPRIZE ever. It's $101 million. It's 101. So Elon had the $100 million prize that got him to fund a few years ago for carbon sequestration, and the first funder of this prize wanted it to be larger than Elon's. I said, okay, you add the extra million. It's for luck. It's for luck.

We did our seed rounds at 101 million. Oh, really? Okay. That's great. That's a new popular number. Anyway, and it's in the field of health. I'll leave it at that. Folks can go to XPRIZE.org to register to see the live event on November 29th.

We're going to be debuting the prize, what it is. It's going to impact 8 billion people. Long story short, it's a nonlinear future because we are able to utilize AI and make things that were seemingly crazy before likely to become inevitable. It's an amazing future we have to live into. Yeah, I mean, again, because it's one-way doors the moment we create a cancer gpt this is something that we're building we have trillions of tokens and the new google tpus and things like that that organizes global cancer knowledge and makes it accessible and useful even if it's just for guiding people that have been diagnosed the world changes the 50 percent of people that have a

cancer diagnosed in their lives in every language and every level, will have someone to talk to and connect them with the resources they need and other people like them and talk to their families, you know? And how insane is that? And so again, these positive stories of the future need to be told, right? Because that will align us to where we need to go as opposed to a future full of uncertainty and craziness and doom. In our last couple of minutes here, buddy, what can we look forward to from stability in the months and years ahead?

We have every model of every type and we'll build it for every nation and we'll give back control to every nation. So coming back to governance here, again, is the nation state the unit of control? Is it? No. My thinking is the disabilities of every nation should have the best and brightest of each.

Because what you've seen is there are amazing people in this sphere. The best and brightest in the world know this is the biggest thing ever and they all want to work in it and it's just finding the right people with the right intention the brightest people go back to singapore or malaysia or others because of the future of their nations and again then now we're doing a big change and we don't talk about all the cool stuff we do we're just taking it because you need to articulate a positive vision of the future because the only scarce resource is actually We've just taken it because you need to articulate that positive vision of the future. Because the only scarce resource is actually this, it's human capital. It's not GPUs. It's not data.

It's about the humans that can see this technology and realize that they can play a part in guiding it for the good of everyone, their own societies and more. And that's, again, what I hope stability can do. Well, I wish you the best of luck, pal. Thank you for joining me in this conversation.

It's been a crazy four or five days and I wish Sam and Greg and the entire OpenAI team stability in their lives. Yeah, I hope they have a nice Thanksgiving. They're an amazing team building world-changing technology. There's such a concentration of talent. I think, again, I really felt

for them over the last few days as much as I post memes and everything. I posted that as well. I think this will bring them closer together and hopefully they can solve the number one problem that I've asked them to solve, which is email. Sol is email. Solve email.

And then we'll crack on from there. All right, cheers, my friend.