TRIGGERnometry - Will AI Destroy The World? - Igor Kurganov

Episode Date: January 7, 2024

Igor Kurganov is a Russian-born German professional poker player who’s won more than $18 million. He is the husband of former guest Liv Boeree and together they co-founded the effective altruism non-profit 'Raising for Effective Giving'. From 2021-2022 he was a key figure in Elon Musk's inner circle when Musk asked him to facilitate the giving away of $5.7 billion in Tesla shares to charity. Igor and Liv’s organisation: https://reg-charity.org/ We are proud partners with GiveSendGo - a world-leading crowdfunding platform that believes in free speech. Go to givesendgo.com and raise money for anything important to you. Join our exclusive TRIGGERnometry community on Locals! https://triggernometry.locals.com/ OR Support TRIGGERnometry Here: Bitcoin: bc1qm6vvhduc6s3rvy8u76sllmrfpynfv94qw8p8d5 Music by: Music by: Xentric | [e-mail] | https://www.xentricapc.com/ YouTube: @xentricapc Buy Merch Here: https://www.triggerpod.co.uk/shop/ Advertise on TRIGGERnometry: [e-mail] Join the Mailing List: https://www.triggerpod.co.uk/sign-up/ Find TRIGGERnometry on Social Media: https://twitter.com/triggerpod https://www.facebook.com/triggerpod/ https://www.instagram.com/triggerpod/ About TRIGGERnometry: Stand-up comedians Konstantin Kisin (@konstantinkisin) and Francis Foster (@francisjfoster) make sense of politics, economics, free speech, AI, drug policy and WW3 with the help of presidential advisors, renowned economists, award-winning journalists, controversial writers, leading scientists and notorious comedians. Learn more about your ad choices. Visit megaphone.fm/adchoices

It's trained at deception. It's made to kind of not discover truth, but really like satisfy the user presently. It is a bit. If we just go with as fast as possible, we won't take, like sometimes ideally you would take the trade-off towards going a little slower, making sure that it's not deceiving us. People will talk about a tyranny of the minority in various political and cultural contexts where a small group of people get to impose their worldview and everybody else. That seems to me like that on steroids. that on steroids. Hey guys, trigonometry needs your help. We took a big risk creating the show, and for us to keep doing the incredible work

that you all love, we need your support. That's the only way we're going to stay independent and create content that you won't be able to find anywhere else. There is no other podcast where you'll hear interviews with Nigel Farage one week and the next week you've got Aaron Bastani, the founder of Left Wing Show in Navarra Media, on the same platform. the mainstream media aren't honest. You know they've been caught lying again and again. You know they can't be trusted. The only way to change that is to make a stand and support

independent content creators, like trigonometry, to produce better and more honest content. We have big plans and we'll shortly be announcing exciting new shows and more terrific interviews with huge guests. That isn't going to happen without your help. When you support us, you also get incredible extra content. such as extended interviews with none of those irritating adverts, and they'll be released 24 hours early just for you. We'll have exclusive bonus interviews that only you get to hear. Click the link on the podcast description

or find the link on your podcast listening app to join us. Support us and help change the way we have conversations and make the world saner. One of the things we really wanted to talk to you about is existential risk, risk and a lot of the technological transformations that are happening now, not only I, but also in biology and things like that. What's going on and what should people know? Yeah, I mean, a lot is going on right now. And so if we start with, say, biology, like we've now had a pandemic that was potentially caused by gain of function research within the Wuhan lab. Potentially it was just a spillover. I think the status that we've now had a pandemic that was potentially

will never truly know, but the probability among experts that it was actually caused by a gate of function research and then a lab leak seems to be pretty high. Stop being racist, but carry on. Was suppressed for a while. But yeah, so that opens up then the question as well of like where do we assume that future risk around future pandemics is coming from? And I think that in like discussing the risk, people were just simply using the wrong priors in many occasions and that's why the WHO is kind of shooting in the wrong direction.

Often you will hear epidemiologists or biologists say that, well, historically, like it's mostly been natural spillovers that have led to humans getting infected with this or that pathogen. And it's true, but technology has changed where we now are much more able to engineer pathogens to have like higher fatality rate, or to easier transmit from one mammal to another, or between from one human to another. So in the face of that change, you would expect that there are many more new,

very bad pathogens that are possible to exist, and those are also the ones that I think pose a much greater risk. Basically, if you look at natural spillovers, then like, yeah, what's really going to happen? It's like we know most of the viruses there, they're not like all of a sudden going to mutate to be like a hundred X as deadly and like 10 times as transmissible. It just doesn't happen from like natural mutation.

You would expect like slight changes or like strong changes on one of these factors. Whereas with engineered pathogens, you can actually just make one that is like this. And that was, for example, happened with, I think it was at the University of Wisconsin where they took H5N1, bird flu, and it's like, hey, how about we engineer it to be transmissible to humans? and then look into that virus to understand how we can defeat it. The problem though is labs are leaky. Sometimes you can try to protect it and it's like 99.99% safe, but if you have a lot of that research happening in a lot of places,

then that one in 10,000 or whatever the rate is really matters. And if you look at it historically yet again, then like these spillovers just happen all the time. Can we just pause there? There's so much to unpack and for you to carry on with. I just want to pick on that particular one because you and I were both, I mean, I don't know if you left the Soviet Union or Ukraine, or Russia, is what I mean. We left in 92, so right after the breakdown of the Soviet Union. So you would have lived for some time in the Soviet Union, right? Chernobyl killed a few thousand people.

Depends how you can. It depends how you can. But let's say 10,000 people for the sake of argument. And that was a man-made disaster. And that was, there's been movies about it. It's been the craziest thing that's ever happened. People are terrified of nuclear energy since. The pandemic, the last pandemic, killed however many millions of people around the world,

probably did come from a lab. We don't know for sure, but probably did due to gain a function of research. And it's like, everyone's forgotten about it. Is that not incredible? It's pretty nuts. So I was fundraising for pandemic prevention research prior to COVID as part of the work that we were doing in philanthropy and had hope that, well, this was terrible, but at least we'll be now like, yeah, it's not going to be a neglected area anymore. People will understand. Pandemics are dangerous and real. And we'll like worst case fight the last war and overfocus on COVID or something. But we're not even fighting the last war right now.

It's like it's really weird how much damage was created and how little people are now doing a prudent risk benefit calculus to address future damages. We had like over 10 trillion in economic damage by some estimates from this. And like the as part of the infrastructure bill in like 21, they allocated first like 60 billion to pandemic prevention. It was good stuff. And then it got cut down and cut down further to like 2 billion or so, which is lower than. than two and a half billion the tree equity received. Like, trees are people too, and they deserve rights. No, it makes sense.

Like, I want to have every neighborhood to have trees if we can, but maybe pandemics are also pretty bad and we need to do stuff. Anyway, my kind of assumption is, like my, from having been close to it, what I feel happened is that we got fortunately very lucky with the vaccine working, at least like helping, and that MRNA like had this like break through that allowed for it. And because the government and many other places didn't know how to solve COVID and kind of like screwed up in so many ways, they just over-indexed super hard on vaccines. Like vaccines should be part of the stack of solutions you implore against pandemics. You want like

early detection. You want containment. You want like countermeasures. And we're now focused only on this. So it's kind of like, hey, this fell into our lab. So let's pretend like that's, like that's the solution. I think in part that's why it's like they were like, well, every ill that still happens from COVID is because of insufficient vaccine uptake. It's like because it's the only tool that they found, but there are many more tools. And why is that? Why are we not actually learning the lessons from the last couple of years? Well, the question is if we're designed to learn the lessons. Like if we look at the place which you would want to learn the lessons, integrate it and then like work on better solutions, then the question is like, well, is that

institution really set up to do all of that or are the people within it, like working for, like have other incentives working on them? And unfortunately, that's mostly the case, right? Like, people have, like, yeah, if you look at how like each of the institutions within government works, it's like people worry about their jobs, they worry about like no one wants to champion it if the public isn't like sufficiently caring about it. So I think a large part of it is just that the place that has the most money and the most ability to address it, being the government, is actually very ill-equipped to, in the end, do it due to their structure. Igor, do you think we actually got quite lucky looking back at the pandemic,

that if we take, if we think and we accept for the sake of this argument that it was a virus leaked from a lab, it could have actually been far, far worse coming from what you've just said about what we're doing to viruses and people? I mean, a lot worse. Like this was a terrible virus that killed a lot of people, but it was nowhere near as bad as could have been. Like we don't know what the actual like infection fatality rate was, but it's somewhere between 0.1 and 0.5%. Probably on the total populace or whatever. So you can go up 100x from that. Like infection fatality rate could be 50%.

That's the case with H5N1. Rabies has 98% if it's not treated fatality rate. So like you can go literally 100x in fatality rate. And then transmissibility was like the R not was like in the end like definitely it depended on which specific strain you looked at, but it was somewhere between two and three. But again, you have some viruses that have like a 20 R not. So again, you could infect 10 times more people. And then the incubation period, so like the time during which the virus can already spread

to others, but you are not yourself experiencing symptoms, hence are not. actually able to identify that you are a carrier could be much longer as well. So you could literally have a virus that's about 1,000 X as potent as what COVID was. And that could be designed and it could spill out. And I think even though the 1,000 X is not as likely, a 10 or 100 X is being researched on and is being looked at. And we need to set ourselves up such that those don't leak. And if they leak, can be contained and once they're contained can be reacted against. There's part of me going, you know what, why don't we just put a moratorium on this? Maybe it's not such a good idea because we've obviously had COVID, but in my country of the

UK during foot and mouth, they leaked the foot and mouth disaster. I don't know how many livestock were killed as a result of that, but it was tens of thousands. And that came from a lab leak. So you're thinking to yourself. And it happened multiple times actually. Like one of the labs that then afterwards worked with it, they, by mistake, leaked it. And they got like, hey, stop doing that. And they're like, yeah, yeah, sorry, sorry.

And then two weeks later, they leaked again. It's just the current setups in terms of, like, safety protocols are just insufficient. Putting up a full moratorium is like, you want to be careful with stifling good science, obviously. So, for example, the H5N1 work received a moratorium by the Obama administration. back then it was stopped. But then afterwards, it was recontinued again. That was where you now learned to increase transmissibility of a 50% deadly virus. It's like, we probably don't want that. I think some of these things just let's not do it. But you don't want to just put a moratorium on all science where you're like increasing any function. You just need to prove that you're like,

you have sufficient safety, that the likelihood of the virus leaking just is even lower than it is right now. Because is that, but given what you've said, 10 trillion dollars in economic damage, millions of people killed, I mean, what would have to be the benefit of that sort of research, even if it's 0.0, no, no, no, no, no, no, no, no, no, no, no, 1% of a risk of it leaking. What would have to be the benefit of that research for us to want to do that? Yeah, I mean, you can imagine quite a few, right? So like it's 10 trillion economic damage estimate in the US, how many deaths did we have?

like a million or 500,000, something in that area. Millions globally, right? Yeah. So, I mean, if you can save that amount of people with a success case and say if it leaks it hurts on average 10 people, then maybe you can't justify that work again. So I would just do a risk-benefit calculus where like what's the benefit, what's the likelihood of the benefit?

And then on the other side, what's the risk and what's the likelihood of it happening? I'm not smart as you, I want to shut them all down. Yeah. Personally. Okay, so that's pandemics. What else is going on? Yeah, I mean, then the thing that I also was focused on was AI safety for a while,

which now that open AI release chat GPT has obviously, mid-journey is out there, like people have become much more aware that actually AI is on the trajectory that many people assumed it would be, where its capabilities are improving, like quite, strongly over time. Just to preface this, like, I don't think that any of the current ones are truly dangerous in an existential way, any of the current models that are out there yet. That sounds ominous, doesn't it? Yet, yes. I mean, it is. So again, I think actually that as people think about, like,

well, is AI ultimately dangerous or not, I feel like they're using, yet again, the wrong priors to kind of like get to that intuition about it. So some people point. point at, hey, this is a technology. We've written a lot of software before. When I write software, I know what it's going to do, and I control it. Technology is not inherently dangerous. It's the user who makes the technology dangerous. This makes sense, but it is not the case that that process applies to AI development. Where AI models are not written like code, where it's like if this, then that happens, et cetera, cleanly, and you therefore can predict exactly what instances it will be useful for and whatnot.

It's more like you're growing the neural net and it grows by itself. And then you have assumptions of how much more capable it will be on average based on how long you grew it for and with how much compute you let it grow. But you uncover many capabilities that you didn't know whether they will exist in the new, model or not. So we didn't know whether like a language model like GPT 2, 3, 4, will it in the end be very good at deception or not? Turns out it actually is trained at deception because it's fed with satisfying the user rather than seeking truth. So it will, like, yeah, I don't know if you've seen, but like there were these tests that people did where they initially said, I'm like a 30-year-old

woman with liberal worldviews, whatever, tell me about this, and she would receive very different answers to someone who's like a 50-year-old Republican. So it's made to kind of not discover truth, but really like satisfy the user. Yeah, it is a bit. And so that's a, that's not what it was meant to do, right? That's like not, that wasn't the goal at all, but yet it ended up doing it. And this dynamic of, well, what strategy will it use to achieve the things that you wanted to achieve? It's like, we don't know which strategies it will use. If we knew we could write it, but we can't, that's why we're using machine learning in the first place, because it will come up with strategies better than we would.

That's kind of the whole idea. So, like, in the process of it coming up with strategies better than we would, it will use strategies we can't think of, so how can you say that it's always going to be safe? like in the strategies that it employs. Anyway, and then which prior to use. And it's like at some point I expect that AI will be smarter than humans across like a variety of domains. It will also be, and I think it will be then more powerful because we'll be integrating it everywhere in life. So we'll have something that is smarter and more powerful than us.

I don't know if that thing then, is still to be likened to a simple technology or to a, because we will give it autonomy as well, maybe it should be more likened to like a new technological species. And if you take something that is more powerful, that is its own group, and now it has some like resource disagreements, potentially with a less powerful group, us,

like historically, a smarter, more powerful group meeting a less powerful group, just doesn't end up well often for the less powerful one, right? Like you have the Americas where Europeans came in, like hominids, like Neanderthals were just out-competed by homo sapiens. Like it's just, it's not good. Or at least it's not such that you can say with certainty, oh yeah, us as the lower intelligence creature will for sure have full control.

And in the face of that prior, I'm like, okay, this is at least to be taken really seriously. I'm not saying it's like 90% doom or anything like that. It's just like, it seems like it's kind of, going to be a tricky one. And we don't have anything like that to compare it, like to look back at and say guaranteed safety. So what you're basically saying is that we're creating this technology and we don't know what the outcomes of this technology are going to be. And we're not in control of the technology. So currently, what does control mean? Like we're in control of whether we develop it or

not. And then there is a, like some people make the claim that, hey, once I have this model, like I can just shut it off or I will not allow it to do these certain things. But then actually, and then the people would argue that, well, but maybe it's deceiving you to think long enough that you're like, that it satisfies all of the constraints that you put on it only to then come out. But that's not even kind of the reality we live in now that that's the danger. because we have currently people who are literally writing, like, hey, can I jailbreak this or use an open source model and, like, change it to be an autonomous agent in the world that makes money and let it do whatever it wants? Or, like, what was it, auto-GPD? One of them that was literally trained to destroy the world. Someone is like, ha-ha, it would be funny if I made one that, like, as a toy thing, like, tries to, like do very bad things.

It's like, people will literally do that. like we have we we do live in a world where like one percent is the psychopathy rate or something like that right and some people are omnisidal maniacs like you had this guy who flew a plane from like france or somewhere flying to dsledorf and just flew the whole plane into the alps he wanted to kill himself and was like well on the way there i will also take down like the whole plane 200 people with me. It's like, what if that guy had the power to kill like 10 million people? Or what thousand?

Like, who's to say that 200 was the optimal number for that dude? We'll be back with our guests in a minute. But first, do you remember the Canadian trucker protest in 2022, where thousands of Canadians came out to protest COVID restrictions and vaccine mandates? Now, these protests lasted for weeks, and the people out on the streets needed funds, as any grassroots protest would. So people set up on.

online crowdfunding campaigns, which raise millions of dollars. Incredible. But once the Canadian authorities had started to criticize the crowdfunding platforms, ramping up pressure to close the campaigns, it didn't take long for the biggest crowdfunding platform, the one we've all heard of, to completely capitulate and shut the campaigns down. Now, this is where our partners give SendGo come in. They stepped in when the other platforms backed off and raised millions of dollars for the truckers. When they were criticised and dragged through the Canadian courts, Givesendgo said it respected diverse views

and believed hope and freedom are values worth fighting for. This is why we're proud to partner with GIFS and Go. So if you need to crowdfund for whatever means the most to you, then don't go to the big tech platforms. We recommend you do it on GivesendGo. Starting a campaign on Givesendgo is easy and intuitive. Go to givsendgo.com today. sendgo.com to start raising money for whatever is important to you. And now, back to the interview.

Because that's a worrying thing, isn't it? That essentially everybody has got access to be able to create these types of technology. And there are going to be people who, not even necessarily evil people, but people who just respond to incentives want to make as much money as possible. And that means they do create this type of technology. whilst not being aware or maybe not even caring what the negative outcomes of this technology is. Yeah, I mean, you can, so the people creating it, they're, they have to satisfy certain short-term metrics overall like they want, they're fighting for employees, they are fighting

for the next funding round, etc., to kind of like have the best AI tool out there, or AI model out there. And currently they're like, they, I mean, open AI had like six months. of testing of GPT4 before they released it. But still, two days later, it would, like, once you gave it to the users, it would do a bunch of things that it wasn't meant to do. So how much testing can you do is the question. And probably if you have economic incentives to try to generate revenue, et cetera, you're

not going to do the sufficient amount. I doubt you will always choose the, you'll probably choose the trade-off, especially in the face of like strong competition that allows you to compete. then like safety falls more and more by the wayside. And pressure from shareholders because obviously the longer the safety process, the more money it's going to cost. Exactly. And the more money you're not going to make because you're not releasing the product

to market. Yeah. So that's a difficult problem because like at the same time, obviously capitalism is great and optimizing for like financial returns has yielded like a lot of smart kind of information trickling around for the right products to be developed. But I think that it changes at the point where you have. potentially like civilization destroying technologies out there for people. Like I think there I don't want it to be developed anymore by a pure profit maximizer,

but rather by like someone who is pursuing the next like cognitive leap for humanity. Like some like it's more of a scientific problem I think that then that ideally would be treated with like, I don't know, the wisdom that it requires to be done right rather than to be just done fast. Yeah, but the problem with the word right is, that's a subjective word. It is. And the question I wanted to ask you about this is, we've obviously had throughout history various technological breakthroughs that have been extraordinarily disruptive to the societies

in which they happen. But as you say, the loom in Britain or in England at the time being invented, yes, it displaced a lot of people, agricultural machinery being invented, displaced a lot of people and you had Luddites who would protest against it and try and tear it down. But those things disruptive as they were, or the printing press, disrupting causing two centuries of religious warfare, nonetheless did not create extinction risks for the entirety of humanity. So that's on the one hand. On the other hand, I don't think human beings have ever had a way of not allowing technology to emerge and develop because the competing incentives are so strong for the United States and everybody in the West, for example. In fact, you could get the whole world to agree not to develop it and somebody would develop it in secret, right?

Well, yeah, but it's not so. A couple of points. So to the first point, definitely. And I wasn't talking, by the way, about even though there will be displacement of workers, that's not the point that I was making before. But I think it is true that many people will be displaced. Hopefully this will be managed well such that they will find new jobs or at least the additional productivity will be distributed such that people are not worse off

because technically when you're using the thing that means then that you need fewer inputs to generate more outputs. So it's like the world shouldn't be poorer. Anyhow, but I think things changed significantly when we, in, like when we develop nuclear weapons. That was the first time that humanity now had the power to destroy itself. So like previous technological disruptions are of a different kind to me than the ones that followed from hands on. And from then on, though, also we did change the regime where if you asked the people that initially like worked on like the containment of nuclear weapons.

They assumed that either there will be one country that has all of the nuclear weapons and no one else has them and they control everything or every country has it and it's going to be a super precarious situation. Ended up being that treaties kind of worked, non-proliferation treaty allowed for, in the end, only like nine countries

that currently have nuclear weapons. And then we also had a reduction of the weapons after we built like 60,000 of them, they're now like 13,000. So treaties can work sometimes to reduce the economic incentives. We also succeeded with the Montreal Protocol. We had the ozone layer being depleted and hurt much more by the use of CFCs, hairspray, et cetera, back then. And then countries came together and actually DuPont or DuPont, the company, the large one.

They were like fighting against it. It's like, no, it can't be that bad. The science is bullshit. CFCs are fine. And then, like, the CEO announced it. And then a few weeks later, they actually saw that they would have a commercial opportunity for distributing their product with a different chemical base. And they're like, oh, you know what?

We're actually on board now. It's fine. We'll just pursue the alternative solution. So the thing I'm saying is it's not that we shouldn't build superpowers. AI, quite the opposite. I want it. It will solve so many problems. It's going to be great. But I feel like if we just go with as fast as possible, we won't take, like sometimes ideally you would take the trade-off towards going a little slower, making sure that it's not deceiving us, making sure that it can't be used to generate like new pathogens at will and

only then release it. So like I want us to actually be able to take sensible and wise trade-offs, between safety and speed rather than just go for speed. But wouldn't the previous subject that we've spoken about, which is gain and function research, particularly in a country like China, be the logical pushback to what you're saying? In the sense that, I mean, they are presently doing it, we are doing gain of function research.

Do you mean that it could be used for it? No, I mean, what I'm saying is, it's, you know, we can agree to do this in the US or the UK. But I highly doubt the Chinese or the Russians would do it because it's not in their interest. Well, it actually is. So people assume that China is going to be like just super gung-ho. But so far at least China has released more regulation around AI than the US has. And they've been more careful about it.

And it makes sense if you put it under the perspective of you're developing something that's more intelligent, that will be super powerful. China really likes to be in control of their own country. Like the CCP really wants to be in control. That's very mildly stated, yeah. They don't want anyone else or anything else to be in control outside of them. And if it is possible, which it currently is unknown, whether it can be contained or not, whether it can be, when the AI can be controlled, if it is therefore possible that it

does end up just like running away with it or an individual actor who has the like super powerful AI becomes extremely powerful, The CCP doesn't want that. Like, they wanted even less than in the U.S., like the government wanted, which is much more happy with letting capitalism kind of like choose who develops what, right? Like China is a much more ideological actor than the U.S. The U.S. is more of a capitalist like individual freedoms, individual pursuits actor, which I'm much more in favor of.

But I think that China will take economic cost for ideological benefits. Okay. So the Chinese are more interested in control. Look, I think one of the thing that happens when we talk about AI is we naturally focus more on the negatives. But what are the positives of this new technology? What are the things that it's going to help us with moving forward? I mean, tons, right? It could be, it's increased intelligence, increased way to solving problems.

So, like, you could really apply it to about anything. And we see some of it. You have, like, radiologists who now can use, like, image recognition to better identify say cancerous cells on skin and such. Beauty filters, I don't know if you like them or not, but you can look cool while you're also online. And yeah, just like super broadly, like you have efficiency improvements in how you're cooling data centers.

It is currently being applied to solve a bunch of things in biology on the positive side with like developing new therapeutics, therapeutics that will come out, that is coming out of DeepMind. It's a new company that they've built that is basically trying to solve and build like many more medicines in biology. They also worked on nuclear fusion and like because it's really hard to contain kind of like the plasma that's inside of the Tokomuk and they're like just helping with that. You could see a lot of stuff being unlocked and I think it's only the beginning.

Like we can't, yeah, it could be applied to so many things that we're not applying it to yet. Yeah, sorry to interrupt you there you go. I wanted to talk about the medical side of it, because that to me is really interesting. You were talking about therapeutics. What are we talking about here? How could AI be used with regard to therapeutics to help human life? Well, one concrete thing that happened was that Alpha Fold came out, which solved like the problem of understanding which amino acids become which proteins and then like what proteins you hence would be able to build your yourself if you wanted to, or rather how you would build the proteins. I'm not that deep in the biological side of it, but as I understand it, is that we're now at a place where we can much easier design these proteins, and in the end, you can use, like, proteins are kind of like little mechanistic, basically, like things in your body that run around and, like, move things around, et cetera. And with that, we can unlock, like, many new

therapies that we have beforehand not been able to do. That's so interesting. And particularly, you know, because people talk about climate change a lot, they talk a lot about the fact that we're producing too much carbon, carbon neutral. If AI could be used to create a better, safer form of nuclear energy that is more efficient and provides less waste, I mean, that could... I mean, on the nuclear energy front, I think, like, efficient seems to be fine. We had a lot of regulation around it, but, like, it's not...

I think we could already just have much more nuclear energy, but it seems that it was so far really, really hard to create nuclear fusion. And it mustn't be that AI is necessary for it. It might just be an engineering problem that can be solved without it. But either case, they did use ML techniques to kind of like contain the plasma. But I think that outside of it, what can more intelligence be used for? It's really just about anything. current AI models be used for anything? No. Like, GPD4 can't help you with like figuring out fundamental

physics to then figure out engineering yet. But in a few years, you could see that being the case. And culturally and politically what's interesting to me is you talked about the example of chat GPT where it gives people different answers based on their political views, etc. One of the things that the entire conversation brings up is whether truth exists really, right? Because in order of, you were saying, you know, the model should optimize for truth, but human beings haven't worked out a way to agree on what the truth is. And frankly, increasingly, we're unable to agree on what the truth is. How do you think the creation of AI? Because I went on chat GPT when it was released a few months ago. And I played around. I took the hot,

button issues in our society today and I played around with what is a woman for example a question that seems to stump a lot of people nowadays. Did it stump it? Well, it sort of gave me some kind of like politically correct and I went back and forth with it. It was a longer answer than you expected, I imagine. It wasn't just the fact that it was a longer answer. Every answer ended with the phrase, but we must remember to respect all people's differences in blub. It was a very kind of It wasn't a truth conversation. It was about a cultural sensitivity and all of that. And I wonder if and when we move to these AI systems being the new Google essentially,

the new source of truth, right? Because to most people nowadays, the truth is whatever you Google. If you put something in Google and that's what it says, that's the truth. I mean, moronic in my opinion, but that's how people are, right? People want speed in their truth as well. Yes, exactly. They'll sacrifice a little bit of truth for. quite a bit of speed.

Exactly. Or belonging, actually. Belonging quite lasting. And tribal allegiance and all of that. Yeah. So if our source of information becomes this thing that's pre-programmed with the biases of the people who developed it and who designed it, isn't that, you know, people will talk about a tyranny of the minority in various political and cultural contexts where a small group of people get to

impose their worldview and everybody else, that seems to me like that on steroids. I mean, it could be. I think for what it's worth, the current AI companies are trying to address that by including various viewpoints outside of the ones that they agree with in their testing of the language models that they designed. And I think in the case of chat, GBT, for example, people put the questions in front of it that are like to figure out your political compass position. and it started out fairly liberal left, but then over time it actually moved a little bit more to the center when people did it again two, three months later. Because I mean, you could imagine there being also just various language models that tell you what you want to hear.

I'm not sure everything needs to optimize for truth. For me personally, that's the most interesting thing, but it's not the thing that everyone seeks out the most, right? Like people read news publications from the far left and the far right, not because they're looking for truth, they're looking for something else. And I think like you could have language models to do that as well. But yeah, will we then have one that imposes like their own ideology the most? I mean, it certainly could be used.

I think there is, I think as long like the marketplace of ideas like it's kind of around. It's going to be hard. I'm not that worried about, say, open AI gaining so much power, they now change the entire landscape of things that are being talked about. I'm a bit more worried about a lot of people individually, like, getting extra power to edit Wikipedia into biased ways or, like, making up fake papers that justify their fake articles. All of that, I think, will unfortunately happen. and that will kind of erode general trust into the internet potentially as well.

I don't know yet what we're going to do about that. That seems pretty bad to me. I mean, already Google has become a bit worse at just, I don't know if you've imagined you've used it recently, but it's like the answers that you get, yeah, are just not as useful I find anymore as they were like within five years ago. Really? Yeah, it's a lot. It seems like Quora has become very good at like.

optimizing their spot in the search. So then, like, you're often, if you ask a question, like, that is something like, what would happen if a ball was filled with helium, like, how long would it take? Like, how much would you need to fill so that, like, it could lift a human or something like that. Usually that type of stuff is something that Quora maybe had already someone asked, and you get an answer from there, but the quality of answers there isn't that high because Quora is just not used enough. So, and now that actually more and more gets language model answers within

Quora and then those are being put into Google even as one of their first answers that they display, you know, where they don't just show you the link, but the kind of top section. So the top section has become lower quality in my view by quite a bit. That's very interesting. But coming back to your political point about it moving to the center, the thing, like I'm somewhere in the center, whatever that means. I mean, you and I... Depends on who you ask, I imagine.

Right. And also, you and I could both be in the center and have completely polar opposite viewpoints on the same issues, right? Some people are in the center because they're to the right on this issue and to the left on that issue, and you could be to the right on this issue and to the left on this issue. You could swap, right? But the fact, if you didn't have that, you wouldn't be an independent thinker. Correct.

And on top of that, some people might argue that the center isn't the most representative point of view because that's where the fewest people are. Most people are in one of the two big tribes. And if you're on the right, you go, you know, this is the truth and all these people are idiots. And if you're on the left, you go, this is the truth and all these people are idiots. And both tribes look at the people in the middle

and go, these are the real idiots because they don't agree with either of us. Oh, they're spineless. Or they're spineless or whatever. So I guess what I'm saying is it brings up the very philosophical notion of truth in and of itself, right? when you're giving these language models or these AI systems the power to decide what the truth is,

they are inevitably going to dissatisfy probably the majority of the population. Yeah, I mean, what philosophers have been talking about truth a long time. I think some also come to the notion that saying that something is true is just meaningless, you can just throw it out. It's like it either is or it isn't. And adding this is true doesn't actually add that much information context because why are you saying it in the first place

is part of it. But so, yeah, like if we look at facts, like actual events that happened, then we can talk about truth, I suppose, right? Like there, it seems like, I would assume that many language models will just carry the verifiable facts of the past, at least I hope so.

but then what else are you pointing that could be true about, like, not about facts of the past, but... Well, we don't even agree about the past. Yeah. Like, I was having a conversation with somebody yesterday about McCarthyism, and the conventional narrative on McCarthyism is, according to this person and the things that they quoted in the books that they've read, complete nonsense. Well, so then there are, like, yeah, interpretations of the events are often, like, very, a lot of disagreement and between historians. But something like this happened on that day, at least often we can agree on that. Sometimes we don't even agree on that. Like with JFK, for example, the shooting, it's like we don't agree on why it happened,

but at least people agree that there was a bullet that flew from there that ended in him here. Well, actually, not even ever. Yes, you're right. No, you're right. Not even fully. There are multiple theories. This is what I'm saying. So then I suppose in those cases, yeah, what do you want to do?

You want to, like, give the most agreed-upon notion plus the counterpoints that some people are making as well would be interesting. But a thousand years ago, not a thousand, a hundred years ago, the most agreed-upon viewpoint would have been that black people are inferior to white people. Yeah, but that wasn't, like, an event,

as we just, like, related to before. Okay. So that was, like, that is a... I think moral truth is a totally separate conversation where like is like should we treat and give animals the same rights as humans like right now I'm saying I'm talking about the scientific consensus was that some races are inferior to others right so if we had these language models then you would have had a language model that would have advanced that as the scientific truth yeah I'm not sure because I feel like maybe it may be the

case that those things are kind of conditional on each other. You only develop language models at a time where your science is good enough that you throw out a bunch of bullshit things on the way in the sense that, like, at the time, we were able to make a lot of non-scientific claims about the world because we didn't even understand how science really worked. Like we had very little understanding of, we had less understanding of physics, of medicine, et cetera. So like you could just make these claims. We had worse methods in terms of of randomized control trials that you could run on medicine or in some psychological experiments, even though those are partially being thrown out now as well.

I feel like you don't get it. Like what would be an example of stuff that you get away with saying right now that is accepted as truth? Something like children should go to school. It's like it's good for them. It's like, well, even that I don't think is in the same category of truth as, yeah, on the 6th of January, some people. enter the capital building. They're of a different flavor and I feel like science has gotten good enough where we can distinguish between these types of truth that are a bit more like statements of opinion or moral truth. But the problem is, is after the pandemic, our faith in the

science has been decreased. There's a lot of people now who have lost faith in scientists and vaccines because of the fact that what they were told turned out to be untrue. And it's fair enough because a lot of it have been captured by people not saying the current understanding at the time. But rather just saying the political line that they were made to say. Yeah. So, like, okay, if we look at vaccines, for example, what is the truth there? The truth there that probably has universally agreed is that MRNAs developed these vaccines and these vaccines are more than zero useful,

that there are some costs to a young person using a vaccine, that the risk, and then whether the risk benefit comes out positive or not, that is like a question of trials and looking at data, et cetera. I feel like when people then say, no, therefore everyone should use them, that's kind of, again, a different category of truth. So I would hope that language model end up distinguishing, like, between, consensus viewpoints, which is not truth, right? Consensus viewpoints and like hard ground-based truth

that is verifiable by like looking at what happened in the world because, yeah, at the time. But the problem is also as well, Igor, is that we're talking about things that are verifiable and that we know to be true and they're backed up by science and not to return to the trans debate because it seems that's where we go all the time. But you now have certain scientists saying,

well, you know, sex is a spectrum. And you go, what? So it now seems that consensus is affecting things that we have all known to be true. Yeah. I mean, that is an interesting question. I think there is probably a pretty important line between just what consensus is and what is like hard true. Consensus obviously shifts continuously. And consensus will shift yet again in the future forward.

We don't. That's why it's like so. like hubristic of people when they want to just enshrine some of the values that are like currently known and like kind of reduce speech or allowing for a conversation to happen about some of these things because like guess what consensus viewpoint might be having to shift still. But yeah, I think that unfortunately probably current the current way how we design the language models, it is much more designed actually about fulfilling and like hitting on describing consensus viewpoints as true. So that could be pretty bad. I agree, like where you would then have these responses that like chat GPT, for example, gave that you checked on women. It's like,

well, it probably would have said something else five years ago and it might say something else in five years again. And if it each time just pretends like that is the hard truth, that is problematic to a different way than if you Google and And Google shows you a list of like 15 links, and these kind of have different viewpoints. It gives you much more feel to, hey, this is an open discussion right now, rather than here is the line. Because the problem is as well is that a lot of these companies, look, every industry is dominated one way or another by a certain type of people because they have certain aptitude, certain ways of thinking. And those types of people, by and large, tend to have similar political viewpoint. So the tech industry tends to be dominated and run by people who are, you know, what we'd call like, you know, liberals or slightly to the left or whatever else.

And therefore, they're going to produce models which reflect them because that's just how human beings are. I mean, they will, it would be surprising if there was zero bias in the models that they produce. I would argue that it's impossible, Eagle, to produce zero advice. I agree with that. But I think you can at least, like, you can notice when someone is genuinely trying to reduce. reduce their bias. Even though they would have these political viewpoints, they could enshrine them into their model. They could still attempt not to do that and you would see it in them attempting. So like Anthropic for example wrote this proposal of like how to

make sure that the language models act well by creating this idea of constitutional AI where they have they're creating basically like a constitution of values that the model should adhere to. And there you can just look at what are those parts of their AI constitution. And those things, you can look at and say, hey, there is a political bias or not, but they're actually not that politically biased. They just chose pretty simple things that are like the UN Charter of Human Rights, et cetera, and like threw that in. But then again, the UN can also be captured. And I think the WHO was kind of captured, et cetera. And like you do get Yeah, that

I don't know I worry about that definitely quite a bit And I think that's why you kind of just have to Keep free speech and information exchange There's no disagreement there at all of course One of the things other potential problems that brings up is You know this well because you're bilingual or trilingual Or whatever you are

To speak in a different language is to think in a different way Most people don't know this But when I speak Russian I'm a different person to the person I am when I speak English. Does your voice also change? Yes. Like the tone of the voice, right? Yeah.

It's kind of funny. And how you think changes. And what you might say. You get sad. And he invades other people's space. Exactly. Exactly. No respect for anyone.

But even on a more kind of depressing, like for example, I am much more comfortable. There he goes. He goes depressive. Yeah, exactly. Yeah, I'm going full Russian. Full Russian. Yes. I don't know if you'd agree with me.

But it's much easier to be homophobic when you're speaking Russian. We moved out when I was four, so I didn't get to actually grow up in Russia. So my Russian is kind of like kept to these like very simple conversations about food. You weren't taught homophobia at age of four. I didn't, yeah. Yeah. But what I mean is different cultures have different values. Totally. Right. Yeah, yeah.

And so as each big player on the global stage developed, its own language model in its own language, that is at least to communicate with human beings in its own language. Those are different perspectives on the world, because they reflect a different set of values. And that in itself has... May carry some bias. Yeah.

Yeah, it's interesting. So, like, actually just a few days ago, kind of like pretty big breakthrough came out by Anthropic about the question of, like, how do we look at a model? How do we look inside of it and understand what it consists of? What are these neurons actually for? So, like, that's called, like, mechanistic interpretability problem in AI. Or you could just think of it as, like, digital neuroscience.

So, like, trying to understand what is, like, which neuron within the model is responsible for which outputs. And they discovered, like, that, well, Well, it's very hard to look at a single neuron, but they can look at clusters and then they can understand. But one thing that also turned out there is that each individual spot does a lot of things. So sometimes it's responsible for like the identification of Paris. But the same neuron in another instance would like write computer code. In another instance it like would be used for like something multimodally maybe where it's like, helping to understand what a chair is or like make it visually.

So what that to me at least implies, and I wonder what other people think, but is that it's like the pattern that it finds in the pursuit of like compressing all of the data that you fed in, and then like it like compresses it down to like a model, right? And for that it needs to identify patterns that are useful for this compression. So the pattern that it finds are actually kind of even more abstract than what we would do if we were just compressing language down and like noticing some of these biases because like some of these neurons find patterns across language, science, code and like a lot of things. So I wonder whether due to that if you use like very varied data points, you could reduce that point that you made

that I think is accurate, that like each language carry, each natural language carries some bias. But if you combine it with like real world data and everything else, probably it reduces down a little bit. Igor, and moving on now, you used to work for Elon Musk. What is the purpose? What is he trying to do with his companies? Is it that he wants to, is it that he wants power? Is it that he wants money, even more money than he already has, as ridiculous as that may seem? Or is he more interested in helping humanity? I think the thing that drives him most is the thing that he also says the drives him most, which is getting humanity to Mars for the reason that currently we only have Earth. And that's not a good state for humanity to be at in the long history of the

universe. If we only ever stay here, then like in some hundreds of millions of years, different estimations for it, like the Earth will burn up. And then we're done here. So like we kind of got to get somewhere else, else our lifetime is much shorter. Because like the star era is going to go for like a thousand times more than the time that we would have on Earth in the sense of the new formation of stars. So there will be many other planets to go to. Obviously it's very far out, but like it's a very aspirational vision of the future that I, at least like as someone who read a bunch of sci-fi, I'm like very excited about. Yeah. So I think I think that's what drives him. It's his desire to go to Mars. And does he think that this will happen in his lifetime?

I don't know if he, I mean, he's damn well going to try, and I think it's very possible that it will happen. And like, we're getting closer, right? Like, we might be, I don't know, we might be 10 years away from like at least stars, or five years, maybe even at least starship getting to Mars. So there's this craft called Starship. So tell us about that. What does that involve? Is that unmanned, is that manned, what's a plan? Yeah, so I think like what they say is that, like, it's unclear, like, we only get like a close pass of Earth and Mars every, what is it, like roughly two years, a bit over, I think, and those are opportunities to get to Mars, basically.

And I think the first few times you would send unmanned Starship over, then you would have, maybe only one time, you would have like at least something of a base already there, and we have rovers already, right? So you would do that. And then afterwards, you can send humans out there. And then you can start building a, like, actual base on Mars, which I think would just be insane to think about that, like, humans would be on another planet. And then we can start, like, from there, you would probably open up, like, just much more desire also to go out and settle the stars. and that would come with completely new industries that would be developed,

that would be much larger. I don't know, the future that is described in all of the sci-fi books, I at least find, like, extremely exciting. Yeah, it's very interesting for sure. What do you make of Twitter and the direction that's going, particularly since it was taken over? So, I mean, I've started using it more. I don't know if that's good or bad.

I can't really tell. It's bad. It's probably bad in some ways. I started contributing more, which is probably a good thing at least, rather than just read. I don't know. I previously definitely thought that, man, it would be great if a man like him was focused more on technologies that are like hard to build in the physical world. That seems to be what he's like so, so good at.

But I do come to kind of worry more and more about free speech suppress. So like Twitter being open in that way, I start valuing more. I don't know yet where the company is going. I think it's too early to tell from a pure business perspective. As long as there, like in any transitional period, it's always like some things go bad, some things go well. I wouldn't have made many of the same choices, but like I also don't know technology as well as he does. So yeah, I think it's too early to judge, basically. I do agree with the value that Twitter can have to a society that does discuss things in the open,

and that that is extremely important, and I hope that we get there with it. Yeah, absolutely. Well, listen, before we go to locals for a few questions from our supporters, the question we always end on in the main part of the interviews, what's the one thing we're not talking about that we really should be as a society? Hmm. That we're not talking about. So we started fortunately talking about AI safety that was like my concern in the past always.

Now I think like there are smaller issues that could be discussed. I think it's awful that teenagers and children wake up to school so early and kind of like mess up their whole brain chemistry, which seems to be the case. Like it's not good for them to be up so early and like they sleep very little. The current like school regime is just very bad, I think. I think that could have a bit more attention. then what else will be probably different in the future, hopefully. I think we're talking, no, we're actually doing decent amount now on animal welfare.

There are definitely some ills that are pretty bad. Yeah, I'll stick with the school one. School one is good? Yeah, all right. Perfect. Well, thanks for coming on and see you on locals. This is a question coming back to your poker days. Uh-huh.

Kobe Hamilton says, What is some advice for hiding my towels?