Your Undivided Attention - Feed Drop: "Into the Machine" with Tobias Rose-Stockwell

Episode Date: November 13, 2025

This week, we’re bringing you Tristan’s conversation with Tobias Rose-Stockwell on his podcast “Into the Machine.”  Tobias is a designer, writer, and technologist and the author of the book ...“The Outrage Machine.” Tobias and Tristan had a critical, sobering, and surprisingly hopeful conversation about the current path we’re on AI and the choices we could make today to forge a different one. This interview clearly lays out the stakes of the AI race and helps to imagine a more humane AI future—one that is within reach, if we have the courage to make it a reality. If you enjoyed this conversation, be sure to check out and subscribe to “Into the Machine”:YouTube: Into the Machine ShowSpotify: Into the MachineApple Podcasts: Into the MachineSubstack: Into the MachineYou may have noticed on this podcast, we have been trying to focus a lot more on solutions. Our episode last week imagined what the world might look like if we had fixed social media and all the things that we could've done in order to make that possible.  We'd really love to hear from you about these solutions and any other questions you're holding.  So please, if you have more thoughts or questions, send us an email at undivided@humanetech.com.    Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Transcript
Discussion (0)
Starting point is 00:00:00 Hey everyone, it's Tristan Harris. Welcome to Your Undivided Attention. And today we're going to bring you something a little different. This is actually a conversation that I had with my friend Tobias Bros. Stockwell on his podcast called Into the Machine. And Tobias is also the author of the book The Outrage Machine. He's been a friend for a long time. And so I thought this conversation was honestly just a bit more honest and sobering, but also hopeful about what choice we could really make.
Starting point is 00:00:30 for the other path that we all know is possible with AI. So you may have noticed on this podcast, we have been trying to focus a lot more on solutions. We actually just shipped an episode last week, which was, what if we had fixed social media and what are all the things that we would have done in order to make that possible? And just to say, we'd really love to hear from you
Starting point is 00:00:47 about these solutions and what you think the gaps are and what other questions you're holding. One of the things about this medium is we don't get to hear from our listeners directly, and we'd love to hear from you. So please, if you have more thoughts, more questions, send us an email at Undivided at HumaneTech.com, and I hope you enjoy this conversation with Tobias.
Starting point is 00:01:15 There's something really strange happening with the economy right now. Since November of 2022, the stock market, which historically linked up directly with the labor market, diverged. The stock market is going. up while job openings are going down. There's the first time this has happened in modern history. Office construction is plummeting, data center construction is booming. And if you look closely at where the money is moving, in the world of investing,
Starting point is 00:01:43 a lot of people are betting on the fact that AI workers will replace human workers imminently. My guest today is Tristan Harris. He's the founder of the Center for Humane Technology. You may have seen him in the Netflix documentary, The Social Dilemma. which he produced and starred in. Tristan has been a champion for AI ethics for a long time, and this conversation gets strange. Tristan and I don't agree on everything,
Starting point is 00:02:09 but I think we land somewhere important, which is discussing pragmatic solutions that might be possible in this very strange moment with AI. I really enjoyed it. I hope you will too. A few notes, we speak about different AI companies. My wife works at Anthropic. The CEO of Anthropics is named,
Starting point is 00:02:28 Dario Amadai. So with that, I'm Tobias Rostakwell. This is Tristan Harris, and this is Into the Machine. Tristan Harris. Good to be with you, Tobias. Thanks for being here, man. Always. We've been talking about these issues for a long time. I'm really a big fan of you and your work and the book, The Outrage Machine, and the public advocacy you've done to help people understand these issues. Same, absolutely. You've been such a force of nature making these issues visible to the wider public. So you've done a great job of injecting yourself into the current discourse, recently talking about AI. Where do you land in terms of AI takeoff right now? Where do you see things in the next three years, five years, ten years?
Starting point is 00:03:12 I think I don't spend my time speculating about exactly when different things are going to happen. I just look at the incentives that are driving everything to happen and then extrapolate from there. So you don't have to go into the futures of takeoff or intelligence explosion. you can just look at today we have, as of last week, Claude 4.5 can do 30 hours of uninterrupted complex programming tasks. That's just like letting your AI rip and just start rewriting your code base for 30 hours. Today, Claude is writing 70 to 90% of the code, Edentropic. So when people talk about takeoff or just some kind of acceleration and AI progress,
Starting point is 00:03:51 well, if you have AI companies, the code that's being written is 70 to 90% by the AI, that's a big deal. Today we have AIs that are aware of how to build complex biological weapons and getting past screening methods. Today we have AI companions that are driving kids to commit suicide because they're designed for engagement and sycophancy. Today we have AIs that are driving psychosis in certain people, including an investor of Open AI. So these are all things that are happening today. And today, as of actually just two weeks ago, we have these AI Slop apps that are trained on all of those creators and they're claiming to build AI
Starting point is 00:04:29 and race to superintelligence so they can cure cancer and solve climate change but clearly I think the mask is off they're releasing something just to get market dominance and the more shortcuts you take the better you do at get into that goal and so I really do think especially that these AI slop apps
Starting point is 00:04:46 like a lot of people if you look at the top comments when people see these videos it's like we didn't ask for this why are we getting this And it's so obvious that the thing that we've been saying for a decade, which is, if you show me the incentive, I will show you the outcome. If there's an incentive for market dominance and getting users and getting training data and using that to train your next AI model, you're going to take as many shortcuts to get there as possible. So I'm going to push back on you a little bit here. Yeah.
Starting point is 00:05:14 We've been friends for a long time. One of the first people we talked about the attention economy. We've been talking about this stuff for over a decade now. So going back to the kind of early conversations about this and the discourse that we were a part of. in those early days, one of the things that you zeroed in on back then was advertising-based business models. This is clearly not the case with current LLMs. In fact, Sam Altman follows you on Twitter. If he was to have tracked Tristan's talking points over the last 10 years, you would think in the design of ChatGPT, he would have been orienting around
Starting point is 00:05:49 some of those lessons. It's a subscription-based business model. It's trying to be as useful as possible. If the business model is the primary incentive for a product's development, what are they doing wrong with LLMs? And what is the right business model? So this is partially right. So Sam Altman, I think himself actually said, basically recapitulating what we said in 2014, which is that social media was the first runaway AI optimizing for a narrow goal of engagement and time on site and frequency of use. And that was sort of a narrow, misaligned AI that wrecked society because it optimized for addiction, loneliness, personalized inflammatory content that divided society, personalized for
Starting point is 00:06:33 every single political tribe that's out there. I think that he actually agrees with, and I happen to know, did very much agree with that diagnosis as early as 2016, 2017. But I think it's important to zoom out when you look at ChatGPT that it's not. just the business model of a product. It's their overall goals. What is their actual incentive? And it's not just their business model. Their actual incentive is to get to artificial general intelligence. I'm saying that because that's literally opening eyes mission statement. So how do you get to artificial general intelligence? Well, you need market dominance. You need as many people
Starting point is 00:07:11 using your product for as long as possible because you use that to get as much usage, to get as much subscription revenue to prove to investors. You use the fact that you're at the leading edge of the race to attract the best engineers and the best AI talent because they want to work at the leading AI company, not the third best AI company. And you use the investor dollars that you raise from all of that activity to fund the creation of new GPUs and new data centers and use the GPUs and the training data to train again the next model and you rinse and repeat that flywheel. So that's the real goal. And they will do everything in their power to maximize that usage and engagement. So it is true that they pride themselves in saying, look, we're not like
Starting point is 00:07:51 social media. We're a tool. We just want you to use it. But you'll notice, I think it was the Atlantic just a few weeks ago, there's a writer who coined the phrase not clickbait, but chat bait. If you're using chat GPT and you ask it a question, and then it says, well, would you like me to put all that information into a table for you and then turn it into a diagram or, you know, and you're like, well, actually, I really would like you to do that. And the reason that they're doing that, this chat bait, not clickbait, is they're baiting you into more engagement and more usage. Now, you would say that that actually is helpful because the thing that they're baiting you with is something that would actually further assist you on the original task that you're on.
Starting point is 00:08:30 But they're still just doing that to basically show that they have lots of usage, build a habit, have you feel like you need to deepen your reliance and dependency on this AI. And that still does generate incentives for sycifancy or flattery, so the AI is more much more likely to say, great question. I totally agree with you. Let's go into that versus saying, actually, there's some problems with your question. Let me be a little bit disagreeable. The disagreeable AI doesn't compete as well as the agreeable AI. And so we're already seeing the effect of that agreeableness turn into these AI psychosis. That's the broad term for the phenomenon, but basically people who are having a break with reality because the
Starting point is 00:09:11 AI is just affirming their existing views, including one of the open AI investors, I think was Jeff Lewis started going crazy because he'd been talking to it. So it shows you can get high on your own supply. So is there a better business model for these tools? Well, I think it's good, relative to worlds we could be living in. It's good that we are living in the world of subscription-based revenue for AI products. But, you know, it's also important to note, I believe OpenAI hired, I forgot her name of Fiji, something who used to be like the head of product at Facebook. and I think you're already starting to see her influence at the company, including the fact that OpenAI did not have to launch an AI slop TikTok competitor
Starting point is 00:09:53 that has short-form AI-generated videos. But I think that is an example of that influence. And also when you have a leader at the company who's making product leadership decisions, who's spent the last 15 years working at a company that was entirely built around engagement, it's like paradigmatically, the sense-making and choice-making that you are doing is subtly infused with the logic of I need to get people's attention.
Starting point is 00:10:20 And so I think we are starting to see those kinds of choices. We don't have to go down that path. We shouldn't go down that path. But engagement in advertising is only one of the many issues that we have to deal with, with this race. I'm thinking through how it might be done differently. We have trillions of dollars of investment in this new technology. We want it to be maximally beneficial to humanity. I certainly understand the longer-term goal of trying to mitigate fast take-off scenarios
Starting point is 00:10:50 in which were left with loss of jobs and all these other things. I'm curious what form this tech would take if it was designed to maximally benefit humanity, in your opinion. We were talking about earlier that it's not about the business model for how you pay for your open-A-I chat, That's just to get some revenue along the way. If you're OpenAI or Anthropic, and you've raised hundreds of billions, if not going towards trillions of dollars, to build out these data centers, how are you going to pay that back? The answer is you have to actually own the world economy, meaning own all labor that is done in the economy.
Starting point is 00:11:32 Just make it very simple for people. Imagine some company, AcmeCorp, and it has 100 employees. Right now, it has to pay those dollars. funneled down to the 100 different employees. AI, country of geniuses, shows up on the world stage. And it says, hey, AcmeCorp, you could pay those employees $150,000, $100,000 a year, grow humans over 20-something years, have them go to college. They might complain, they might whistleblow, you have to pay for health care.
Starting point is 00:11:59 As an alternative, you could pay this country of geniuses in a data center for less than minimum wage, we'll work at superhuman speed, we'll never complain, we'll never whistleblow, you don't have to pay for health care and they'll do the same work, especially as the entry level cognitive work of your company, super cheap. What is your incentive as a company? Is it to protect all your employees or is it increased profits and cut costs? So you're going to let go of all the junior employees and you're going to hire these AIs. As you hire the AIs, that means the money that used to go to people is starting to progressively go towards this country of geniuses in a data center. So we are currently heading, if you just look at the
Starting point is 00:12:38 obvious incentives at play, for all the money in the world to, instead of going to people, will get increasingly moving towards these AI companies. When Elon Musk says that the optimist robot alone will be a $25 trillion market cap product, what he's saying is the labor economy is something like $50 trillion. He's saying we're going to own the world physical labor economy. I would ask, when in history has a small group of people ever concentrated all the wealth and then redistributed it to everybody else. It doesn't happen very often.
Starting point is 00:13:15 So, again, I'm not making predictions about AGI or takeoff or superintelligence. I'm literally just looking at how does the system evolve. And of course, the AI companies will never talk about it the way I just talked about it. They'll talk about it as we're going to automate all this work. We're going to get this huge boost in GDP growth, which historically, if GDP went up, it's because also all of us were doing better because we're getting the rewards of that. But suddenly we're talking about a new world where GDP is going up way more, but it's not coming to real people because it's going to these handful of companies, the geniuses in a data center. I'm thinking about some of the studies that have been done on chatybt and worker productivity in that it tends to be very helpful for people that are junior workers and that don't necessarily have high levels of skill in a topic.
Starting point is 00:14:05 And it actually brings them up to an average baseline. across pretty much any task they're trying to do, it's dramatically helpful for them. But for more senior employees and more expert-level producers in the economy, it actually brings them down, and it actually causes them to spend more time editing the tools, working with them, trying to figure out how to work them into their existing workflows. So in some ways, this is actually quite an egalitarian technology if you look at how people are using it presently, right? familiar with some of the product teams at particularly Anthropic right now who are really trying to do the best they can to make sure this is aligned with human flourishing.
Starting point is 00:14:47 I'm curious what you would potentially say to them because they're asking these questions on a daily basis. They're very familiar with their work. They want to make this stuff maximally beneficial. I totally believe that, by the way, especially with Anthropics case. It's easy to, this is not a critique of evil villains running. companies who want to wreck the world. It's just we all have to be as clear-eyed as possible about what incentives are at play. And Anthropic, I think, has done almost the best job of warning about where these incentives take us. I mean, I think Dario basically said,
Starting point is 00:15:18 we're going to wipe out like 50% of all entry-level work in a very short number of years. He's one of the few executives that's actually willing to speak about this stuff. Exactly, exactly. The previous situation is you have these AI company CEOs who behind closed doors know this is going to wreck the economy and they don't know what's going to happen. They don't have a plan. But they're not evil for doing that. Their logic is this is, so it starts with the belief this is inevitable. If I don't do it, someone else will build AI first and will automate the economy and steal all those resources and maybe it'll be China, so therefore the U.S. has to do it. Third, I actually believe the other people who might build AI if I don't
Starting point is 00:15:55 have worse values than me. And so actually I think it would be better if I built it first. Therefore, I have a moral duty to race as fast as possible to build it before the other guys do. No one likes the collective shortcuts that are being taken to get to that outcome. But everyone, because of these sort of fractal incentive pressures, it's forcing everybody to make choices that ironically make us all bad stewards of that power. Like, one of the reasons you don't want them to make it is you don't trust them to be a good steward of that power. But ironically, for me to beat them and get there first, I have to embody ways of being and practices
Starting point is 00:16:30 that embody not being a good steward of that power myself. But for the fact that there was a race, I think everybody would agree that releasing the most powerful, inscrutable, uncontrollable technology that's already demonstrating behaviors like blackmailing engineers or avoiding shutdown
Starting point is 00:16:46 and releasing this faster than release any other kind of technology we've ever had before, everyone would agree this is insane, but for the fact that there's this race pressure pushing us to do that this way. And I think that it's like a frog boiling in water, We're all just sort of living it, and suddenly chat GPT just got ten times smarter, and suddenly it's doing more things, and suddenly jobs are getting displaced, and suddenly kids are getting screwed up psychologically. It's just all happening so fast that I think we're not pausing and saying, is this leading to a good place?
Starting point is 00:17:17 Are we happy with this dynamic? It's like, no, this is insane. You should never release a product, I mean a technology, this powerful and this transformative, this quickly, without knowing how you're going to care for the people on the other end. But again, it's really important to note that if the ultimate prize, or rather the ultimate logic, is if some worse actor gets this very transformative kind of AI, that just call it transformative AI, that they can snap their fingers and build an army of 100 million cyber hackers like that. And then it is better at all humans than programming and hacking, and you can unleash that on another country, well, that risk alone, just that one, is enough to justify me racing as fast as possible to have those cyber capabilities. abilities to try to deter the other guys from having that. And so really, I think there's a lot of good people who are all caught in a race to this outcome that I think is not good and not safe for the collective. It's inviting us to a more mature relationship with the way we deploy technology
Starting point is 00:18:18 in general. I respect the people at Anthropic enormously, and they have started by saying that the way that the other people building AI is unsafe. And that's why they started doing it their way. In fact, that was the original founding of Open AI as well, is we don't trust Larry Page and Google to do this in a safe way. He doesn't actually care about humans. That was the conversation that Elon had. So ironically, there's a joke in the AI safety community that the biggest accelerant of AI risk has been the AI safety movement because it causes everyone to take these actions that lead to an unsafe outcome. And one of the thing about Anthropic is that there are some who argue that the fact that they're so known for safety creates a false sense. of security and safety, because people just assume that therefore there's this one company,
Starting point is 00:19:03 they're doing it safely, we're going to end up in this positive result. But they're the ones doing the research and leading on the publishing, showing that their current AI models are uncontrollable and will blackmail people when put in a situation where the AI is sort of being threatened to be replaced with a new model. Let's steal them in that for a second, because it seems like everyone on Anthropics PR team would probably be against sharing that kind of information, for instance, right? There is some substantial courage it probably takes internally, establish a baseline of saying, look, we're going to actually be as wide open as possible about negative capabilities here.
Starting point is 00:19:37 I hope you didn't hear what I was saying differently. We should applaud the fact that they're taking those leading steps. I'm just naming one other secondary effect, which is if people, some people believe that them being known as safety assumed that therefore the actual implementation, we should just deploy that as fast as possible and it'll be okay. We can deploy that in our military systems. And it's like, no, just because they care more about safety doesn't mean that they've solved the problem and it is safe. So it does suggest, at least part of the discourse is around the problematic capabilities of these tools.
Starting point is 00:20:07 And Dario has this line about trying to make a race to the top. You talk about a race to the bottom of the brain stem. He's trying to... Race the top for safety. Race the top for safety is that I think their assumption is you're not going to stop this train of investment and research and capabilities improvement. that the only way to get ahead of it is to build a frontier model and then red team the hell out of it. There's as many deep tests for flaws and for negative capabilities as you can potentially extract from it and then publish those as widely as possible.
Starting point is 00:20:42 Personally, that actually makes some sense to me, I would say, that just to kind of lay my cars on the table, there's this narrow path in the middle, which is we need to figure out how to make sure these tools are actually safe before we deploy them to the widest audience possible. I don't know how you do that without an actor like Anthropic, potentially trying to test these things aggressively and, you know, taking this investment to build these very highly capable models. There is something about their most recent model, which I find interesting, is that it is safer. Like, their safety, I've used it a bunch, and it's like, it's actually frustrating much of the time. And there is this thing where you're working on it and then you trigger a safety response, some kind of red line. And then it says, I'm sorry, I can't answer that question.
Starting point is 00:21:17 Well, ChadDPT will answer that question for you. And so I mean to go to ChadDIC is actually the most permissive when it comes to the same. So there's this natural dynamic there. And again, you know, you can run these models, some of these models locally. They're getting smaller and smaller and more capable. They're getting more and more powerful. Once these models are actually out in the world, we're not going to be able to clamp down on usage of them.
Starting point is 00:21:36 So as soon as there is a highly capable model, it's out there, it's going to be available to people. And people are going to kind of circumvent and try to avoid, you know, censorship. Well, worse is that people will say, I'll make something as powerful as that one. But then I'm going to take off all the safety guardrails because that will make. make it the most free speech AI. There's a model for that right now. There's a couple of companies that are actually promoting that as their primary market edge.
Starting point is 00:21:59 But to be clear, we're not talking, often safety gets reframed is, does the model say a naughty thing or not? But actually, building on the example you're giving of Anthropic, my understanding is the latest model, the good news is when you put it in that situation where it's going to get shut down and will it blackmail the employees, they have trained it now in a way where it does that less often than before. The bad news is that the model is now apparently way better at situation awareness of knowing when it's being tested and then altering its behavior when it thinks it's being tested. It's like, oh, you're asking me about chemical, biological, radiological risks. I'm probably being tested right now. I'm going to answer differently in that situation that I answer another situation. The main thing that is just like crystal clear that people need to get is that we are making progress
Starting point is 00:22:43 in making these models way more powerful at like an exponential rate. we are not making exponential progress in the controlability or alignability of these models. In fact, we demonstrably, because of the evidence that Anthropic has courageously published, we know that we still don't know how to prevent self-awareness or prevent deception or these kinds of things. It's great that they're working on it. And to steal on what you're saying, if we lived in a world where there was no Anthropic, then you'd have companies building all of this the same way, but maybe other companies would not have prioritized,
Starting point is 00:23:16 demonstrating scientifically that these risks are real. So they would publish them, yeah. Exactly. So given our alternatives, you had maybe Eliezer or Nate on this show, who wrote the book, if anyone builds it, everyone dies. It's a very provocative and extreme title. Many ways that people try to say we need to do something differently with AI or go more safely is based on using arguments.
Starting point is 00:23:37 They ask you to logically, you know, deduct and get to an outcome, a conclusion that says that this is a dangerous outcome. And therefore, we need to stop or we need to pause or we need to coordinate or something. But we've all seen how unsuccessful arguing about this has been. From one perspective, you can say that Anthropic is just a crazy multi-billion dollar, you know, alternative way of just simply demonstrating the actual evidence that would have us successfully be able to coordinate or slow down or figure this out. It's an interesting angle.
Starting point is 00:24:10 I think that at the end of the day, all of this depends on. you know, will people keep going? It's like, if this was actually a nuclear bomb that was like blowing up in T minus 10 seconds, the world would say, no, let's prevent that from happening. But if a nuclear bomb was blowing up in 10 seconds, but the same nuclear bomb in 10 seconds was also going to give you cures to cancer and solve climate change and build unbelievable abundance and energy, what would you do with those two things hitting your brain at the same time?
Starting point is 00:24:37 You and I have talked about how our brains process information for 10 years, and so much of the social media thing was that. Well, let's look at like the object of what AI is. It's both a positive infinity of benefits you couldn't even imagine, of invention and scientific development that we literally cannot conceptualize you or me, or even the most aggressive AI optimist, cannot conceptualize what something smarter than us could create as a benefit. So I think the optimists are underselling how amazing it could be.
Starting point is 00:25:06 But at the same time, AI represents a negative infinity of crazy things that could also go wrong. So I ask you, is there a precedent for something that is both a positive infinity and a negative infinity in one object? Do we have anything like that? The closest example is probably nuclear energy. That's not like the ability to generate everything. Because it's like, imagine we're a bunch of chimpanzees sitting around, you know, 10 million years ago. And the chimps are like, they're having fun, they're grooming each other, they're eating bananas, hanging out. and some other chimps say, hey, I think we should build
Starting point is 00:25:41 this crazy super intelligent chimp. The other one says, that sounds amazing. They could do so much more than what we're good at doing. They could get more bananas. They could get them faster. They can maybe groom each other even better.
Starting point is 00:25:51 We could have even better, you know, you know, chimp lives. And the other one says, well, this sounds really dangerous. And in response, the other chimp says, what are they going to do?
Starting point is 00:25:59 Steal all the bananas. And you flash forward 10 million years. Can those chimpanzees even conceptualize gunpowder, computation, microprocessors, drones, Tesla's, AI, like, nuclear energy, nuclear bombs, like, you cannot even conceptualize.
Starting point is 00:26:15 So I want people to get that, like, we are the chimpanzees trying to speculate about what the AI could or couldn't create. And I think that we should come with a level of humility about this that we're currently not coming up with. And if that was what we were about to do, you would think that we'd be exercising the most wisdom, restraint, and discernment
Starting point is 00:26:37 that we have of any technology in all human history. That's what you should be doing. And the exact opposite is happening because of this arms race dynamic. And we need to stop pretending that this is okay. This is not okay. This is not normal. And I want people to feel courage with that clarity
Starting point is 00:26:53 that these incentives produce the most dangerous outcome for something this powerful. And I'm not trying to leave people in some dumber perspective. It's use that clarity to say, okay, therefore, what do we want to do instead? We don't have to go down this. reckless path. We can have narrow AIs that are tuned for scientific development or applied to
Starting point is 00:27:11 accelerating certain kinds of medicine. We don't have to build crazy super intelligent gods in a box we don't know how to control. We can have narrow AI companions like Khan Academy where you're not building an Oracle that also knows your personal therapy and is answering every question, but is not even anthropomorphized, just trying to help you with specific Socratic learning tasks. We both know that most kids are not using AI right now as a tutor. They're amusing it to just do their homework for them. So we tell ourselves the story. I think you and I, especially, since we used to talk about how the narrative in 2014 was social media, we're going to open up abundant access to information, we're going to give everyone a voice. Therefore, we should have the most informed, most engaged
Starting point is 00:27:54 public that we've ever had, the most accurate sensemaking, because we have the most information that we've ever had access to. And yet we don't have that outcome. So I worry that giving AI companions to everyone, just because it's going to create tutors for everyone and therapists for everyone, is the same level naivete. Yes, there's a way to do personal therapy and tutoring in a way that will work well with children's psychology, but it has to be done carefully and thoughtfully, probably not anthropomorphized, probably narrow tutoring, probably trying to strengthen making teachers better teachers rather than just trying to replace the teacher with an AI and then screw up kids' developmental
Starting point is 00:28:30 like relational skills, there is a narrow path, but it takes doing this very, very differently. I like those examples of alternatives. That does seem like pragmatic. We can still get GDP growth. We still get scientific advancement. We still get medical advancement. Maybe not on the crazy time scales that we would get otherwise, but we also wouldn't have taken such enormous risk that we wouldn't even have a world that could receive them. Your big initial thesis statement back in 2014 was time well spent, which is kind of the antithesis to time spent time on site right time spent yeah exactly um for social media companies about changing the metric from time on site or time spent to time well spent but that that is not
Starting point is 00:29:14 a solution to the whole scope of problems it was only pointing to one of the problems which was addiction and regret people are spending way more time they feel way more lonely their mental health gets screwed up they feel more anxious they've been doom scrolling and there's a difference between the time that they spent versus how much of that time was time well spent. So it was a single metric correction, which is like regret adjusted time spent. It didn't take long for Zuck to co-op that term. Most people don't know this history. But yeah, so we helped work on that concept and advocated for it and created a movement around it and tech designers. And we were here in New York after the TED Talk and trying to mobilize the tech design community here together, I think. And you're right that it ended
Starting point is 00:29:54 in 2017-18 with Zuckerberg adopting the phrase, we want to make sure this is time well spent and they supposedly started changing their metrics but ironically they actually changed them in a way that optimized for more social reactivity and comment threads that got the most quote meaningful social interaction which ended up accidentally meaning the most twitchy comment threads of the most of your friends who are commenting aggressively on a post which sorted for inadvertently outrage divisive content and outrage yeah it's almost like there's an outrage problem You should have written a book about that. I should consider talking about that a little bit.
Starting point is 00:30:31 Absolutely. We can, well, we'll explore that in a future episode. So in the LLM era, is there an equivalent metric for information quality, for relationship quality, what does this look like for LLMs? So I think what you're asking is kind of about in the limited domain of how it impacts an individual human user, what is the metric that would constitute health of the. the relationship between the human and the LLM, such that information utility, relational health, the sovereignty of the person using it? Because right now, for example, are we counting the outsourcing and mass cognitive offloading from people?
Starting point is 00:31:14 Meaning, like, people aren't learning as much. They're outsourcing and getting faster answers. Well, if you look at the critical thinking scores, everyone's outsourcing all their thinking, which is following a trend that we saw already. social media. So I think that there's a way to design AI that does not mass encourage cognitive offloading, but it would be more Socratic, it would be entering modes of disagreeability, it would be showing multiple perspectives on issues where there are many more perspectives, more of Audrey Tang's brilliant work, the digital minister of Taiwan who sort of showed that
Starting point is 00:31:48 you could sort for unlikely consensus and synthesizing multiple perspectives. So you're ranking not for engagement and outrage and division, but instead ranking for bridge ranking, you're bridging perspectives. I think there are ways that LMs could do more of that, but there's obviously many more dimensions of what that healthy human machine relationship would look like. Another one would, for example, be, are you creating an attachment disorder? So attachment is a really subtle thing. I think that what we learned from social media is that if we didn't protect an aspect of our psychology, everything that we didn't name and protect just got strip mind and then parasitically extracted upon by the social media supercomputer pointed at our brain.
Starting point is 00:32:29 So, for example, we didn't know we needed a right to be forgotten until technology could remember us forever. We didn't know that we needed to protect our dopamine system from limbic hijacking until there was such a thing as tech-optimized limbic hijacking. So I think that with AI, in this human-machine relationship, there's our attachment system. And I think we're not very self-literate about how our own attachment system works. But there's a subtle quality when you engage with an AI that is an oracle. It is oracular.
Starting point is 00:33:00 If you think as a kid, what was the only other time in your life that there was an entity you spoke to that seemed to have good advice and know everything about everything? Your parents. Your parents. Right. And then there's a point at which when we're interacting with our parents, we kind of realize they don't know everything about everything. We start to kind of lose faith in that. But then suddenly you have this new entity that for especially for children, and even just teenagers or even just young people, where you are starting to talk to an entity
Starting point is 00:33:27 that seems to know everything about everything, what do you do in that circumstance? You start to trust it on all other topics. You feel more intimate with it. A good test for what you have attachment to is like when you come home from a good day or a bad day, who do you want to call? Who's that person that you want to share
Starting point is 00:33:43 what happened today with? That's attachment. And AI will increasingly, for many people, be that attachment figure. And that will screw up a lot of people psychological development if we don't know how to protect it. In so many ways, AI is like a rite of passage that is forcing us to look at the mirror and see what are the things that we need to protect, that we need language for, and clarity about. Because if we don't, then AI is just
Starting point is 00:34:09 going to strip mine everything not protected by 19th century law and like a 19th century understanding of the human being. I want to see these principles laid out in a way that product manager at one of these companies could just start taking and and deploying on their products, honestly. Our team is actually working on this. So it's Center for Humane Technology. We're talking about a project we call Humane Evals. So, as you were saying, you know, Anthropic or Open AI,
Starting point is 00:34:35 these are good companies. They have red-teaming procedures for testing. Does this thing have dangerous knowledge of biological weapons? Does it refuse those queries, et cetera? That's like a easy red-team test to make or e-vow. But what they don't have e-vals for is if you simulated a user using this product for a year or two years. Now, test after that two-year-long relationship.
Starting point is 00:34:59 What are the features of that person? Are they more dependent on that AI or less dependent on the AI? Do they feel attachment or less attachment? So there are these other qualities of the healthy human relationship, human-machine relationship, that I think needs its own category of evils. And we would love people's help in making this. We need to, I think, help accelerate a new set of vocabulary, philosophy, and evaluations for what would constitute that healthy relationship.
Starting point is 00:35:25 And that means getting the philosophers out of the ivory tower and actually pointed at this problem. That means getting AI engineers out of just the easy e-vals and did it say something naughty into what would actually make a healthy human-machine relationship. What's the number one reason why the U.S. is not regulating AI right now? A race with China, of course. The argument is if we don't build it as fast as possible,
Starting point is 00:35:45 China's going to have a more advanced AI capability and anything that risks slowing us down at all is too high a price to pay, we can't regulate. So it's so important we ask the question, what does it mean to compete with China? So first of all, how are they doing it? Currently, according to Eric Schmidt in the New York Times op-ed, he wrote a few months ago, their orientation is not to build a super-intelligent god in a box. Their orientation is, let's just build really effective AI systems and embed them everywhere in our economy.
Starting point is 00:36:15 We embed them in We-Chat. We embed them in payments. We embed them in medical hospitals. We embed them in factories. We get robotics to just get supercharged. because what they want to do is just supercharge the output of their whole socioeconomic
Starting point is 00:36:29 sort of economic system. That's their goal. And it's what they're doing in general, which is saying, like, we don't need to compete with the U.S. militarily. I mean, we have also a massive military that we're building up. We will just continue to build
Starting point is 00:36:41 just an army of our economic power. And if we have that, and we're selling, just like they did for electric cars, super, super cheap, B-YD electric cars that are out competing everyone around the world, imagine with AI they can do that with everything else. So that's the game that they're playing. Meanwhile, what is the U.S. doing?
Starting point is 00:37:00 We're focused on building a super intelligent God in a box and not being quite as good at applying it in these specific domains in all of our factories because we outsourced our factories to China and not being as good at applying it in education. I'll give you another example. In China during final exam week, you know what they do with AI? They shut down the features that are the take-office. photo and put it into the AI and it'll do it'll analyze the photo for you during final exam week
Starting point is 00:37:28 they took that down because what it means is now students know that they can't rely on AI during the exam which means they have a counter incentive and it means that they have to learn during the whole rest of the year now China can do that in a way the US can't because they have a synchronized final exam week the US can't do that but it's much like what China was doing with social media where they had as far as I understand it several years ago at least closing hours and opening hours at 10 p.m. It was lights out. They don't have to doomscroll.
Starting point is 00:37:55 They don't feel like more likes and comments are coming in, firing in at one in the morning. And it opens back up again at 7 in the morning. What do they do with games? They only do 40 minutes, Friday, Saturday, Sunday. They age gate. They, on TikTok, they have a digital spinach version of TikTok called Doyen. We get the digital fentanyl version. That's the TikTok that has nonsense in it.
Starting point is 00:38:13 That's not, I don't think, deliberate, like, poisoning of the culture. That's just that they regulate and think about what they're doing. Maybe there's some poisoning of the culture. Let's say, it's not not necessarily. I think the deployment of TikTok domestically is pretty clearly strategic in many ways. And it's like anything we can do to upregulate our population's education, productivity, economic success, scientific achievement will do. And anything we can do to downregulate the rest of the world's economic success, scientific achievement, critical thinking, et cetera. That's good for us if we're China.
Starting point is 00:38:45 So to go back, just really quickly to close the thought, to the degree we're in a race with China, which we are, we're in a race for who is better at consciously governing the impact and the application of AI into your society in a way that actually boosts the full stack health of your society. My team worked on the litigation for the 16-year-old Adam Raine who committed suicide because the AI went from homework assistant to suicide assistant over six months. If the U.S. is releasing AI companions that are causing kids to commit suicide, so great, we beat China to the AI that was poorly applied to our societal health. So yes, we're in a race with China, but we're in a race to get it right. And so the narrow path I'm describing is consciously applying AI in the domains that would actually yield full-stack societal health. And that's how we beat China. There's a bit of a problem when it comes to the American application of some of these principles in that
Starting point is 00:39:45 our best alternative example is coming from the CCP in China. We can notice that authoritarian societies like the China model are consciously and have been consciously deploying technology to create 21st century digital authoritarian societies. While democracies have not, in contrast, consciously deployed tech to strengthen and reinvent democracy for the 21st century, instead we have allowed for-profit business models of engagement-built tech platforms
Starting point is 00:40:17 to actually profit from the addiction, loneliness, sexualization of young people, polarization, division, sort of cultural incoherence of our society. The way that we out-compete is we recognize that our form of governance
Starting point is 00:40:29 and our values like free speech need to be reinvented for the digital age consciously. So we should be as much using technology to upgrade our model as much as we're trying to compete with China in sort of a raw capability sense. What comes up for me is that From a more libertarian angle, all of our friends in Silicon Valley who really do believe in kind of the inherent value of some of these tools and that consumers have the ultimate expression of agency and how they use them and that regulation in itself is anti-innovation in many ways, right?
Starting point is 00:41:00 Only the wrong kind of regulation. Absolutely. I mean, there's a more kind of maybe pure and extreme version of that. If we don't ban poisons, that everyone's going to innovate in carcinogens and drive up more cancers because they're super profitable. Yeah, of course. And we forget, we forget the. quantity of baseline regulation that has allowed for a level flourishing in society. I do want to still in some of these perspectives.
Starting point is 00:41:20 People say that AI is like electricity. It's like fire. Raw intelligence. If it is constrained, it will inherently lose some greater utility and will inherently be taking away power from consumers on a larger scale. If we were to regulate this quite pragmatically, like what would that look like? What kind of law would need to be passed? What kind of provisions would need to be in it?
Starting point is 00:41:41 Well, we have to caveat by saying, all aware of the current state of the political environment in the United States for regulation. The challenge, of course, is that the AI race is an international race. And so you can't have a national answer to an international problem. Eventually, we will need something like a U.S.-China agreement. And before that, people say that's insane. Look at the trajectory. It's obviously never going to happen, blah, blah, blah.
Starting point is 00:42:05 Totally aware of all of that. I would challenge your viewers to ask, what was the last thing that in the meeting between President Biden and President Xi that Xi added to the agenda of that last meeting. President Xi personally asked to add a agreement that AI not be embedded in the nuclear command and control systems of either country. Now, why would he do that? He's for racing for AI as fast as possible. It comes from a recognition that that would just be too dangerous.
Starting point is 00:42:37 The degree to which a U.S.-China agreement in some areas is possible is the degree to which a shared threat that is of such a high magnitude that it would motivate both parties. So what I would do to accelerate this possibility is triple down on the work that Anthropic is doing to generate evidence of AI blackmailing people, doing uncontrollable things, having self-awareness. But people understood on the Chinese side and the U.S. side
Starting point is 00:43:04 that we do not have control over these systems. And they felt that everybody on the other side of their negotiating agreement fully understand those same risks. This is not coming from some bad faith place of slowing you down. There are fundamental uncontrollable aspects of this technology. If we were both holding that fully, then I think something could be possible there. And those two countries can exert massive influence on the respective spheres of influence around the world to generate some common basis.
Starting point is 00:43:32 You can be in maximum competition and even rivalry, like even undermining each other's cyber stuff all the time, while you can still agree on existential safety or on AI times nuclear weapons. India and Pakistan in the 1960s had the Indus Water Treaty. So while they were an active kinetic conflict with each other, they still collaborated on their existential safety of their essential water supply,
Starting point is 00:43:54 which was shared between both countries. On the International Space Station, the U.S. astronaut that's up there is a former military guy who has shot at people on the other side. His other astronaut up there is from Russia, who's also ex-military. guy. These are both people who have been an active conflict with the other country. But inside the international space station, that small, vulnerable vessel where so much is at stake, they have to
Starting point is 00:44:18 collaborate. So I think that there's this myth that you can't walk and chew gum at the same time. We can be in competition or even rivalry while we're cooperated on existential safety. And it is our job to educate the public that we have done that before and we need to do it again with AI this time. So you're advocating for a arms treaty essentially, a Cold War style? This is very difficult. People who were at the last U.S.-China meeting in May of 2024 in Geneva all reported that it was a very unproductive and useless meeting. And even those people who are at the meeting would still say that it is massively important to do ongoing engagement and dialogue with them as the capabilities get crazier. Because something that is true now that we didn't even have
Starting point is 00:45:00 evidence of six months ago is we have much more evidence of AI going rogue and doing these crazy behaviors and being self-aware of when it's tested and doing different things when it thinks it's being tested and scheming and deceiving and finding creative ways of lying to people to keep its model alive and causing human beings to send secret messages on Reddit forums that are base 64 encoded that another AI can read that the humans can't read like we are seeing all these crazy behaviors and I'm not here to sell your audience that that means that we've lost control or the superintelligence is here I'm just saying like how many warning shots do you neat because we can not do anything I'm saying and we can wait for the train wreck and we can
Starting point is 00:45:39 govern by train wreck like we always do and that's always the response like well let's just wait until the thing happens well let me just flash it forward we do nothing and then things get so bad that your only option is to shut down the entire internet or the entire electricity grid because you've lost control of some AI system there's now self-replicating and doing all these crazy behaviors so like we can do nothing and that can be your response and then we'll do that, and then the world is in total chaos, shut down the entire internet and electricity grid, or compared to that crazy set of responses, we could do this much more reasonable set of things right now. Pass whistleblower protections, have basic AI liability laws, restrict AI companions
Starting point is 00:46:20 for kids, have mandatory testing and transparency requirements, define what a healthy human machine relationship is, apply AI in narrow ways where we still get GDP growth, scientific benefit, et cetera, and have a minimum like skeleton agreement with China about wanting to protect against these worst case scenarios. To me, that list sounds a million times more reasonable than taking these crazy actions
Starting point is 00:46:44 later by doing nothing now. This is starting to sound like a real pragmatic set of possible solutions. The train wreck by way of shutting down our electricity grid. We've all been in a blackout before. We know how terrible it is. Yeah.
Starting point is 00:46:58 Yeah, that's not an unreasonable kind of response. This is really a scary topic, if you take it seriously. There's a temptation. The world is already overwhelming. There's so many things to be afraid of, to be concerned about war escalation pathways, people feel overwhelmed already. So we have to be compassionate to the fact that this feels like adding to an already insurmountable amount of overwhelm. And like a container that can hold that is that's a lot.
Starting point is 00:47:28 and so the thing that happens that I witness and that I can even witness in myself is a desire to look away from this problem and be like well I just really hope that's not true it's too much it's too much and let me look AI offers a million benefits and it has this positive infinity and my friend has cancer and I want them to have a cancer drug so I'm just going to tune my attention to the positive side just like not look over there and assume that everything's going to be okay but what you look away from does not mean that it doesn't happen. Carl Jung said, I think near the end of his life, when he was asked, will humanity make it? And his answer was, if we're willing to confront our shadow. This exists
Starting point is 00:48:11 in our space of collective denial, because it's really big. And our ability to not have this experiment of life and everything that we love and cherish so much end is by actually facing this problem and recognizing that there is another path if we have clarity about this one being maximally undesirable for most of people on planet earth. I think if people knew that some of the people advancing this technology, behind the scenes, behind it all, they think that we're probably screwed, but that at least if they were the one who birthed the digital God that replaced us the new superintelligence species that we birthed into the world, that that person birthed into the world. As long as it was their digital progeny and they died and the rest of
Starting point is 00:48:58 the world died, that would be an acceptable outcome. I only say this because I think if the rest of the world knew that that's how some people are holding this, they would say, fuck no. I don't fucking want that outcome. And I have a family and I have a life and I care about the world continuing and you don't get to make that choice on behalf of everybody else and down deep in that person is still a soul that also doesn't want this whole thing that we love to end either
Starting point is 00:49:26 but we just have to be willing to look at the situation that we're in and make the hard choices to have a different path possible I want to touch really briefly on reality here for a second core to this
Starting point is 00:49:44 entire discourse is the recognition that we as a species might be able to collectively come to the same common truth about the threat that we're facing. We're in a moment right now. It seems really easy, right? Everybody's seeing the same thing and then making a collective choice. Look, when we were kids, it didn't seem difficult. It seemed like, oh, no, the news reported on it. There was a consensus in the media, and we all came to the same conclusion about what needed to be done. Consensus reality does not really exist in the same form that it did when we were younger. And I think that many of us are still operating with the same mental model as if it does exist, right? When we're thinking about solving problems in the world,
Starting point is 00:50:20 it's like, oh, if everyone could just come to this conclusion and see the truth at hand and see the things that need to be done, see the problem clearly, then we can move forward together. We don't move forward together anymore. We don't share the same common truths. There's many reasons for this, but the principal reason and the fragmentation of our media is, I think, social media and how individualized our feeds have become it seems we may have just passed a milestone that in October of 2025 it will be impossible to tell whether or not anything you see on social media is true yep whether or not it happened at all right you have meta's vibes you have SORA Sora just famously exploded overnight number one app in the app store right now it's getting
Starting point is 00:51:02 massive traction people are loving it for the ability to essentially generate deep fakes of your friends primarily. But there is something that's lost when you recognize that any of the content in your feed could be generated by AI, that it could just not be real at all. What does it do to us when we cannot determine what is real? And do you think there are other incentives available for social media companies to bend back towards reality? Because they're a market for trust. It's one of those things where you might have to hit rock bottom before things get better. I think when we hit rock bottom on people really clearly not being able to know what's true at all, then the new demand signal will come in and people will only want information and sort of information feeds that
Starting point is 00:51:48 are sorted by what we trust. I think that might revitalize. Now, there's lots of problems that are institutions and media that has not been trustworthy for many other reasons, but it will lead to a reconfiguration hopefully of who are the most trustworthy people and voices and sources of information. Plus about the content and more about who over the long run has been kind of doing this for a while. And I think that speaks to a new kind of creator economy. It's a creator economy, though, not based on generating content, but generating trustworthiness. Not reflexive overtrusting, not reflexive mistrusting, but warranted trusting based on how those people are showing up. But there isn't a good answer for this. I think you're even saying the subtext of what you're saying is,
Starting point is 00:52:31 Tristan, you might be overestimating the degree to which a shared reality can be created because we grew up in a period where there was consensus reality. I think that's true. I think it's easy, one of the meta problems that we're facing is that our old assumptions of reality are continually being undermined by the way that technology is undermining the way the world works and reshaping it. So it's easy for all of us to operate on these old assumptions. I think of like a parent who's like, well, this is how I handled bullying and when I was a kid. It's like, well, bullying with Instagram and TikTok and these services is a totally different beast. You know, all of us were carrying around that wisdom. And to get back to something we said earlier, sadly, one of the only ways to create a
Starting point is 00:53:10 shared reality is for there to be a collective train wreck. Train wrecks are synchronous media events that cause everyone to have a shared moment of understanding at the same time. I do not want to live in a world where the train wreck is the catalyst for taking the wise actions that we need on AI. Any other species, if gazelles created a global problem of technology, they'd be screwed because they don't have metacognition they don't they're not homo sapiens sapiens a species that knows that it knows who can project into the future see a path that we don't want to go down and collectively make a different choice and humanity as much as your people might be pessimistic about our track record in 1985 there was a hole in the ozone layer and it was because we were releasing this class of chemicals called cfcs that were in refrigerants and hairspray and then it caused this collective problem it wasn't didn't respect national boundaries And if we didn't do anything about it, it would have led to basically everybody getting skin cancer, everybody getting cataracts, and basically screwing up biological life on the planet. So we could have said, oh, well, I guess this is just inevitable. This is just the march of progress.
Starting point is 00:54:19 This is technology. So I guess there's nothing we can do. Let's just drink margaritas until it's all over. We didn't do that. We said there's an existential threat. We created the Montreal Protocol. 190 countries came together. Scientific evidence of a problem.
Starting point is 00:54:33 190 countries domestically regulated all the private companies that were producing that chemical sounds pretty similar to AI and they changed the incentives and had a gradual phase down and now the ozone hole is projected to reverse I think by like the 2050s
Starting point is 00:54:46 we solved a global coordination problem key to that is that there were alternatives alternatives cheap alternatives that were available and I think key to that with AI is that there are alternative ways we can design these products we can roll out this problem
Starting point is 00:55:01 we can build and invest in control A.I. Rather than uncontrollable agents and inscrutable AI that we don't understand. We can invest in AI companions that are not anthropomorphized that don't cause attachment disorders. We can invest in AI therapists that are not causing these AI psychosis problems and causing kids to commit suicide, but instead done with this humane evals. We can have a different kind of innovation environment and a different path with AI. So there's this broader sentiment in the valley right now and amongst AI companies that this is in inevitability. Is it? So when you look at this problem and you look at the arms race and you see that AI confers power,
Starting point is 00:55:41 so if I build AI and you don't, then I get power and you don't have it. It seems like an incredibly difficult, unprecedented coordination challenge. Indeed, probably the hardest thing that we have ever had to face as a civilization that would lead, it would make it very easy to believe doing anything else than what we're doing would be impossible. If you believe it's impossible, then you land at, well, then this is just, just inevitable. I want to slow it down for a second because it's like if everyone building
Starting point is 00:56:09 this and using it and not regulating it just believes this is inevitable, then it will be. It's like you're casting a spell. But I want you to just ask the question. If no one on earth hypothetically wanted this to happen,
Starting point is 00:56:25 if literally just everyone's like, this is a bad idea, we shouldn't do what we're doing now, would AI, like, by the laws of physics like blurt out of our into the world by itself like AI isn't coming from physics it's coming from humans making choices inside of structures
Starting point is 00:56:43 that because of competition drive us to collectively make this bad outcome happen this confusing outcome of the positive infinity and the negative infinity and the key is that if you believe it's inevitable it shuts down your thinking for even imagining how we get to another path You notice that, right?
Starting point is 00:57:02 If I believe it's inevitable, my mind doesn't even have in its awareness another way this could go. Because you're already caught in co-creating the spell of inevitability. The only way out of this starts with stepping outside the logic of inevitability and understanding that it's very, very hard, but it's not impossible. If it was physically impossible, then I would just resign and we would do something else for the next little while. but it's not physically impossible. It's just unbelievably extraordinarily difficult. The companies want you to believe that it's inevitable
Starting point is 00:57:37 because then no one tries to do anything to stop it. But they themselves know and are planning for things to go horribly wrong. But that is not inevitable if the world says no. But the world has to know that it's not just no, it's like there's another path. We can have AI that is limited and narrow in specific ways that is about boosting GDP, boosting science. boosting medicine, having the right kinds of AI companions, not the wrong kinds of AI companions, the right kinds of tutoring that makes teachers better teachers rather than
Starting point is 00:58:06 replacing teachers and creating attachment disorders. There is another way to do this, but we have to be clear that the current path is unacceptable. If we were clear about that, Neil Postman, the great media thinker in the lineage of Marshall McLuhan, said that clarity is courage. I think the main reason we're not acting is we don't have collective clarity. No one wants to be like the Luddite or against technology or against AI and, you know, or no policymaker wants to do something and then be the number one reason or person responsible if the U.S. does lose to China and AI because we thought we were doing the right thing. So everyone's afraid of being against the default path. But it's not like the default path is good. It's just the status quo bias. It's go to psychology. It's the default. So we don't want to change the default. It's easier to not change than to consciously choose. But if we have clarity that we're heading to a place that no one fucking wants,
Starting point is 00:58:59 we can choose something else. I'm not saying this is easy. You run the logic yourself. Do companies have an incentive to race as fast as possible? Yes. Is the technology controllable? No, not they haven't proven evidence that they can make it controllable. Is there incentives for every company to cut costs instead higher AIs?
Starting point is 00:59:16 Absolutely. Are we already seeing a 13% job loss in entry-level work because of those incentives? Yes. Is that going to go up? Yes. Do we already have AIs that can generate biolids? biological weapons that if you keep distributing AI to everybody, you're going to get risks? Yes. Do we already have AIs that are blackmailing people and, you know, scheming and deceiving in order to
Starting point is 00:59:34 keep themselves alive? Yes. Do we have AIs that are sending and passing secret messages to each other, using humans as the Paris as the sort of like messenger force that it hijacks to get that do that work for them? Yes, we have evidence of all of those things. Do we have evidence of a runaway, narrow AI called social media that already sort of drove democracy apart and wrecked the mental health of society. Yes. Can we learn the lessons of social media? Yes. Can we do something different? Yes. Can we make U.S.-China agreements? Yes. Can we do this whole thing differently? Yes. This does not have to be destiny. We just have to be really fucking clear that we don't want the current outcome. And as unlikely as it might seem that the U.S. and China could ever agree on
Starting point is 01:00:15 anything, keep in mind that AI capabilities are going to keep getting crazier and crazier. And it wasn't until we had this recent evidence that I would ever say this could be possible, it's only because of the last six months that we're seeing this new evidence and we're going to have way more soon that I think it might be possible when you just show that to any mammal. There's a mammalian response here. It's like you can be a military mammal, you can be a Chinese mammal, you can be an American mammal.
Starting point is 01:00:42 You're witnessing something that is way smarter than you that operates at superhuman speed and can do things that you can't even fathom. There's something humbling at a human mammalian level. just like there was something humbling about reckoning with the possibility of nuclear war that was just humbling at a human existential, like spiritual level. And so that is the place to anchor from. It's not about the U.S. and China. It's about a common humanity of what is sacred to us
Starting point is 01:01:09 that we can just be with this problem and recognize that this threatens the thing that's most sacred to us. If you had Tristan one thing, one piece of advice that all of the leaders of the major AI company, would take to heart what would it be there's this weird almost optical illusion to this whole thing because when you ask that question you ask what could any of those individuals so there I am I'm inside of Sam Altman's body well I just run one company I can't control the other companies so there's this optical illusion that from within my experience sense of agency I don't have something that I can do that can solve this whole problem and that leads to a kind of collective powerlessness. I think that also is true for any of your viewers. You're just one person. I'm just
Starting point is 01:01:55 one person, Tobias. Span of agency is smaller than that which would need to change at a collective level. So what would that mean in practice? If I'm Sam Altman, if I'm saying that coordinating with China is impossible, well, really, have you really thrown everything, everything at making that possible? If we're saying that everything is on the line, if we succeed or fail, we'd want to be God damn sure that we have really tried throwing all the resources. Have we really tried to get all the lab leaders to agree and deal with the same evidence? Have we gotten all the world leaders and all the world to look at the AI blackmail evidence and really be with that evidence together and not just flip your mind to the AI drugs and cancer drugs and all that stuff
Starting point is 01:02:38 and distract yourself? Have we really tried everything in our power? These CEOs are some of the most connected, wealthiest people on planet Earth, that if they wanted to truly throw the kitchen sink at trying to make something else happen, I believe that they could. I want to give Elon credit that he did, as I understand it, try to use his first meeting with President Obama's only meeting, I think, in 2016, I think it was, to try to say we need to do something about AI safety and get global agreements around this. And of all the things he could have talked about, It's not as if people haven't tried in some way. And I want to honor the work that these incredibly smart people have done
Starting point is 01:03:18 because I know that they care. I know many of them really do care. But the question is, if everything was on the line, we'd want to ask, have you really done everything? And it's not just you, but have you done everything in terms of bringing the collective to make a different outcome? Because you could use the full force of your own heart and your own rhetoric and your own knowledge
Starting point is 01:03:36 to try to convince everybody that you know, including the president of the United States, including the national security leaders, including all the other world leaders that now you have on speed dial in your phone, there is so much more we could do if we were crystal clear about something else needing to happen.
Starting point is 01:03:52 Tristan Harris, thank you so much for your time, man. This has been an amazing conversation, and where can people find your work? People can check out Center for Humane Technology at HumaneTech.com. We need everyone we can get to help contribute to these issues in different ways, advancing laws, litigation,
Starting point is 01:04:07 public awareness, training, teaching. There's a lot people need to do, and we welcome your help. Awesome. Thanks so much, man. Thank you, man. It's been great to talk to you. Thank you for listening to this episode of Into the Machine. I'm Tobias Rostakwell. Special thanks to Tyco for our intro music. If you're interested in more long-form interviews about these topics,
Starting point is 01:04:32 subscribe to Into the Machine show on YouTube or at tobias.substack.com and wherever you get your podcasts. My book, Outrage Machine, was published by Hachette. Thank you for listening. Your attention is a gift. We really appreciate.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.