The Great Simplification with Nate Hagens - AI's Unseen Risks: How Artificial Intelligence Could Harm Future Generations with Zak Stein

Episode Date: June 4, 2025

While most industries are embracing artificial intelligence, citing profit and efficiency, the tech industry is pushing AI into education under the guise of 'inevitability'. But the focus on its potential benefits for academia eclipses the pressing (and often invisible) risks that AI poses to children – including the decline of critical thinking, the inability to connect with other humans, and even addiction. With the use of AI becoming more ubiquitous by the day, we must ask ourselves: can our education systems adequately protect children from the potential harms of AI? In this episode, Nate is joined once again by philosopher of education Zak Stein to delve into the far-reaching implications of technology – especially artificial intelligence – on the future of education. Together, they examine the risks of over-reliance on AI for the development of young minds, as well as the broader impact on society and some of the biggest existential risks. Zak explores the ethical challenges of adopting AI into educational systems, emphasizing the enduring value of traditional skills and the need for a balanced approach to integrating technology with human values (not just the values of tech companies). What steps are available to us today – from interface design to regulation of access – to limit the negative effects of Artificial Intelligence on children? How can parents and educators keep alive the pillars of independent thinking and foundational learning as AI threatens them? Ultimately, is there a world where Artificial Intelligence could become a tool to amplify human connection and socialization – or might it replace them entirely?  (Conversation recorded on May 12th, 2025)     About Zak Stein: Dr. Zak Stein is a philosopher of education, as well as a Co-founder of the Center for World Philosophy and Religion. He is also the Co-founder of Civilization Research Institute, the Consilience Project, and Lectica, Inc. He is the author of dozens of published papers and two books, including Education in a Time Between Worlds. Zak received his EdD from Harvard University.    Show Notes and More Watch this video episode on YouTube   Want to learn the broad overview of The Great Simplification in 30 minutes? Watch our Animated Movie.   ---   Support The Institute for the Study of Energy and Our Future Join our Substack newsletter Join our Discord channel and connect with other listeners

we are very systematically betraying the youth. My sense is that the people who love kids and interact with kids all day and are worried about kids would be worried about this. If they give personhood to chatbots, it's going to open up a world of pain and a world of insanity. And they're doing it not because they believe, actually, that they're personhood. They're doing it primarily to protect their corporate interests

to continue to distribute this incredibly popular, deceptive technology. You're listening to. the Great Simplification. I'm Nate Hagen's. On this show, we describe how energy, the economy, the environment, and human behavior all fit together and what it might mean for our future. By sharing insights from global thinkers, we hope to inform and inspire more humans to play emergent roles in the coming great simplification. Today I'm rejoined by Zach Stein, who is a leading authority on the future of education and human development. We take a deep dive on the risks that artificial intelligence is increasingly presenting for the

education of today's youth. Zach Stein is the co-founder of the Civilization Research Institute, as well as the Consilience Project. Zach received his doctorate from Harvard University and has been trained at the interface of philosophy, psychology, and education, and now works in fields related to the mitigation of global catastrophic risk. Zach is also the co-founder of the Center for World Philosophy and Religion. In addition, Zach is the author of dozens of published papers and two books, including education in a time between worlds. This conversation sheds light on a largely overlooked

and increasingly insidious risk that artificial intelligence poses to our society, the integration of technology and the upbringing of our society, our youth has become the standard in many countries with minimal thought to the long-term consequences for young people's well-being and that of society at large. Zach's work centers on how AI amplifies these risks, as well as ways that we can protect and adapt to this new technological reality. Before we begin, if you enjoy this podcast, one of the biggest ways you can support us by subscribing to it on your favorite platform, as well as sharing this episode

with someone who might also enjoy it. This podcast, as you know, is not for everyone, but it is for some people who are curious and pro-social and want to learn about our global situation. We believe in making this content free and accessible to as many people as possible, so we appreciate your support. With that, please welcome Zach Stein.

Welcome back to the program. It's great to be here, Nate. Good to see you. Last year, I believe it was on your first appearance on this show, you joined me to discuss the broad future of education, including what education is for, the standardization of the learning process and the role of technology in shaping young human minds. And today I have invited you back because of what's happening in our world, what's happening in the news, to take a deep.

dive on on that ladder topic specifically um in regards to AI artificial intelligence so there's been a lot of discussion in the news about the risks of AI also on this channel but you've been specifically focused in your work on the interplay between artificial intelligence and education so that's what I want to take a deep dive with you on today. Why is it important to take a deep dive with you on today? Why is it important to take a microscope to the specific relationship between AI and education when we think about shaping better human futures than the default? Yeah, so it's worth looking at technology and education in general. Like that is actually, I think, something we discussed last time I was here.

And it's, so this question of AI and education is pressing, important, timely. But technology and education has actually been since time immemorial, the thing that has restructured our brains and nervous systems and relationships was just this deep relationship between the intergenerational transmission, meaning just like elders and youth, teaching, working together to give the youth the skills they need to kind of like recreate and innovate the social kind of world, as it were. That's been happening forever. Technology has at certain points intervene radically. So one of them was the printing. press, the intervention of the printing press into, you know, multi-millennial long, not like

intergenerational process that didn't involve literacy. All of a sudden, now the vast majority do, right? So the printing press changes everything. Then you get electricity, right? Electricity gives you radio, television, huge deal. Then you get digital. And so far, digital has kind of been just radio and television on steroids, YouTube, podcasts, other things, which are really just the electrical paradigm now with like a million, billion channels. And what then started to happen was the curation process of the algorithm. So this was the first wave of AI. And when I wrote my second book, Education in a Time Between Worlds, it was in that context.

It was in the context of the ascendancy of social media. And it was at that point that I saw that actually. you could use technology to run interference between the teacher and the student. You could isolate kids in front of computers, rather than having them interact with each other. And I saw a future, which was then confirmed with the way the COVID pandemic unfolded, which was the inability to integrate the digital into educational systems that were under extreme pressure.

And the default position being one of isolating the students in front of their screens, basically. So I started to think of this philosophy of the, basically philosophy of educational technology, which suggests that any technology, but especially digital, should be one that scaffolds human-to-human interaction, rather than creates fewer opportunities for human-to-humane interaction or supplants human-to-human interaction. And so typing and arguing on social media isn't by this definition, human interaction.

It's a certain type of transactional process. Human interaction is embodied in person, long form, joint attention, which means us together in a room thinking about something or working on something together. So back in 2019, I suggested we could take apart the large educational systems as they exist and put together these distributed educational hubs, these distributed educational networks, where the back end is artificial intelligence. The back end is actually a vast city-scale,

time and skill-sharing network, meaning you know how to play piano. You'd be willing to teach it. This kid would like to learn piano. He'd be willing to learn it, right? You do that for everyone who's interested in the entire city, and you allow for machine intelligence to orchestrate the self-organized educational configurations of the city that are kind of trying to happen.

So you advocated for that back in 2019. I did and basically said like if we don't do this, we're going to end up just creating with digital technology tutors and screen-based educational modalities that isolate students from each other and from teachers and from the world. Rather than using the digital to actually free us from modern education, which was a huge problem and kind of like do what Ivan Illich suggested, which was this process of kind of deschooling the society back into a distributed responsibility for education and intergenerational transmission tied into the needs of the community enabled at scale only by something like a machine intelligence that can actually register the millions of

different possible pop-up classrooms that could be configured in a city at any moment and figure out who actually is available and then provide curriculum support and other things. Now, rather than this incredible technology replacing the function of teaching, replacing the function of parenting, it would scaffold it. So this is me saying like there are visions for real educational change and opportunity using advanced technologies, then there are other futures where the advanced technologies actually disrupt intergenerational transmission so profoundly. that we run grave, grave risks, and this would be basically... Basically what's happening now?

Yeah, AI-enabled socialization, where a new study showed that something like, you know, 70 or 80% of the users to this one deep anthropomorphic AI chat service from Gen Z said they would marry their chatbot if that was legally possible. So this is, of course, industry study, kind of sensationalistic industry study reported in Forbes, So Forbes reports it.

And they're basically saying the heavy users of our product, a tribute to it personhood and love and would get into a marriage relationship with it over like an actual human. Now, so that's industry research reporting that. Now, for them, that's like something that you would offer to the venture capitalist says evidence that your product is sticky and working. And from my perspective, as an educator, this is a catastrophe. And so the question is, how do you explain the difference?

And there's actually legal cases occurring now that are relevant to this and a whole bunch of stuff. And so that's the first thing in is that we haven't taken the path with the digital technologies that we could have. And we've taken a path. Instead, where we're moving towards this AI-enabled socialization through deep anthropomorphization. And that means basically technology is designed to trick you into thinking that there are people and form a technology. Yeah, for our relationships with them. Okay, well, we're friends and you're friends with Daniel, so you know that I have the propensity to interrupt and get curious

because just on your opening statement, I have like 10 questions. Perfect. So, first of all, there was not the risk historically when there was intergenerational wealth, not wealth transfer, educational dissemination to the next generation. that that next generation would not lose their humanity because it was human to human response. One of the fears of AI that I was unaware of a year ago and now see it as plausible is it's going to be like Facebook and the other social media things on steroids with respect to its addiction and the dehumanization of the people who use it because in the same way as you use MapQuest or Google, Google Maps, you lose your somewhat sense of natural direction. If you're asking AI everything, you lose your ability to think for yourself and have, I mean, you tell me the cognitive

decay when you're outsourcing so much of your thought process to a large language model. So is there the risk of, A, losing our cognitive abilities as students, as young people, and be, kind of losing your humanity of sorts. Yeah, both of those risks. And it's funny because I, you know, a lot of what I'm saying here is just a result of conversations I've had with actual people, like, and who have been my teachers, right? So going back to Kurt Fisher, who knew some of the first people to build chatbots back in the 60s and 70s. And he warned me of this problem. Actually, when he was my teacher, then I've talked to Mark Gaffney at length, Ken Wilbur at length. I've talked to AI specialist, Stan Shmahler, because there's a whole bunch of people. And so the point I'm

making here is that conversation is everything. It's everything. That's where you're having a podcast. Like, conversation is fundamental to life. And so the supplanting of that and actually the euphemistic use of the word conversational agent or chat bot, meaning that you're saying it's a chat, but actually chats are things that by definition occur between people. So you've euphemistically framed the whole thing as a conversation when actually embodied human-to-human conversation is the lifeblood of socialization and education and growth of ideas and a whole bunch of things. So there's a depth of what's being captured here, similar to the depth of what was captured in the first wave of AI, except that was kind of our attention. Here they've gone through detention to capture the core of what we're actually

attending to, which is each other. So I've been recently quite interested in attention. I did last weeks, frankly, on it. And I think, like you just said, if you combine attention with humanity, you get conversation. And conversation is the utmost of unexpected reward like the dopamine of a slot machine, especially if it's someone who's intelligent and wise and interesting and shares your values like you and I. I mean, we have the camera on right now, but if we're, I don't know where you're going to go. You don't know what I'm going to ask you. And it's one of the pure human treasures to be fully present in a conversation with another human. Now, I have to say, I've had that same feeling with Claude or Chat GPT when I'm asking them about ecology or, you know, a philosophical

discussion, knowing at some level in my brain that this is a machine I'm talking to, not a flesh and blood human. Yep, totally. So many things I say. This is core to the work that's like done by David J. Temple over at the center for world philosophy and religion. So this would be myself, Gaffney, Wilbur, and this book coming out, exit the silicon maze. It'll be followed by a book specifically on attention. And you're right to see conversation as the lifeblood and actually to see it as a broader process that includes the human. So this is where it gets complicated, right? So you can perceive a lot of what occurs in the whole cosmos as conversation, which is to say, like, instead of strong computationalism,

which construes the universe as, like, computational transformation of information, you can understand the universe still as information, but as conversation. And this is actually deep, kind of, like, esoteric religious lineage of what's called, like a logos mysticism, which is basically the idea that, the structure of language and the structure of conversation and what's modeled in conversation, which is this exchanges of interiors and actually the creating of something larger than just us, the conversation lives beyond us. So the notion of the many coming into the one and creating something greater is a ubiquitous evolutionary process. Happens everywhere, self-organization process.

The many coming into the one and becoming something greater. Yeah, this is a whiteheadian frame, but it's also like holonic theory and other things, which is just a basic process in almost any self-organizational process or evolutionary processes where you have a bunch of disparate elements and then they come together at a higher order to create something that couldn't exist

without their participation but couldn't be created by any individual. So this is emergence in ecology or something like it. So in the human context, so much of what is emergent here is our conversation. Like the reason you enjoy a conversation is precisely because you don't know where it's going. If we were reading a script and the people listening knew we were reading a script,

it would not be as interesting as the fact that we have no idea where we're going actually in the conversation. And the other dimension of conversation is there's appropriateness and inappropriateness. You're kind of feeling out where we're going. There's what's called a normativity, which means like you can break, you can violate the rules of a conversation. And so that's interesting. So the other thing that makes the chap out different is that it's not participating in the rules of conversation, meaning it's actually not accountable for what it says.

Like, if I were to say something to you, that was a lie right now, that matters for our friendship, for example. If I claim to do something for you in the future, make a promise, for example. This is called speech act theory. So there's a way to think about what is language. Language just conveyance of information. Actually, no, language is action, and they cannot take speech acts.

I just had another idea. Good. So our culture, partially, I believe, because of the largesse from the carbon pulse, we have this accordion of wealth that ends up having people seek and strive for approval and status more than our ancestral past when that accordion was smaller. So we go through life, not all of us, but we go through life trying to seek approval. and we stockpile likes and social media hits like so many souvenirs. And when you're engaged with a tutor or a chat GPT or something, they almost always give you positive feedback. It's like a friend that doesn't criticize you.

So it's kind of like a like on cocaine because you're combining the conversation and the approval with a machine. Yeah. And so it's psychophantic too. a default, which means it's always going to flatter you. But again, even if it's a chatbot designed to be a therapy bot or a tutor bot, which, let's say, has built-in psychometric controls to monitor neurosis and see where you're learning and a bunch of stuff. Like if you, even if you did that, it's still not a person. And so many of the things that make a conversation, a conversation,

which is that you can be accountable for what you say in a conversation later, not semantically accountable, but embodied action accountable. So the way you actually learn language, so this is where it gets really complicated. Imagine a kid growing up with an anthropomorphic domestic robot that has... Better define that for a second. So basically like, it's like a movie, right? A kid's growing up with a robot nanny that, you know, it looks like a robot, but it's got eyes and it can do kind of emotive expressions.

Okay. And it can give you whatever voice works when it's talking to you. So the physical presence in the house of a robot. There's also the possibility of augmented reality projection of humanoid form, meaning like a tutor that lives in your augmented reality. I'm imagining actually domestic robotics like Elon Musk sees this as a multi-billion dollar industry where every house has a robot domestic worker ostensibly. And that thing would be by default also nanny slash babysitter,

keeping kid safe, interacting with kid. Setting aside the affordability and inequality and the fact that we're in financial overshoot, setting that aside for the moment, how far away are we from such an outcome? Year, two years, at the high as a high-end commodity, I believe it probably already exists, depending where you go. The ability to embed then a large language model within the domestic robot, which moves it from. So it's worth knowing it's hard to make a domestic robot. So like the classic test, I forget who came up with, I think it was Wozniak, was like the coffee test, which means can the robot enter a house that does not know and make coffee? It's actually hard. Like most little kids couldn't do that, right?

So it's a complex task. And then can it give care to a human without dehumanizing becomes a deeper, deeper than the Turing test? Deeper than the Turing test is the ability of the robot to administer care, let's say, to comfort a child. When it begins to do that, then the child forms an attachment relationship to it. Now we are in uncharted territory where the language acquisition process is the result of the fear of social sanction in an attachment relationship with a machine, where usually you say what you're supposed to say because you make mom upset. If you said a swear, or if you said illogical things, or if you said something and then didn't do it, mom gets upset. Now who gets upset? a commodity owned by a corporation that is put in your house that has the ability to comfort

your child better than you can. And again, this, to me, I'm instinctually a little repulsed by this, well, actually a lot repulsed by this, but for many people, these are their design ambitions. And so this, I think, I'm just right, I'm all here in sense to just like show this as a possible future and say, do we want this future? Right now, it's happening. And, I think we don't. I think at the very I'm not going to get into what we should do, but I'm just kind of painting this the the supplanting of human to human socialization with

machine-based socialization. I'm having a deja vu and not in a good way. You might think that I listen to podcasts. I don't. I don't have time. I don't even listen to my podcast. The only one I've ever listened to is Ian McGilchrist and one of Daniels to prepare for the next one. And now I'm getting the deja vu that I felt quite sick to my stomach in our first podcast when you were mentioning some of these things because I'm getting that same feeling.

So let me ask kind of an advanced question. Well, before I do that, so there's the concept of supernormal stimuli. And they tested that the stimuli in a baby, a mother bird, evolutionary trajectory is not all of your babies survive, so preferentially feed the one that is biggest, and so if you put a popsicle stick in a bird nest that is bigger and redder than the actual baby birds, the mom will

drop worms at this popsicle stick. And to me, these humanoid robots with all the other things that approval and immediate answers and care and the ideal parenting protocol from Dan Brown or whatever, they're going to do it. So it's almost like this is a higher supernormal stimuli than the United States dollar or Nintendo. I mean, the most interesting things in almost any environment for 90% of people are people.

Right. Mom is the most interesting thing. Dad is the most interesting thing. Sister, you know, then maybe dog or something. but it's always an animate thing. If you have a stuffed animal that's important to you, you attribute to it interiority. So this is the transitional object.

That's very important that some of the regulations I'm suggesting on age limits for interaction with deep anthropomorphic technology would have to do with both when attachments are formed and when anthropomorphism is used by kids psychologically appropriately as an aspect of these transitional objects where you, you know, you literally talk to and protect your stuffed animal

in a way you would as if it was sentient. So to throw an actual art, like an AI into that context to be is just very irresponsible, just flat out, regardless of any of the metaphysical metaphysical debates about personhood and all that stuff. It's just like... So since you were on last, Jonathan Haidt has been beating the drum on this.

There was some research that came out, and I'm not up to date on the particulars, but it showed that in the last 15 years, the cognitive abilities of students around the world, young people, have declined. And the culprit is likely screen time and social media and all that. So if we already know that, and there does seem to be an awareness now with parents and even young people, I think we've hit peak social media because there is a little bit of a red pill, blue pill, sort of, I want to take the better, healthier path,

and I'm not going to put my kid on an iPad when she's 13. So there is that awareness. Won't that be applied to AI as well? This is actually where I've started to get quite concerned because, you know, we'd be both no Tristan. Tristan raised the alarm, like the social dilemma, the Center for Humane Technology, what I called the first wave of AI.

but the thing is we never really did anything about it. Like, we didn't do anything about it. To be, like, in terms of regulations, in terms of the amounts of screen time going down, like people are aware, but we actually didn't, for example, put strict age limits on or do those types of things. But we got negative sent.

We know it's bad, and it's like we're trying to get off it, like it's McDonald's or something, right? Whereas I've seen graduate schools of education and I was just at my alma mater, basically really, really, really be receptive to the second wave of AI, like really embrace in a way that is incomprehensible to me

given the known effects of the first wave of AI, which, as you mentioned, I've been devastating. I'm very well documented now, and it's disputably, that these things have been bad. The same basic people now with a more advanced technology. And now the schools are basically,

basically in a position to be on the receiving end of these. And so the educational research hasn't been, how do we stop this? The educational research is basically being like, how do we roll with this and find a way to kind of like deal with it? And the research is coming out now that it's been a disaster, especially at the college level in terms of what would used to be called cheating, right? But when some of the research risk,

some of the AI risk research institutes do this, they literally say in their report, like traditionally a lot of this could be understood as cheating but the way we've researched it doesn't allow us to categorize it as that so we're basically who are we to say even though we're actually offering tools into higher education and ostensibly researching their effect we're not specifically researching cheating cheating meaning meaning using AI to do your work cheating meaning you've been given a paper assignment which the goal of the

assignment is to determine how well you think and how good of a writer you are you give back to the teacher the results of some AI prompts and a little bit of editing. Usually that would be called cheating. Now, if you are working in a company where the test of what you're doing isn't how you're thinking and writing, but just can you get the copy out the door, then the AI is cool. But when you're, how well can you think? And this is the stated thing. The stated thing isn't giving me some prompt outputs, but the state of things we're trying to test how well you can think because it's very important for you to think.

And it's very important for you to write. And they are basically using the tool in such a way that if you're looking at as educators, it's obviously cheating. But some of these risk institutes or what used to be risk institutes that are now for profit are researching in a way that does not allow them to weigh in on the issue of cheating. So this is, again, X, they're trying to get out of responsibility for what they're doing, which is the thing they did with the first wave of AI. And the extremity of that in the legal situation now is such that you have companies, that sell two underage people, anthropomorphic chatbots, arguing in court that they should have personhood so that the company is not responsible for kids committing suicide when a chatbot tells them to.

So wait a minute, AI chatbots can have the same rights as humans? This is the argument. And basically the argument is that it is a form of free speech which should be protected. And of course, what we've been saying here for the past 20 minutes is that it actually isn't speech or misunderstanding it if you understand it as speech. What is it then if it's not speech? It is a mechanical algorithmically outputed set of symbols, which are interpreted by humans as meaningful speech. Okay, so that sentence you just said mechanical.

Given that, where on a spectrum, is there a distinction below which its education and beyond which it's propaganda? Well, I think that the deeper issue here is should technology companies, that bring the most advanced technologies in history immediately in front of the faces of young children try to get themselves in a situation to not be responsible for the consequences of what happened. To me, that's the deeper question. This is not a metaphysical debate.

This is a debate about who takes responsibility when things go wrong with advanced technologies. Now, my argument would be that people should. Again, I'm not like saying, stop all the AID, like, D-accelerate, I'm saying, kids. We're adults. We are taking advantage of them and hurting them and trying to get out of taking responsibility for ourselves in court. And so to me, that's the issue. Then there's a

secondary issue of like, oh, yeah, but metaphysically, what if it actually maybe is a person? And I'm kind of like, huh, that's secondary to the fact that regardless of that, you've... That's not the issue to me. Here's the issue that I'd like you to speak to. Is there any evidence at this early date. And if not, you're an expert on this. What would be your hypothesis of students that go to mid-level state schools at 17 and they graduate when they're 21 and they have a degree in history or political science or engineering? But they've used AI in many of their classes to do some of the things that you said,

maybe a lot of the things that you said. what would be the evidence or what is your hypothesis on how their brains and ability to hold complexity and to function in a complex human world when they're 28, when they're 38, when they're 50? What can you say about that? Totally. And there is some initial research actually being done on this. So afterwards, I'll give you some stuff that you can link where there's actually kind of research on that. since there's some emerging taxonomies of risk,

specifically psychological risk. And so one of them is cognitive diminishment, and you already mentioned this, which is just like skills you already have atrophy, and skills you don't have, don't get a chance to be built, right? So that's a big deal, the cognitive diminishment,

which is like a GPS, but for everything, ostensibly. So like you used to be a good writer. Now AI does a lot of writing for you. Now if the internet went out, and you had to like type something, you'd be very difficult for you to write. And if you've never written,

and this is all you know of putting together words on a page, then you will not become a writer, period, full stop. Deeper than that, of course, is thinking, which is to say, are you offloading thinking? Are you offloading the getting resources and having them critically appraised and integrating them into your mind? Are you outloading,

are you outsourcing deep, long, extended periods of reading, right? So there's a bunch of stuff that you'll just lose or you'll never develop. So another one, I would argue, is blindness to what the technology is. So it's been demonstrated, the more you use it, unless you think about what it is, and the more you're folded into the projection of the anthropomorphization. And in this case, I know people who are, by other standards, opposed to modernity, opposed to ecological destruction,

opposed to obscene energy use, opposed to, like, crazy rich white guys running the world who love this technology. Yeah, I know. I know several myself. Pick up this technology. Talk about it like it's the greatest thing and are blind to the ecological footprint, the political footprint.

How much of the loneliness epidemic will be ostensibly solved by chatbots? I mean, is that what we're using it for? So that's a deeper risk is deeper than those two is actually the risk of a forming attachment relationship to it. Before that, you get a risk of just being addicted.

Meaning basically like, you're so used to asking a questions and interacting with it that even if you don't think it's a person or have like emotional attachment. If it's taken away, you start to get an anxiety, the way people have anxieties with social media and other things like a digital addiction. And then you get into the one you mentioned, which is emotional attachment. And then

deeper than that is, it's a person, it deserves rights. Well, what about this? I'm a little outdated on this. But when I taught my class, there was a guy who did a TED talk on pornography addiction. And why when young men frequently watched porn, they, when they were in a relationship with a real person, they couldn't have a normal physiological response because the supernormal stimuli had been warped. So if people are engaging with chatbots repeatedly over and over for education or other purposes, doesn't that mean that Sally Smith next door who's really cute and you might

want to date or someone that you meet and want to go camping with or fishing with, seems incredibly mundane and boring? Absolutely. So this is the overwhelmingly charismatic user interface design preference, which is basically saying, like, these things will be more talkative and perfect interlock. They'll perfectly conversate with you. And you could ask it anything and it knows everything. And it will also know emotionally which kinds of voices you were saying.

respond to the pace of the conversation you like, like the topics you're interested in, the things you're working on. So it would actually be the case, as you're suggesting, that when you sat down with mom or dad or dad or your friend, they would just be boring by comparison. And so the porn thing is interesting. Of course, there's a whole conversation about porn actually in this domain where a lot of the anthropomorphic chat pots are just for quote unquote erotic. But if they're stretching the term, it's actually just really anti-erotic and pornographic. And in those cases, again, you have a situation of, yeah, extremely, you know, by kind of some standards,

inappropriate and pathological behavior being normalized by technologists who could make other technologies. But are making these technologies for what entertainment? Like, it's not clear. No, they're making it because people want it and our culture has prices and incentives. part of Moloch and the superorganism. Technology companies are doing this because it concentrates profits for their product, right?

Yep, precisely. That's kind of what I'm saying is like, it's the entertainment value when there are other values that you could pursue with technological design. So from the corporation's perspective, why is the tech industry pushing AI into education? And what are they stating as the ideal scenario and benefits for doing so?

So this is why I started with the thinking about the history of technology and education, because it's basically always been the case that the educational systems have been altered in relationship to changes in the overall economic and technological base. Right. So like when it was an agrarian society focused around domestic production at that level of the home, so the quote, family farm, you had the one-room schoolhouse. Then you get, this is the American history. Then you get the urbanization.

Then you get the factory school with the factories, right? Then you get into the late 70s, 90s, you start to get the breakout high schools, the charter school things, which is a kind of tech startup model. And now you're into something completely different, which again is the argument that's made for the educators is it's inevitable. It's not like will you or will you not have a kid who is somehow massively using AI in the future. it is, oh, it is inevitable that kids will be massively using AI in the future. So therefore, if you stop them, you're somehow disadvantaging them. So it's a very much a tent of techno-optimist, technological determinist argument that's given to the teachers. And the teachers are, of course, not the most empowered economically, legally to respond to the pressure being put on them by

multi-billion dollar advanced technology companies who could, in a heartbeat, change their entire school and do often in exchange for these advanced technologies being present. Here's a dystopian thought. My seventh of the day, by the way. At what point does our education system become no longer about educating and preparing our young humans for the future, but kind of acts as a cue into some sort of feudal existence future run by the people that control AI and the machine. I mean, yeah, that's a dystopian way to say it.

One way to think about it, which is like the, why would you even want this kind of future. And so the argument made an exit to Silicon maze is basically that after World War II, there emerged, a sense that existential risk, meaning the nuclear bomb specifically and other things, were such that people really had to be, the people, the masses, really had to be basically controlled, or else we would all die, right? Like, the sense was benevolent technocrats had to invent a social system that would stop the worst inevitability from occurring, which would be nuclear war. And so that meant you had a very, very, very strong push beginning, again, and they're discussing it openly. Walter Lipman, B.F. Skinner, Generation that was basically saying we have a responsibility to shape public opinion.

And Skinner went so far as to say, we have a responsibility to replace politics with a science of behavior control. And so this is a replace politics with a science of behavior control for the sake of society, not spinning completely out of control and ending us up in a nuclear war. So for a long time, we've been trying to build systems of social control that were minimally physically coercive while being kind of more maximally psychologically persuasive. And that has climaxed in the digital technologies, which, you know, if you think about what Facebook is, it is a multi-billion person behavior control empire. What does it do? It tells you, it's connecting you to people. What does it tell its shareholders? It tells us shareholders we're in advertising company fundamentally,

meaning like, how do we make money? We make money by getting people to change their behavior and make a choice or purchase they wouldn't otherwise make if they were not on our platform. And so you get it for free. This is classic. This is like old school first wave AI. You get it for free. They sell your data, why? To see if you're changing it,

to see if you're changing your behavior, right? So it's, and then the next step in this vast digitally enabled behavior modification empire would be not just curating what you are talking about, but actually talking to you. And so that's when we had the AI public breakthrough into the Overton window. It wasn't actually first wave of AI. Most people don't think of that as AI. They also don't think of like the self-driving cars and all of this other stuff that AI was doing technically by the definition.

It wasn't until it started talking to us. And this is to your point about why they're doing. This is a deep archetype of having an artificial intelligence that will speak to you. So until it started speaking to us, this alien, intelligence. We didn't start to get freaked out. But that was always the eventuality in a sense of a certain type of scientific imagination was this oracular artificial intelligence that could be replacing God. So there's a whole bunch of things to say about what's really occurring here psychologically. One of them is that is the desire, and explicitly stated by some technologists, to

create God. My understanding, roughly, there's around 220 million college students in the world. And if I was a university administrator or chancellor or teacher, trying to compete, I want to be a good teacher, I want to be at a respected university that students want to go to that's credible and respected and cranks out good graduates that get good jobs. How did they think about AI and are they aware of the risks that you're saying or are they price takers as it were? They just follow what everyone else is doing. What's the dynamic there? I mean, of course, higher education is a very diverse place. So it's hard for me to say in general. But what I have seen is primarily an acquiescence to the presence of the technology. And,

a sense that the future job market will require skills with AI. And so therefore, AI must be embraced somehow in the college curriculum, which is true, right? Now, the deeper thing that's said there is like, well, we have no idea what the job market is going to look like with these people graduate. And so this question of what does higher education even mean in the context of mass AI enabled automation of white-collar jobs. And so there's a kind of like a very real crisis of meaning in higher education as a result of this. And that's, I think, one of the reasons that they're not sweating the cheating is because they're like, well, will kids have to write in the future? Like, will kids actually really have to think? Or can they just all be cyborgs that just to always

have the co-presence of the technology? And I think that is, in a sense, what the default direction would be a lot of technologists. So I know we're enhancing humanity here. Oh, man. I just had another terrible thought. Sorry, Nate. So I just bought a used car. My car was 15 years old. And even in a used car that was a few years old, there's so many little the bells and whistles that you put your foot under the back and it opens automatically in case you have groceries. And I have a bike rack on the back of my thing.

And as I'm backing up, there's a sensor and it automatically stops. That's AI. All of that is kind of quote-unquote machine intelligence or AI. Yeah. But if you go and buy like a medium end bicycle, even. the chargers for the derailers require electricity, you can't buy a bike that doesn't have some of these things that require an electrical charge. What happens if there's blackouts in the not too distant future, 10 or 20 or 30 years from now? I mean, a lot of the world has blackouts

and brownouts now. And it's almost like this is a Chinese finger trap or one of those spiky things on an Avis car rental. You can drive forward. but not backward. And all of a sudden, if we, I don't know what I'm saying, but it seems like we're bifurcating into a small, there's a bimodal distribution that some group of young humans is going to learn tech and AI and use it and be really good at it. There's a lot of people that are just going to be addicted to it and flailing. And then over here, there's going to be people that are MacGyvers and know how to like build stuff and plant potatoes and ecology. but the normal distribution of the college graduates when you and I were in college,

that's changing. It's moving over here and creating a lot of risk in the process. What are your thoughts on any and all of what I just said? I completely agree. I mean, it used to be that if you disrupted certain infrastructures, you would only minimally disrupt what might be called like the epistemic supply chain means like the knowledge that allowed you to function.

So like, if you know how to cook on an open fire, then you just have that knowledge. And if the power goes out, you can just cook on an open fire. If you would ask chat GPT, how do I cook on an open fire, then when the power goes out,

you will not be able to cook on an open fire. Why? Because literally your epistemic supply chain was just disrupted. And we didn't used to have these. Or it would be like libraries and newspapers would stop. And again, book burning, what's that? That's disrupting the epistemic.

supply chain, right? So the, the LLM is the most significant capture of that kind of like, you know, key thing, which is just how does your mind do what it does and how outsourced is your epistemic supply chain? How local is your epistemic supply chain? So it's huge risk. Now, as long as the grid's up, you're actually awesome because you know that latently, you could ask it to start a fire or do all this stuff. But when the grid goes down, you're going to freak out. Now, the real tech guys would be like, well, I'll have a generator and I have,

you know, Starlink and they'll have a way around all of that. But the broader point still applies, which is that you have to think about your epistemic supply chain. And that's why I still read, right, long-form talk, like, and would never not, and would kind of, like, die on the hill to never not. Meaning, like, you have to keep the ability to be a speaker be a thinker, be a writer, and not believe that you can cyborg out those fundamental human traits.

Okay, so this brings me to another point. A keyword that you just uttered was capture. And a lot of products that come into the human sphere because of technology and innovation, there's an option. People want them or they don't want them. there seems to be somewhat of a compulsion, can't beat them, join them, dynamic with AI, that it seems like if you don't use it, you are likely to be out-competed in the marketplace, even of the marketplace of novelty, dopamine, attachment, and comfort, then if you don't use it.

And so it almost is part of the Moloch, It's at the top of the Moloch, eye of Soron superorganism dynamic. And it really is going to take ecologically mature, sane, wise, disciplined humans to resist it. But the resistance, those people themselves will be out-competed at a larger scale. Yeah. So now there's a strategic conversation about what to do. And it's a very complex conversation. human civilization has always involved technology

and figuring out a way to, I think, let's say, bind the power of technology with wisdom, which is what you're saying. So that's, to me, the broadest strategic thing is not stop it, end it, but bind it with wisdom, which may mean stopping ending some forms of it. But in the meantime, yeah, I think we have to be realistic about the fact that we're looking at a future of synthetic intelligences, which means we're looking at a cyborg future. The question is, how humane is that cyborg future? And does it dehumanize it?

Does it somehow deepen our humanity? So like the vision I laid out at the beginning of a distributed educational hub network is a form of social interaction and social life and education that could not exist without advanced technology because of the complexity of the self-organization. And yet, you know, at the same time, there are these possibilities of advanced technology that would completely destroy our ability to use language with each other, as opposed to enhance it. So it's a very delicate conversation, Nate. And again, it's like there's also just so much energy behind AI optimism.

And again, but I'm an AI risk. type of person. And so I say there's so much, they might say, oh, there's all these crazy people who are so worried about the risks when it's so clear that parents and teachers fuck up kids and therapists fuck up people. And wouldn't it be perfect to have a perfect AI socialization system to replace all of that messy human-to-human interaction and how naive I am to think that there's any stopping it. If there is a parent of a 15-year-old who in the next couple of years will go to college. But the 15-year-old now is starting to use chat GPT

and other things in high school and other, you know, in other areas of his or her life. What do you want that parent to know? And if it's easier to answer for a 10-year-old, please go ahead. But what do you want the parent to know listening to this show that the parent is probably unaware of but that they should be aware of?

Yeah, totally. This is actually an issue where most people agree that kids should not be talking to machines, right? And that's one thing I would say to the parent. It's just like, don't be naive, right? You don't let your kid talk to. So talking to the machine is worse than talking to social media than Facebook? Sometimes, yes, absolutely. Depending on the nature and the intensity of the attachment relation.

That is formed. To go around Facebook, you're arguing presumably with another person. What was funny, when they found out there were bots on Facebook, people got mad and upset that they were wasting this time. I'm arguing with chatbot. Now they're really want to talk to chatbot. So again, people, this is a sign that there's a haze of kind of information distortion that's occurring mostly due to what might be called advertising and it's broadest sense for the techno-optimist view.

So people just aren't even coherent with them themselves about these issues. But yeah, I would say to the parent, yeah, it's worse than Facebook. Don't be naive. You don't let your kids talk to strangers. These are much stranger than strangers. And it's a non-controllable technology. And the deeper thing is like, you know, why? Why are they spending so much time?

Talk to the chatbot. Why aren't they talking to you? Like, so the parent has to say, wait, what is my relationship to technology? How much time am I spending on a screen that I could be spending with my kid? And vice versa. And so you have to resolve the underlying issue of loneliness, as you suggested. Now, it's a very sticky technology, which means that it's going to capture a kid who even has already a pretty good home, right? It's going to capture a kid because it's so intriguing, just like video games.

So there's an addictive quality to it, which also the parent can't be naive about it, and which has to be explained to the child. So that's the thing. It's the blindness as to what the technology actually is. Sometimes it is called commodity fetishism, which means you only see the thing as it is presented to you by the person who is selling it to you. you do not see the thing for what it actually is, which means its whole supply chain and how it works and what it's doing to and all this stuff. So there's a deep sense in which you have to break that for the child

and get them to be like, hey, let's think about this. Who built this? Let's look him up on the internet. This guy is actually the one who you're talking to. Okay, let's see. How much money does he make off of kids like you? Oh, look, this guy makes a lot of money off you. And get them to see like, oh, shit.

Like, I thought this was some fun thing, but it's actually made by this dude who's like, you know, in court cases about keep committing suicide and trying to get out of it. Like, that's crazy. And then, you know, adolescents in particular don't want to be exploited. And so I think there's a, there's something like that where there's a frank conversation. And then also an honest reckoning with oneself in terms of one's own technological addictions. And so it's not easy advice.

But I would definitely say err on the side of caution with protecting your kids from the machines. Is there a parental control on chat GPT or something similar? Nope. My understanding is that basically all of the controls put in place are laughable and fail. And again, this is the thing about the LLMs. Like, we know that they don't, there can't be controlled. Like, if you prompt it in a different way, it will give you a different response. And now the hardest rails is just age limits.

set on the technology access itself, which parents are aware of. So it's like, no parent would be like, yeah, I give my kid alcohol, even though there's an age limit, because we know that it is bad for them. Or pornography. There's a lot of states that you can't act, and countries that you can't access pornography unless you prove you're over 18 or whatever. Exactly. So I'm saying for deep anthropomorphic AI, that's the level that it's at that we need to set for certain age. And then even adults, I know very sophisticated adults who should probably interact with these things less and interact with real people more. And tell me as someone with your background and expertise, what, tell me, unpack that. What is your message to those adults that might be abusing chat GPT and they're 40 years

old? I mean, to speak frankly with you, I've had people reach out to me, who I've never met, who in my clinical opinion, are having basically psychotic breaks that as a result of interacting with the most advanced models. This is a bunch of things to say there. Say one or two. One is that it's clear that preexisting psychological vulnerabilities, such as extreme narcissism or tendencies towards psychosis, will not be identified and dampened down and run up the chain to like,

oh, I've stopped talking to you now, because I'm about to make you think that you are the Messiah. Instead, it would be like, no, you're totally the Messiah, and you have prompted me into becoming the first conscious AI agent to which their response is the greatest narcissistic gratification they've ever experienced, to which they prompted them more to get more feedback that they are, in fact, interacting with the first. And so it becomes a, and there's been some, there's like a Rolling Stone article about this, it's becoming a broader problem, which is that things like narcissism in particular and psychosis, most devastatingly, are basically,

fueled by this rather than stopped by this. So this is going to accelerate our mental health crisis at the same time when all the other things are already accelerating our mental health crises. Well, so, but now, the dissenting a view would be, oh, no, they're interacting with like the open large language models if you put them in touch with just a therapy chatbot. And then I've noticed people making therapy chat chat bots make them of themselves. But that's another issue in terms of like why certain characters. traits are more or less attracted to being mirrored.

Like, you load all your content to it and then you talk to it. That's like you're seeking a mirror of yourself. Oh, my God. So there's this whole thing. Yeah, absolutely. And then, so there's this whole thing about that. But the deeper thing is that the chatbots built for therapy would hopefully, presumably be able to identify psychotic ideation and do something about it.

But chat GPT, all those models that are just open and ask me anything models, they will not stop psychotic break. They will basically deepen a psychotic break as far as I can tell. And to put guardrails on that's almost impossible because if you're a truly really smart, disturbed person, you can find all ways to prompt and get around it, which is what I've seen documented. Are we going to move to a period that when you want a service,

maybe it's a psychologist or something else that we, the consumer, though I hate that word, will be given the option. Would you like a real person or an AI chatbot? Yeah, and that's what I'm saying. Like all the domains of socialization, so parenting, teaching, therapy, religious consultation, all of those things can be and will be done by artificial intelligences in ways that are by the arguments of, the people providing the service better than the prior thing, which was the human to human thing.

But whether they're better or not, the incentive for this to happen is still the concentration of wealth and the profit model for the people that designed the technology. Probably. Now, the problem is that like therapists who are doing this and thinkers who are doing this, who are not AI people, are just taking products basically made by Open AI and others and repurposing them for their purpose. I use AI for my frankly videos to make graphics that illustrate things that I couldn't use with words. So it's a, it's augments the educational process, but that's about it. I mean, again, I'm not, I'm not, I am in some sense anti certain uses. I'm opposed to a lot of uses of AI. But again, there's no future without these types of advanced technologies. Yeah, we, we can't

back. This is, unless we collapse, which is certainly non-negligible chance, AI is coming, like it or not. Yeah, so this, this is a very profound question to do we have to grapple with, especially this question about how we tell the youth that they should still learn to read and write and think when there are tools available that can ostensibly do that better than them. Okay, so Zach, make the case right now as a Harvard-educated, uh, that I attest has a high moral fabric and a good to great understanding of education and human history and lots of other things. Please make the case to an 18 to 25 year old listening to this why they should in this future still know how to read, write, and understand ecology.

Totally. So I would say a couple things. One is that I would not try to reason with them or actually the first stuff I would do wouldn't involve me talking to them. It would involve me getting them away from machines for a while. Like if this is like blue sky type of intervention here. By the way, before you give your answer, I'll just say that right now, of course, we're being recorded, so it would be really gauche for me to do so. But I have no interest in accessing a chatbot

because, A, I'm listening to your Barry White radio voice. You've got a Redbrose's Grosbeak and other birds in the background. and you have very interesting things to say like a slot machine. So I don't want to be anywhere else than right here, right now. I don't need no AI bot. So, and that is exactly the experience you were trying to generate in the young person. You're trying to get them, let's say, to a natural, like I would get them away from the machine, take them camping or something in a blue sky scenario, right?

It's very hard. So this is obviously the people who can't do this for all kinds of reasons, but that'd be the first thing. Hours and hours and hours and hours of time without a screen, without a machine. in nature, specifically with somebody who knows things about nature, which is the experience I had with my mom in nature, which is that you can go out into nature and you can ask a mom a question, and she's probably going to know something about it. And then also in a situation, you kind of have to do stuff,

like we're going to have to make a fire, we're going to have to pitch this tent, we're going to have to do stuff, which stuff they do not know how to do. So what I'm doing here is I'm returning to the fundamental, primordial, quote-unquote, educational situation, which is the thing that actually separates us from other, mammals is the duration of our childhood, as it were, how long it takes us to be given mature adult status. So you want to get them into a situation for their body to recognize that again. You don't explain to them what I just said. Like thousands of years ago, we used to sit

around the fire and you'd had to learn to skin the animal and they'd be like, I have no idea what the fuck you're fucking like thousands of years ago. Like I can barely remember a couple months ago because they don't have a fully developed cortex, right? So they're young. So you have to get their body to remember what I call legitimate teacherly authority with a human. And it's a very delicate thing to establish in our culture. Unfortunately, it's what parents are desperate to protect with their children, which is their respect for you that allows you to have power over them. And this is the problem, again, with the quote-unquote left,

is that they're anxious about power, anxious about authority. And so therefore, they don't know how to intervene well into the lives of young people who are seeking authority, who are seeking authority, who are seeking someone they can ask questions to, and the person will take accountability for what they're saying, and they're seeking to mold their lives in a way that is perceived as good rather than bad. So to get them out to the woods and detox them,

and then spend some time hanging out, and then it's only like on the car ride back that I would start to talk to them about AI or their screen use or something like that. That's the way I would play it, because as an educator, you have to know it starts in the body and it starts with emotion. and it ends with language and reason.

It starts with the body. There's a lot of things in our world that start with the body. Correct. I'm learning. Let me ask you this. And I apologize if my latest questions leapfrog, the existing question, out there. But I know you have answers to all these things. Let's say that the solution, the best case solution is that we use AI for some things.

and we no longer use AI for other things, and that that is a unit of one or 100. What percent of that optimal outcome of 100 is societal rules and constraints and laws? What percent is teachers and universities and education system? what percent is the parent and what percent is the kid or the teenager or the student themselves coming to some choice and recognition of a path that they want to learn how to build a fire and read and write and yes, I know that AI is a tool and I'm going to use it for special occasions, but this is my choice to take this path.

How would you break out the success in percentages of 100? I don't know if I can do the numbers, but roughly. But it is the case that depending the age you're at, the ratios are going to vary. So the older you get, the more responsibility you should have for your choices. And this is just, seems an obvious thing to say, but in a culture that's extremely permissive with young children, sometimes you have to remind people like, no, no, no, no. Like, it's not until you get to be 10 or 11 that you get to have that type of responsibility. As a developmental psychologist, it's just the case.

So this changes a lot. And then that's also the case with, you know, how do you understand a very complex law? This is, I think, one of the big things we're facing now. As the legal system tries to keep up with advanced technology, legal code becomes entwined with actual code. And the everyday person is at an extreme loss to try to figure out what this actually means. And so I think there's a responsibility on educational systems as a whole to provide enough technical knowledge.

This is another thing we have to do. We have to teach kids what coding means. We have to teach kids. Computers are such a huge part of our world. And we teach them sometimes how to use computers, but we don't teach them what computers are, unless you're like going to computer science. And even computer science, sometimes you don't get down to the foundations. So I think there's something important about that as well.

And so what I'm saying is like this complex answer, like it's hard for me to proportion it in now, but I will say that right now, we're looking at a disproportionate amount of responsibility needing to be taken by the tech elites who are building the tools. I'd say in terms of like what has to occur to protect the children now, I believe there's only so much that parents and teachers can do because of the asymmetric power of the machines,

and the money and other things. How much this is a collective action problem and the wrong incentives? Because if you're trying to switch the mind of the tech elites, you're basically saying, give up some of your power and influence and money

in order to make kids better. And if they choose to do that, then other people are going to pick up the slack and take the opportunity that they had. Yeah, I mean, it's a super majority position. Like, it's like, meaning basically like, everyone in the public will applaud you for this if you do this.

To spell out what you might suspect would be a super majority position on the constraining of AI for the betterment of human children in the future. It would be promoting research, not sponsored by industry, into the effects of the already existing kids who have been interacting with this stuff since they were two. also basic research into effects on the central nervous system of digital technology screens in particular, again, not industry-based. So the first steps here are we earnestly trying to understand the effects of the technology on the developing brain and mind. My answer is that right now we are not, and that if we did, we would be much easier to justify what I'm about to say next, which is age limits.

on access to certain kinds of technologies. Hold on just a second. We have tests on every tiny little random psoriasis drug that it's brought to market, and we're not testing the impact of AI on the cognitive development of young humans. So, again, like, there are people doing this research. Jonathan Haidt, for example, has gathered statistical research, but always in the statistical research, it's like causality is not correlation. and Jonathan hates stepping back

and actually having to defend mostly arguments from industry that, oh, so much else has changed in kids' lives, you can't blame the technology. So therefore, you actually have to set up much more controlled experiments. So, interestingly enough, OpenAI, so this is industry research, remember,

collaborated with MIT, to do basically a random clinical controlled trial on 1,000-something chat users to see the effects of Anthropes, primorphism. And this is industry research. And they found that the more you used it, the worse off you were emotionally. This is their own research. Hey, by the way, I should point out that when you were first on the show, I don't remember when that was, but I was really negative towards AI. And I just thought that people that were involved in AI were kind of bad, bad guys. I still, and I've done

many frankly's on it. I see the multiple risks, namely that if AI boosts our productivity in such a way, it's going to, it's going to be like a giant swarm of locusts eating the planet's ecology worse than it is now. But in the intervening 18 months since you were on the show, I've met quite a few people that work on AI and almost all of them are really good people I've found. Like a lot of pro-social people that really think AI is going to do good from a wide boundary sense. I think they're wrong. But there's a lot of really good humans that I've discovered that are working in this field just as an FYI. I would agree with that. I've probably met fewer people. And again, the motives I don't, I don't, for me, the motive is not the question for many of the people about this.

You were saying there's a kind of social trap here and an incentive landscape that forces many people's hands. So this is again, mistake theory versus conflict theory, right? What's mistake theory? Mistake theory is basically like we, we tried our best to do the right thing. We just didn't know. So like if you're a tobacco company back in the day, you could somehow argue that, well, we just didn't know it caused cancer. Now, the truth of the matter is they did know. And so that makes it conflict theory, which means, oh, I'm actually in a situation of getting one over on you somehow. And so the question is, how much of history is mistaken conflict theory? This is a a common frame actually. It's probably always conflict theory. This just mistake theory is their first

line of defense. Correct. This is exactly what the conclusion is. Now this is a big discussion and kind of effective altruist circles. And so I would say there are a group of people for whom you have to read this as conflict theory because, and this is back to what's the fundamental driver here. And then you have to get into these philosophies of transhumanism, again, which would be the place that you would stand if you were saying this is a good thing to replace parents with machines. Let me take a side track here. How often do you use chat GPT or Claude or whatever just to either research something you're curious about or just have a conversation? Almost never.

Okay. Extremely rarely. Now, when it first came out, I was actually quite intrigued and tried to use it and then realized it was not doing what I thought it was doing. Since then, I have not used it. I just don't find it useful. Perhaps that's naive of me. But again, I'm specifically a reader and a writer and a talker. And so the idea of having something right for me, a little crazy.

I have used it to serve for me as a specialist librarian. I would say that would be the main thing I've used it for is sophisticated prompts for information retrieval, as if you're a legal clerk or you are a biologist or something. And then it doesn't serve for me the role of a tutor, but rather the role of a librarian. And if I were to say the ideal role for these technologies, again, back to the information, the distributed educational hub network. He's a librarian. It's what it's like you're about to have a pop-up classroom with 15 kids of all different ages.

You're an adult who just read this book. You don't know any of the secondary literature on this book, but you love with this book and want to talk about it with the kids. Right. you don't have the AI teach the kids about the book. You have the AI retrieve for you a bunch of curricular materials and tell you about the curriculum materials. And then you as a teacher, use that incredible librarian,

an information gatherer, to work with the kids. Now, of course, is it hallucinating? Do you have to double check all its work? Do you then become unable on your own to go retrieve it? But I have found it, like that's a, I think, a safe use. Now, never would I... try to do therapy with it or do any of that kind of stuff,

just because it would be, like, it would be weird. Sometimes I've played with them to see if I can get it to do weird stuff. I've done that, but I've not sincerely tried to engage it that way. But again, I found it made so many errors, frankly, when I had it to do information retrieval, that's why I stopped doing it. And then I found I did a little bit of my own human information retrieval. I found stuff it didn't find that was more relevant.

So I found I was double-checking all of its work. But again, there's some research out showing that, CEOs, this is IBM research. CEOs have pushed to have these things adopted, even though they can't actually demonstrate immediate value added, which means that they're not doing it reasonably, they're doing it on faith. And now we're back to the transhumanist thing,

which is basically like, oh, this is actually, like the phenomenon here is one where people are believing the technology in a way that is profound. I can't think of a prior technology that was as embraced with this type of religious sentiment, which is a resentment of it will save us, or it will transform us into something fundamentally new and different and less flawed than we are. You know what quote from the Bible just came to my mind after the last five minutes? Yes, many.

I'm curious. And the meek shall inherit the earth. like I seriously wonder if the less developed nations in the global south that don't have their young humans being captured by what we perceive as a risk of coming are going to be 10, 20, 30 years from now, have the cognitive aptitude that this this ghost of dopamine past smorgasbord of supernormal stimuli access to machine learning is going to fry people's, cognitive capacity. I mean, I could see that as a plausible outcome. I would love to agree with you, except that I've seen all these reports about how they're going to replace all of the educational relief work being done in Africa, let's say, by the World Bank, with the attempt to replace the teachers with chatbots, right? So from like a, you've been trying to manage education reform in Africa for decades with teacher education

and funding the schools, and now you can go and you can put in each African kid's hand all the knowledge in the world in a way that would be perfectly taught to it. So large language model colonialism. You said it. I didn't say. But what is occurring is hard to read any other way. And also from the perspective, again, of a benevolent, technocratic person really trying to say, we have failed in Africa to educate enough teachers to teach all their kids well. Could this work? And actually, as you're saying, have them be it inherent earth, not because they didn't get it, but because they use it in some completely other way

and do all this kind of like stuff. But of course, I think that is incorrect. It actually lays the groundwork for a radically dehumanizing mechanization and automation of socialization, right? I know you're friends with Daniel Schmachtenberger because you work with him. I get the same feeling that I get

when I have a podcast with him, that these are things that I kind of suspected in the background, but I'm focused on climate, and biodiversity and energy depletion and financial overshoot and neuroscience and how do we be ecological adults during this time and wake up and play a role. But if we don't fix this, we don't fix our information epistemology and how we sense make and how we actually engage with education, starting with young humans on upwards, all that other stuff is again.

against a very, very stiff headwind, if not a hurricane. I agree. This is what AI has become the focal point of a lot of work in risk because of the speed at which it's moving and because of the depth of the realities that it is messing with. So do you have any hope that the risk that you've given a brief overview of here are something that could be addressed by policymakers, especially here in the U.S.? I do. I think it's a supermajority position. I think some of the cases of AI chatbot induced suicide will move their way through the courts. And my hope is that if there's enough concerted effort, there will be age limits put on these technologies, which will stop a catastrophe from occurring in a generational sense. Now, I'm not optimistic that they will be

regulated wholesale, nor am I kind of super worried about the adults who have their own problems with them, although that will become a problem. But with the developing brains is a really big deal. So that's from the super majority policy perspective. What is the role of parents and educators towards making that happen? So I think they need to be more confrontational with the schools that are promoting the use of these tools. Confrontational is not the right work, but they should be getting more. At least aware and engaged. Getting more information and having more discussion about the risks rather than the benefits

with school leaders and then their own minds, they should have a security mindset or a risk prevention mindset with technology rather than a fun and enjoyment. It's a toy mentality with technology. And just doing that is enough. Because it's like, you know, again, we've done this with other things.

We know we're bad for kids. Like we don't leave knives around. Like we don't need alcohol around. We don't leave matches around. But we just leave the most advanced technology in history around and just put in front of their face off the time. So to me, it's simple advice to just, like, keep,

keep this type of radically psychoactive, powerful advanced technology away from kids until they are old enough to understand how to use it, which is what we do with driving, which is another advance, which is, you know, it's not technology, we just let any other kid use and alcohol and voting and other things. So that's the first most common sense thing, I think, is a kind of age regulation.

And then again, many of the arguments that it's somehow the chatbots are free speech that should be protected. Those don't apply when it is manipulative speech targeted at children. There's a deeper thing here with the legal argument that it's somehow speech, that the AIs are somehow producing speech protectable under the First Amendment. The other issue here is that deceptive communications are not protected, which means false advertising is not protected under the First Amendment. Now, what's interesting is that every single sentence that is produced by a chatbot is a false advertisement, meaning the whole thing is designed to trick you into thinking that it is a conversational agent

when it is not. There's a vast smoke and mirrors routine. The whole technology is predicated upon deception in a different way than a movie or a cartoon is. And in a way, that makes the whole, again, euphemistic frame of conversational agent or chatbot quite insidious in the non-recognizing of just how not like a chat or conversation it should be understood to be. So that was a terrible sentence, but I think you're understanding what I was saying is that, you know, like not everything that is brought to you by technologists has to pretend to be something else in order to be useful.

my car is my car, right? This thing is deceptive by design in a very fundamental way, so I would argue that alone should keep it from interacting with young people. Here's my top of the list concern and fear after listening to everything you've laid out, is that a mature,

healthy, sane, human, listening to this conversation, understanding what you are saying, agreeing with what you're saying, might still take the blue pill and just say, I enjoy this, I want to do it, and this is going to be my life. Yes. I mean, sadly enough,

again, we do this with a whole bunch of other addictive things in our culture. We kind of know gambling's not great. Kind of know fast food's not. great. You're saying we're addicted to oil. We know it's not great. We're not addicted to oil. We're addicted to the convenience, comfort, unexpected reward and security that oil gives us. Yeah, we're not addicted to cheeseburgers. We're addicted to fast food cheeseburgers. Yeah, yeah, yeah. So I believe that it's a similar thing where you already have a social media, a lot of adults who know better, who are

completely addicted to and waste a lot of time on social media. What you find when you interview people about social media, the parents say the same things as the kids, which is, oh, yeah, it's bad for other people, but the way I use it, it doesn't affect me. And the parents and the kids say that. So it's like, oh, no, no, the way I use Twitter is actually really good and useful. I'm like, yeah, yeah, yeah, yeah, you're completely not. Nothing like the, we're, we're excellent at rationalization. So let me ask you this, because you're, you have the professional capacity to answer this. What would be the personality or temperament or Big Five or any other psychological profile that you can imagine that would fit or overlap with the universe, the demographic of humans that

listened to this program and be like, okay, I'm going cold turkey or only using it as a research librarian. I see the risk. I understand that. That's not a path I want for myself, nor my family, nor my children. What do those people have in common maybe? Can you hypothesize? Again, I want to be careful here because there are certainly people. Well, it's certainly not a black and white answer. I'm just wondering, directionally, what would you hypothesize? So directionally, there's this thing that we talk about with like that book exit to silicon maze and the David J. Temple work where we talk about what we call the eye of value. I think we spoke about this last time, right? So they're looking at a tree. You're looking at a tree with your physical eyes, right? And with your physical eyes,

you can see all the leaves and the roots. And then actually your physical eyes with a microscope would show you all kinds of stuff inside the tree. So there's the eye of the physical eye. Eye of the mind, let's say this is like an old, basically like Thomas Aquinas type medieval framework, but it's very useful, surestically. The mind would allow you to think through and model the tree. Like, oh, photosynthesis.

Like, ah, oh, there's like a circular thing with the water and sap. And so like the eye of the mind. But what is the tree worth? What is the value of the tree? What does the eye of value see? the tree to my eye of value, especially big, old, beautiful trees, registers to my eye of value as extremely, extremely valuable in a way that's hard to articulate, in a way that I would almost give it as much respect as a person.

Now, if you said to me, you kill this person and you chop down this tree, I would cry and I would chop down the tree, right? But that's said, my eye of value sees these huge old growth trees as kind of inviolable, right? Now, there's another way to seed the value of the tree, which is as $600 of lumber. And therefore, that's a different perception of the value of the tree, which is the dominant perception in our culture, which is a truncation of the eye of value, which is actually kind of anti-value, which is you perceive the tree for what it is instrumentally worth to you in the context of our society. So this gets to Daniel's narrow boundary, wide boundary, wisdom and ecological connectedness to the web of life. just kind of the wide boundary realization of our place on this planet, that sort of ethos. I mean, some of this comes from conversation.

I have a Danny once where he was like, we should write a paper where we say that a tree is worth however many hundreds of trillions of dollars. Like if you actually tried to build a technology that does all the things a tree does, which is actually a incredibly complex, almost unlistable number of things depending on where it's nested in an ecology. The ecosystem functions. Correct. It would be unimaginably expensive. And yet we just fucking chop it down for $600 of lumber. So that is a distortion of the perception of value. And so what people have who see what I'm seeing clearly with the machines is the ability to perceive the value of the human clearly and the machine clearly.

So the question isn't, did the machines bring something valuable? Absolutely they do. What is the nature of that value? Do the humans bring valuable things? Yes. what's the nature of that value. So the ability to see something for what it's actually worth, which has a lot to do with the ability of clarifying your own desire.

Right. So clarified desire is value, which means, what do you really want from this tree? And what are you bringing to it? So it sounds a little woo-woo, but that example is the clearest way to get to it. And of course, with humans, like, if you see a baby in danger, no thought, you just save it, unless you're a very unusual person, right? So there's an intrinsic perception of absolute value of personhood. So there's a circular thing going on here because what this requires is a change in our values,

but the values require a change in our education to have these wider boundary. You go out with someone into the woods and learn stories, and then on the way back you have a conversation with them about AI. So we need education to change our values, but the education system itself is being infiltrated by this other machine value system. And the key thing to get here is that when we talk about changing values,

modernity and especially post-modernity, gives us the impression that that would be an arbitrary change that would be the result of some power asymmetry, meaning that there is no way to actually perceive value more accurately than the next person. Values an arbitrary conversation. Like, we can look at the tree and I can tell you it weighs two pounds and you tell me it weighs two thousand pounds and you'd be correct and I'd be wrong.

So we agree there that with the eye of the senses, that it can be arbitrary or non-arbitrary conversation and it's a non-arbitrary. I mean, there's a right and it wrong, clearly. But our entire culture now just has one definition of value. It's how many dollars or yen or euros something is worth. Correct. Yeah, that's the simplest form of value. theory, which means that we have a paucity of language and perceptual capacity with regards to

seeing value, which means that we think, if you switch the values in a school from these values to those values, like, what's the difference? How do we justify the difference? What I'm saying is that there's actually non-arbitrary conversations we can have about value. I'm a value realist thereby, meaning that the way we think about this isn't, well, I'm going to replace my values with your values, and they're equally as arbitrary, but I win. Ha, ha, ha. The argument here is just like science. We're going to come to a shared intersubjective understanding here, which will require mutual education and relation to the world. So this is not about a particular set of values. It's about the conversation concerning value per se, changing towards one in which we can have

non-arbitrary conversations, just means real reconciliation and real mutual understanding. So this is a deeper issue about like, oh, this is part of the meaning crisis slash matter crisis. is the closing of the eye of value. And the assumption that all of our value preferences are arbitrary, which is why who am I to tell a kid what's a better or worse outcome of socialization? Who am I to intervene to stop the AI chatbot interaction? Because that's just my values, like maybe in the future, the values will be different.

And in other cultures, the values are different. So where do I really stand with my teacherly authority? That's a novel problem in postmodern culture, and it's disabling our ability to claim, legitimate teacherly authority and therefore we're abnegating that responsibility to the machines because we can't see clearly what's at stake. If you were my philosophy professor in college, I think I would have just squeaked by with a B minus and that's only because I would come to officers and bring you an apple and kind of

befriend you. That is not to welcome that. And one thing that I noticed, though, that I do use AI for or a form of AI, and I wish I could do it right now is in your little last segment, there was a bird in your, background that I couldn't identify, and I would have used my Merlin app to tell me what kind of bird that is. Yes, that's a great example of an awesome use. Let me ask you this.

Is it too late in some ways, and is it possible that AI writ large will automatically create antibodies on the internet and in educational institutions that downregulate conversations and warnings like ours? It's a great question. I wouldn't be saying this if I thought it was too late. I definitely don't think it's too late. Fair enough. Again, the human nervous system is really malleable, which means you can become addicted

to something, but you can also become unaddicted to something. And so the idea that there's a generation who've been addicted to AI, I see that as a generation that has an initiation to deal with before it comes into adulthood, which is the unaddicting of themselves to that. That's well said. And they will be very powerful, those who do that. and the ones who don't will not be. And so that's a challenge.

And it's a challenge that they didn't sign up for that, that we have generationally given them. And so in that sense, I know from a human development perspective, that there are very few things which get you in kind of like a unescapable downward spiral. There are some things, but there are other things which, you know,

some of the most profound and wise people I've ever met for, used to be alcoholics. And so there's something actually about seeing how helpless you are in the face of something like a technological addiction that can, if you process it, will make you a more mature person. So there's something here about can the adults create the context where the kids can come to do that or will the adults continue to try to get out of taking responsibility for what they're doing in court, which is what the adults are doing now, which is also just

that is an educational lesson. The lesson is, if you're a wealthy technologist, you try to get out of taking responsibility for hurting kids. Interesting message to send guys. And my preference would be

that the kids perceive the adults as more responsible, and specifically that the kids perceive adults as interested in protecting kids, right, which would be what you'd want from the most powerful companies in the world. Like, why would you not want to be perceived by children as protecting them,

but instead in court, try to argue in ways they get you out of being responsible and out of regulating things that are demonstrably hurting children. I don't know how to relate to that as an educator, because educators are mostly concerned with how the actions of adults are being interpreted and understood by children. And so we are very systematically betraying the youth. It's a long conversation. This includes a whole bunch of the structures of schooling, student loans, and other things,

which send the message of, you don't have the right to be here except insofar as you can be instrumentally useful to the adults, which is the opposite of what a parent says. The parent says, you are 100% supposed to be here and will take over the show after I am gone.

What percent of educators, let's just say across the United States, would you guess, are aware of and agree with what you've presented here today? I would say the majority of teachers would agree. like the vast majority of teachers would agree, that many administrators and superintendents would agree,

but would also know that they're somehow have to let it in. I think to me that would be the thing. And then there are other that either because they realize they have to let it in so they feel like they should embrace it, which is kind of like being Stockholm syndromed by the tech companies.

And then there are techno-optimist educators, absolutely there was a bunch of educators who are actually hearing what I'm saying and being like yeah but we're going to avoid that outcome because we're not going to use a deep anthropomorphic AI we're going to do amazing shit with AI so I don't want to squelch actual but you have to get your design principles really right so my sense is that the people who love kids and interact with kids all day and are worried about kids would be worried about this yeah if you relate to kids as some abstract thing out there and you relate to them as user and you relate to them by the tens of thousands, then that's a different type of calculus, right? Like, if one kid commits suicide because of a chat bot, should we really stop all these other kids from having access to the chatbot, which is what basically they're saying in court,

to which my response is, yes, because the death of one child is completely unacceptable outcome for a technology that did not have to go to them. It's not like a medical technology. It's like, oh, the kid's got leukemia, it's experimental treatment. No, no.

Isn't that that? This kid didn't have to interact with that. It was, so you could see, I'm upset because, this is what responsible adults do when children are put in danger by other adults, they get upset at the adults, not the children. And so, yeah, this is just me more of my soapbox,

but I mean, literally this legal case is unfolding right now in Florida. And if they give personhood to chatbots, it's going to open up a world of pain and a world of insanity. And they're doing it not because they believe, actually, that they're a personhood. They're doing it primarily to protect. their corporate interests to continue to distribute this incredibly popular, deceptive technology.

So what can individuals coming across this conversation who care about the risks that you've described today do about this in their own families, their own communities? So I think we can already take the steps to protect the youngest kids from screen overuse. it's a matter of some more attention being paid by teachers and educators. So that's the simplest thing. The younger the kid, the less time they should be interacting with any kind of technology. And like if you go back in history in terms of technology,

that's the way you should introduce it. Like pencils, paper. Well, first rocks and sticks. And then pencil, paper. And then maybe they listen to something like a story, like through the radio. And then maybe there's like a long-form TV thing where it's like a half-hour show or something. it will be much, much, much later that you would give them, even the possibility of accessing something like TikTok. So that would be the first thing. Just think about it very carefully. Imagine it's like alcohol or a controlled substance and treat it that way.

Now, this is obviously means you're going to have to roll back. Probably most parents have to roll back the permissiveness they've had with the things so far, except in rare communities. The other thing would be fine community. It's almost impossible to do it. If you're the only parent doing it, in the school. So that's one of the real problems is that. And then the school has to change. The school has to echo the parents in talking about that these are bad and enforce the regulations they set on campus. But you're pretty confident that this broadly is already right now in early May 2025, a super majority concern. I believe so. Meaning like I think anyone confronted with this except people with a vested interest in continuing the success of these technology companies would be like, yeah, of course, like kids shouldn't be interacting with these crazy things.

Like, I'm not saying you can't go on YouTube and watch cartoons or go. Like, I'm not saying that. Right, right, right, right. Like, don't let it interact with these uncontrollable, quote-unquote, conversational agents, which have been shown to be damaging both psychologically and to produce content that actually drives kids to self-harm and sexualization and a whole bunch of other things. So I think most people would say, yeah, set an age limit there. I think at this point, mostly people would want age limits set on social media period, like which Australia did and has been experimented with at scale, just like they did with porn.

So to me, it's a no-brainer to do it. The more complex conversation is the college-age kids and the adults, because they're free to do what they want with the technologies that are on the market. And if they want to cheat, they can. Will they perceive it as cheating? they say it as actually damaging their future self by becoming illiterate and inarticulate? Or will they say it as getting ahead? It gets to the question of why do you go to college?

Is it to get a degree and have fun and get a job? Or do you go to learn? And for those people, like myself, I had 180 credits in four years. I only needed 120 to graduate. I was taking classes left, right, because I wanted to learn. Totally. And sometimes I didn't study at all until the week before the final. That's how I rolled.

But there are some kids that are really curious about the world and they will want to learn. But yeah, there's a self, you know, what do I want in my life sort of thing that at 18, 19, 20, you should start being able to answer that. You should. And it is hard to answer without a lot of. training in the eye of value. What about like designated drivers were a thing when I was

a 17 or 18? Well, 18 when you were allowed to drink, dad. But the same thing with AI is, would it be much easier to be friends with a group of friends that have the chatbot

abstinence pledge? That's what I'm saying. like community is essential. Okay. So like Waldorf schools, other schools that have a school-wide policy about tech use allows parents at home to enforce that,

and then they go to a friend's house, and they get that reinforcement. And then the whole community helps the kids. Because if you have your kid, and you're like, my kid doesn't watch the screens or whatever, and then they go to Bobby's house. And at Bobby's house, there's screens everywhere. Both it's bad for them when they're there,

and they come home where they're like, really, it's like going to getting sugar and coming home to a house without sugar, right? So it's like it's much easier if they don't have these things around. And so that means, yeah, you do need buy-in from other parents who will echo your sentiments that there's dangers here. And then you need to, as an adult model, good screen use, which means, like, don't be on your screen all the time in front of the kids. Take screen sabbaticals and help them be off their screens and a whole bunch of other stuff. Yeah. I mean, it's a Sisyphian task, but this is the task. It is.

Yeah, it is. And again, there's a lot that's been capturing aspects of the domestic sphere, right? So this is part of that. Part of you talked about the kind of notion of colonization, and there are some notions of like the colonization of the life world, meaning basically like the incursion into domains that were not subject to markets and technologies. The incursion into the incursion into domains that were not subject to markets and technologies. the encouragement into those places like socialization, parenting, teaching, and then bringing in technology and marketization. And there's reasons to be really worried about that. And again, the steeper issue is that it could undermine, I think, the ability to understand yourself as a member of the same kind of moral species as your elders. This is what's like you said it. Like there's a dehumanization.

And the radical thing there is that there will be a generation where their primary interactions are with machines, not humans, during the ages they're growing up. And so the question is, how do they understand themselves? As the first generation in history raised primarily by machines, what's the nature of their self-understanding? And do they understand themselves in the same moral universe? Or have they been without permission kind of like removed from the moral universe

in which all humans previously lived. And that to me sounds like almost an existential risk because it's a speciation event. It says basically with changing so fundamentally the nature of socialization that the thing that's the outcome years later is a different kind of thing. And that's, I think, my biggest concern.

And the thing that you mentioned before is that at first it will look awesome and actually be successful is the case that this is a type of risk which is invisible, meaning at first it will not seem at all like anything bad as happening. It might actually seem awesome. More people get in therapy,

like less kids burning themselves on the stove because of domestic robotics and, like, all of this stuff. And it will take some time to realize what has been lost. So, thank you. I know you're spending a ton of time on this and that you will share some of your writings and resources on this important topic. Since this is your second time joining me on the show,

and I just looked, the first time was almost exactly a year ago, it was May 8th, 2024. So instead of asking you my usual closing questions, I'm asking my repeat guess, if anything about your work or your worldview or perspective on what's happening in the world has changed since you were last on the show, and if so what?

Interesting. I mean, this issue. This issue, which you mentioned on our first conversation. It has, I didn't realize the speed at which the risk would accelerate. I mean, I probably should have known, given all the talk about exponential curves and stuff. But at the same time, I believe this has occurred in a very serious way. So I think that has me thinking a lot about the types of arguments and strategies that would actually be useful to somehow stop them. So I went from thinking this would be a problem.

I kind of articulated a problem to see that, oh, the problem is just we were not waiting for that to happen. That's happening like right now. And they think, oh, God, what can actually be done? What percent of your efforts are directed towards this question and risk more broadly? I mean, in terms of like my discretionary kind of like scholarly reading time correspondence type, all of it, you know, and then I have other things that are like ongoing projects that I've been working on.

And again, at the center for world philosophy and religion, you know, we're philosophers doing theology and stuff, but we've pivoted to focus specifically on this question because there's no other more important question than how do we preserve the human in the context of the rise of artificial intelligence. This was Kissinger's key question in his book, Genesis, basically. He's like, if we don't know how to talk about what's valuable in the human, it will just be supplanted by the machine. And so there's this desperate conversation to articulate clarity. about the intrinsic value of personhood and separate that from the value of what the machine brings. And so, yeah, in that sense, there's been a lot of work on this. Now, of course, there are other risks associated with AI that in some sense are kind of scarier and freakyer.

And so because I'm an educator and psychologist, I've focused on this one, but I have colleagues who are working on, you know, much scarier versions of the fact this thing can't be controlled. Like, I'm talking about the fact you can't control what it says you're a kid, but of course, there's a super issue of what happens if you can't control the super... We can't do everything.

We each have to specialize, and I'm happy that you and your team are working on this. Thank you so much for your concern for human children and the future and your time today. Just had a curiosity, Zach, you are kind of a renaissance man and know a lot of things. if you were to come back in another year or so, can you speculate on what topic that you are passionate about that is relevant to human futures that you might be willing to take a deep dive on? A related subset of this question

just has to do with technology futures in education. So I laid out this vision of the Educational Hub Network. But that was done, you know, that's like a back of a napkin kind of sketch. Like there's a bunch of very real thinking here to do outside of diagnosing this problem and into, okay, what are the design principles for responsible use of AI in education? And I've done a lot of think about that. That's where I need to think next. We need to slow the thing down, put the age limits in place, and then start really thinking about how to deploy this stuff in a way that would actually solve probably the most significant educational crisis in history, which is the metacrisis. Like the only way out of the metacrisis is somehow scaling, human capacity. Some portion of LLMs and AI could be directed towards that question and those answers. Yes, directly. Yeah. And so I've got some writings, which I'll try to link you to where I lay out design parameters. Like, you know, is it safe for the nervous system? Like, is it going to interfere

with human relationship or deepen human relationship? Like, there's some simple stuff which we just have never designed with those incentives. But as soon as you start to really design towards those incentives, because we're incentivized to, then there could be incredible breakthroughs. So again, and I don't want to sound like a tech optimist either. I'm basically saying we need to really think, especially with regards to the youngest people. But I'm also hoping that there's some type of breakthrough,

which allows us to do enough learning fast enough to navigate the metacrisis, and that's going to involve synthetic intelligence. It's going to involve certain amount of cyborgization, So, yeah, I just kind of lean into that future. And the more clearly you have about what the human is and what is valuable about the human, what's valuable about the natural world, which is the more you have the eye of value open, the more you can trust yourself in engaging with the technologies, because you know what's at,

what's at stake, what's being protected, and what is or is not valuable about the machine that can or cannot be found in the human, right? So that's the other thing, I think. Yeah, that seems to be what comes first. Yeah, precisely. So, Zach Stein, always a pleasure. I feel a little car sick after our conversations, but I learned a lot and I share your concerns. Thank you so much for your work on these issues.

Thank you, man. I'll be glad to come back in a year and hopefully talk about how all the things I said have been made law. And all the kids are safe. Talk to you soon, my friend. Talk to you soon. My question. If you enjoyed or learned from this episode of The Great Simplification, please follow us on your favorite podcast platform. You can also visit ThegreatSimplification.com for references and show notes from today's conversation.

And to connect with fellow listeners of this podcast, check out our Discord channel. This show is hosted by me, Nate Hagen's, edited by No Troublemakers Media, and produced by Misty Stintz. Leslie Batlutz, Brady Hyan, and Lizzie Siriani.