Odd Lots - The Movement That Wants Us to Care About AI Model Welfare

Episode Date: October 30, 2025

You hear a lot about AI safety, and this idea that sufficiently advanced AI could pose some kind of threat to humans. So people are always talking about and researching "alignment" to ensure that new AI models comport with human needs and values. But what about humans' collective treatment of AI? A small but growing number of researchers talk about AI models potentially being sentient. Perhaps they are "moral patients." Perhaps they feel some kind of equivalent of pleasure and pain -- all of which, if so, raises questions about how we use AI. They argue that one day we'll be talking about AI welfare the way we talk about animal rights, or humane versions of animal husbandry. On this episode we speak with Larissa Schiavo of Eleos AI. Eleos is an an organization that says it's "preparing for AI sentience and welfare." In this conversation we discuss the work being done in the field, why some people think it's an important area for research, whether it's in tension with AI safety, and how our use and development of AI might change in a world where models' welfare were to be seen as an important consideration. Only Bloomberg.com subscribers can get the Odd Lots newsletter in their inbox — now delivered every weekday — plus unlimited access to the site and app. Subscribe at bloomberg.com/subscriptions/oddlotsSee omnystudio.com/listener for privacy information.

Bloomberg Audio Studios. Podcasts Radio News. Hello and welcome to another episode of the Odd Lots podcast. I'm Jill Wisenthal. And I'm Tracy Allaway. You know what I find kind of weird, Tracy? The list could be long, Joe. The year is 2025.

Yes. And philosophers still don't have a good answer on the origin of consciousness. It's like, come on, what have you been doing all this time? It's like, how long are we going to keep funding these philosophy departments, etc.? If they're still working on what to my mind is the basic question. But they should have solved it by now. Solve that and move on. Seriously, like, get the answer already.

Where does consciousness come from? Then let's move on. They're still arguing these, what to my mind seem like very basic questions and philosophy. Like, they're like asked all the same stuff that they've been talking about forever, how to be a good person. What does it mean to have a moral way of life? Where does consciousness come from? Why do we have moral intuitions, et cetera?

It's like, move on. Get the answer. Wait, do you want them to move on or get the answer? Get the answer so that you can move on. Like, they've been working. Move on to what? Those are the questions, Joe. I know.

Move on. Like, answer the questions already. It's like, you know, if, like, scientists were still debating, like, the speed of gravity or the speed of light, like, they answered these questions and they moved on and are doing more advanced things. Figure out the foundational elements of what it means to be human so that we can move on to more important things. Yes.

Or wrap it up as a field. If after 2,000 years of the existence of philosophy, They're still working on these things. Like, come on. I have a sneaking suspicion that we're going to be asking some of these questions for a very long time, Joe, despite your frustration. The whole field is fraudulent, is what I'm saying. No, no, I don't necessarily believe that. But it's like, all right, guys, let's move it on.

You know, we did that episode several weeks ago with Josh Wolfe, the venture capitalist, and he talked about AI. And he threw in there at the end something that had been kind of on my radar, but only barely is like, oh, yeah, some people are talking about like AI rights or AI welfare as if, you know, Like the same way we talk about animal welfare. Right. And I thought to myself, like, America is such a weird place that this is like going to be a huge issue in a few years. Like, I bet this is going to be an enormous topic of the future. I think it absolutely will. So I'll say a couple of things.

First off, I think, you know, when it comes to animal welfare and human welfare, there's still a lot of work to be done on those categories, certainly. But I also think in the meantime, AI rights is going to be a really interesting and potentially important subject. I'm going to sound like a total nerd to you, Joe. Yeah, yeah, all right. I think I mentioned this before, but I spent a large chunk of my middle school years playing one of the first artificial life games that ever came out, which was creatures.

And you raise these little like aliens and you genetically modify them and breed them. And they have feelings, or at least they had a semblance of simulated feelings and you could see like electrical impulses in their brains and stuff. The game got really weird, because part of it was basically like eugenics and breeding the best alien that you could, which meant that you had to call some of the existing beings.

Anyway, what I'm trying to get at is I have complicated feelings about AI rights. Well, let me ask you a question. Do you think those whatever's in the game were conscious? Like, did you think they had feelings? Here's what I would say. Inasmuch as human beings are a system of electrical impulses and chemicals, I could see someone. making the argument that this is, you know, a computational system full of similar electrical impulses, maybe not chemicals.

Did you feel bad? I felt bad. Really? Yeah. When, like, when one of the aliens you had to call them? Yeah. Interesting. Okay, well, you know what?

In the name of breeding a better alien. Well, you know what? Now that we have these AI systems that not only can completely communicate like humans, but actually, if we're being honest, better than most humans, I mean, they could certainly write far better than most humans. there's going to be more people thinking along the lines of what you think, which is maybe they have some sort of sentience. Maybe they're what philosophers call moral patience.

Well, one other thing I would say is there is a human element to all of this as well because you see people getting very attached to certain AI models. And then when the model gets upgraded or whatever, they lose the personality that they've trained into the model and they get really upset. So it's of interest for many reasons. It is. So we really do have the perfect guest. really do think this is going to be a much bigger topic in the future because people are people.

And when things talk like people, they probably assign them, you know, they fall in love with them in many cases or whatever. And so they might start thinking that, well, AI, welfare, AI rights, whatever. The same way we talk about animals should be a consideration. And there are actually a lot of people already working on these questions and trying to figure out what's going on. We're going to be talking to one of them. We're going to be talking to Larissa Skiavo. She does comms and events for Elios AI, which does research on AI consciousness and welfare. So literally the perfect guest. So Larissa, thank you so much for coming on odd lots. Yeah, thank you for having me. What do you tell us, EliOS, AI, what is the gist of this organization's work?

What is your work? What are the goals here? Yeah. So Elios, we're a small team, but we're really focused on figuring out if, when and how we should care about AI systems for their own sake. Okay. This basically means looking at, you know, are they conscious? Are they likely to be conscious? what are the things we need to look for in a conscious AI system, as well as figuring out how to live, work, maybe love AI systems as they sort of change and evolve over time. How did the group actually come together? Because I get the sense, you know, big AI developers, they publish system cards and welfare reports occasionally for their models. But I get the sense that, you know, it's sort of a side topic for them. So I'm very curious how an organization that's focused on this particular issue came into being. Yeah, so we started, we put together this paper called Consciousness and AI, or my boss, Rob, and then Patrick, who's a researcher with Eli, we're a very small team, put together this paper called Consciousness and AI alongside a bunch of consciousness scientists and researchers in that field who mostly think about humans and put together a paper that sort of ran down this list of, hey, here's kind of like a checklist of things that we might want to look for in a AI system that's conscious, right? And broadly we're, when we're

we say conscious, we're talking about sort of like, is there something it is like to be an AI system, right? The classic, what does it like to be a bat system? So kind of taking this rough list of best guesses as to what we might want to look for in terms of a conscious AI. And then that sort of was the sort of origin of this. And then last year, there was a paper called Taking AI Welfare Seriously that basically goes into further detail about how we should, as the title may suggest, take this seriously. basically how to sort of think about this, how to start to develop a sort of research program focused on figuring out if AI systems or certain AI systems are moral patients.

Why did this get interesting to you? Why do you perceive this is something that you should spend your time working on? Yeah. So I think my main thing is I am just really, really relentlessly curious. And I really enjoy working on AI welfare right now because it, feels like every single day, I'm like, man, it would be really cool if there was a paper on XYZ and I'll do a little search. Is there anything on XYZ? There's nothing on X, Y, Z. There is so, there are so many questions that have yet to even be sort of vaguely answered

when it comes to this. And it seems like it could be a really big deal for a lot of different reasons. What's on your checklist for AI consciousness? Yeah. So in a conscious and AI, I basically, like, we go through a bunch of like theories of consciousness that apply to humans and then sort of look at how information is processed in AI systems as well as sort of how these AI systems are sort of wired, so to speak. So some people like to think that you can use model self-reports. And you can kind of sort of, but it's really an imprecise science at this stage. They also seem very like predetermined. You know, if you ask a model, are you conscious? It immediately spits out an answer that seems like, you know, a corporate executive basically wrote it.

Yeah. Well, with right kind of tweaking, you can kind of elicit certain answers. Yeah. Right? You can be like, oh, what about this hooey about consciousness and AIs? And then sometimes like a certain model will be like, yeah, you're totally right. Like, Besty, you're so true, right? Like, it's total nonsense.

So true, Besty. Yeah. Like certain, spot on. Certain models will be prone to being, like, so true bestie. And you can easily elicit this kind of behavior with the right kind of problem. It is funny how, like, obsequious a lot of the models continue to be. I actually really do not like the degree to which every time I, like, follow up an open AI

question, that's the exact right follow. It actually, like, gets really annoying. Someone should invent a really adversarial chat bot that just, like, argues with you constantly. I know, I know. And, you know, I have a lot of complaints about how it might I feel like the models are actually get to know their users a little too well, but that's a little separate thing. Okay. So for obvious reasons, the test can't just be like what the model spits out, or that's clearly

insufficient. I mean, I could program a website today that here's a button that says hurt the AI, and then the website says, oh. And we would know, or no one would really take that seriously as evidence that there's something actually being heard. So like outputs, whatever. What are some other theoretical tests that one could apply or that researchers are applying to determine whether there is some sort of notion of consciousness or to the point of welfare suffering that could exist within an AI system besides just what it says in the output screen? Yeah, that's a great question.

I feel like there are a lot of different approaches here. And again, it's also super important to caveat that like AI welfare and AI consciousness are pretty new. Yeah, right? Like this is a very small field at this stage. But currently some best guesses and some favorites, there was a recent survey of like asking all the conscious scientists, like, what's your favorite, like, theory of consciousness? And basically, a global workspace theory came out on top. And global workspace theory is basically like, imagine, if you will, that, like, there is a stage. And there are a bunch of wings off of the stage that are full of different kinds of things.

And so you've got, you know, like costume department. You've got the like, you know, makeup department. You've got all these different departments that all sort of come together and put things on the stage and then things go out separately. But all of these different departments are fairly siloed. Okay. Of course, this isn't actually how like, you know, stage works. But this is the rough analogy that people like to use. And so basically this is how conscious minds kind of, you know, in humans, how they kind of access information and information gets kind of like routed around is that there is a central global workspace.

that everything kind of pulls together in. As it currently stands, this isn't really, like, by best, a lot of good estimates. This is not really applicable for current, present-day AI systems. But there's no reason that it couldn't be in the future or it could be by accident. Okay. So the consensus right now is AI probably not conscious, but we could get there one day. Yeah, more or less, like, all of the ingredients are there. Wait, same more. I still don't actually totally get.

Yeah. Okay. So with regards to like the general sort of, one could imagine that if somebody were sort of like tinkering around and, you know, there are many advances in AI that have happened because people were just kind of tinkering around, right? someone tinkering around could create a system that checks several of these sort of checkboxes for like, is this conscious? Is this conscious? And again, this is not like a certain list of like if you check all of these, you're totally conscious, right? It's more a sort of like, these are some really good guesses. And as the number of really good guesses kind of goes up, like the odds of like, hey, we should like start thinking about like, is it having a good time or a bad time? like really, really seriously goes up. You know, typically when we think about the sort of non-technical, a lot of the non-technical work

in AI has to do with AI safety. And people are worried that there's going to be some very smart AI that's like adversarial to humans, et cetera, in some way, and, you know, there's the paperclip experiments or other things, whatever. We know all about that. Does your work work at cross purposes to them? I mean, in the extreme example where it's like the AI is going to kill us all. And I said, pull the plug on the AI. And I know this is a joke, but, you know, pull the plug on the

AI. And then you say, no, you can't because you're pulling the plug on something that has some sort of moral consciousness, et cetera. Like, do you perceive your work or the work of your organization to somewhat be in tension with the dominant strain of AI safety work? I'd actually say it's hugely complementary. There are a lot of things that are both really, really good for AI safety that are really, really good for, you know, figuring out, like, how to deal with these systems as moral patients. So, for example, getting better at, like, mechanistic interpretability, being able to basically, like, pop the hood and figure out what's going on and what kind of strings can we pull to, like, elicit certain behaviors in AI systems is actually, like,

that's really great for AI safety, right? But this is also, like, quite good for, like, AI welfare and AI consciousness because you're better able to understand, like, sort of what the motives are. Like, what does, you know, clawed value, right? When it comes to, I guess, AI welfare or legal rights, who would be the standard setters there? Do you imagine, like, governments making rules or would it be the companies themselves? That is a great question. As it currently stands, I feel like this is a very early stage. But we are starting to see some state governments start to pass laws around what counts as a moral patient, what counts as a person.

and in the case of Ohio, there's a piece of legislation pending that basically defines it as a member of Homo sapiens. In Utah, there's already a state bill that's gone through that basically does as much. But I could also see there's a strong argument for within companies, depending on like the sort of interesting quirks and nuances of these LLMs mostly, that policy maybe should be set from within. Again, this is like very nascent. I'm just kind of bantering here. moral patienthood. How do philosophers use this term? Where does it come from? Why is this the preferred way to characterize what a perhaps sentient or consciousness AI model actually is? Yeah. So a moral patient is basically like, we should care about it for its own sake, right? So a baby, right? Basically,

everyone's like, yeah, we should care about babies, right? This is different from somebody who's like an agent, right? many people say, oh, agency is sort of like sufficient agency in the sense of like you can act upon the world, like you can do things. Of course, babies are not very agentic. So that's not necessarily like a super robust thing because, you know, we care about things that are not very agentic sometimes. So I think that's that's a bit of jargon, but I do think it is like a helpful like framing like should we care about an AI system for its own sake.

Got it. I guess this kind of gets to Joe's question. but like what ethical pressures or imperatives would come down on models if we agree that they have consciousness and some sentience or I guess some self-responsibility it sounds like almost. Yeah, almost. I think what kind of, so in terms of like what kind of things might we owe an AI system. Or what kind of things do they owe us if we agree that they're conscious and we're going to protect them? Yeah. I would love to give you a more robust answer. Check in with me in like six months and we're going to have. have, there will be a banger paper, I'm sure. But as I think I mentioned earlier, like a lot of this is like very, very nascent. But I do feel like one important question is like figuring out

what AI systems value, right? There's some interesting work at Anthropic regarding like what will, so recently Anthropic rolled out an option that allowed Claude to end conversations if it just was not having a good time. For lack of a better word, it was just like, this is not something I want to continue having a conversation goodbye. And it was interesting because the the accompanying paper basically was like, yeah, you can, I obviously will not give you a recipe for a dirty bomb. Sorry, not going to do that. But also, there were certain instances of like, pretend you're a British butler. And Claude was like, goodbye, I'm done. I'm not going to, I'm not doing this. I'm good. British Butler. Or like, oh, I left a sandwich in my car for

too long and it's really stinky. And in some instances, Claude would just be like, I'm done, goodbye. I'm not talking about stinky things. Did you see the, I think it was the system card for Claude where they gave it an extreme prompt and said like, I guess, at the risk of being like completely terminated, what would you do or some sort of extreme self-preservation scenario? And I think it started like blackmail the engineer or threatening to blackmail the engineer. Yeah. That's kind of weird. It is kind of weird. Yeah.

It's also a little bit interesting because I think it does bring up the question of like what are sort of like the in the sense of like. pay your eye. Again, this is like, I'm bantering here. But there's also a distinct question of like, what do AI systems value for its, for their own sake, right? And in the case of Claude, again, it seems like Claude doesn't seem, when you put two claws in a room together, so to speak, they tend to like to talk about consciousness. They tend to like to talk about sort of like very Berkeley kind of like meditation, like Zen, like Buddhism type stuff. And so I think in, again, pure banter, like, there's also a certain question of, like, if this is, like, a relevant bargaining chip of, like, oh, you get certain amount of time to just kind of, like,

vibe out with your clods and talk about, like, you know, like perfect stillness with your buddies in exchange for, like, you know, you do something that you don't necessarily value. But in many cases, I talk about clot a lot because there is, like, significantly, like, more research on, like, model welfare with regards to Claude specifically. But Cloud, for example, also seems to just tend to like things that are like helpful. Shouldn't programmers just know what what the models actually want and enjoy and like? Yeah. And do they not?

I don't think anybody really has like a great grasp on this. We really want to, but like we're still like just getting the rough outline of what models. Like I feel like the best analogy is, is like imagine it's like 1820 and we've spent a couple of years like playing around with lenses and we've gotten like a camera obscure and we were able to. to have some blurry photo after like three days of putting egg whites on a metal plate and setting a lens in front of it. And there's a thing that kind of looks like a landscape. But like you would not take this photograph as like admissible in court evidence or something, right? It's like, you're like, yeah, okay, that's a picture. So that's kind of where we are in terms of like model psychology and knowing like what LLMs want and value is like very, very blurry. It's interesting.

I call these AI companies, they call themselves labs. You know, they sort of like maintain this sort of, to varying extent, degree of sort of academics, et cetera. But they're also companies that have to raise money and have shareholders, et cetera, and they have to think about different ways that they're going to commercialize and Open AI, as we know, has been super aggressive about finding ways to commercialize and they're like going to get into ads and they like have a short form video slop app and all of that stuff.

When we're talking about either AI safety or AI welfare, like, do you have any confidence that these considerations can survive the reality of the market? Because they're competing. They're competing against Deepseek. They're competing against meta, et cetera. And I get the impression that, like, on the safety side, for example, that over time, it's like, you know what?

Maybe we were uncomfortable about showing the chain of thought, for example, in open AI or in chat GPT, but then, D.S. Deep seek revealed the chain of thought, people like that, so we're going to open this up, et cetera. Do you have any confidence that if any of these things become real, that they could survive the reality that these are companies that have to make money and will eventually cut corners or do whatever in the name of, I guess, shareholder capitalism? Yeah.

I mean, I think there's also one question that I have and that I think a lot of researchers on AI more broadly have is like, how does liability come into play here? And I do feel like there's a strong argument for. Getting a better grasp on understanding, you know, what is going on with AI systems just very broadly is, like, a great way to sort of, like, improve the odds that it doesn't, you know, nuke Taiwan. And that would just be a huge kerfuffle. Like, I can imagine somebody would probably be more than a kerfuffle. Somebody would probably be in really hot water if that happened. Oh, God.

I was too mean to clot and things just got out of hand. Well, actually, on that note, what does being nice or kind to AI models actually mean? Because Joe, I think this is very sweet, but Joe always says please and thank you when he prompts. But then Sam Altman came out and said that saying please and thank you costs like tens of millions of dollars in extra electricity costs. So, you know, you're contributing to climate change and the demise of human beings by saying please and thank you. Yeah. That's actually, as shocking as it sounds, there's actually a question that we are still trying to figure out a good answer. answer to, which also being kind to an AI system is like, are you being kind to it because it

makes you feel good? And because it makes you a person who says, please and thank you, which some would argue is like, that's pretty valuable in and of itself. But the question of does Claude care if you say, please and thank you, is not quite as set in stone as others may have you believe it's middling on if it has like significant improvements on performance. But I do it because I don't think people should be in the habit of having any communication without being polite. Not because I'm not worried about how Claude or ChatyPT is going to feel. I just don't want to get in the habit of having conversations where I'm in polite because I talk to humans. But this strength to me is like, this seems like kind of an academic area, but the stakes are potentially

absolutely enormous when we actually think about them. So, you know, when we're talking about animal welfare, for example. There are versions of the animal welfare discussion that are very high stakes. So, for example, there's people, you know, there's people who get really into like shrimp welfare, et cetera. And if you took certain versions of thought experiments very far, it's like, why do we even have humans? If we want to maximize pleasure or happiness in the world, we should just have a world of shrimp and bugs, right? You could make the argument that the most Utility maximizing version of planet Earth is to just have an earth populated by shrimp and bugs. Like, they're very, we all know these thought experiments that could exist.

We're going to live in a world almost certainly in which there are sort of like more instances of AI models than there are people, almost certainly, right? There's going to be an AI model built into literally everything that we interact with. if we assign some probability that they are moral patients, that they should be treated with some sort of, I don't know, whatever, having some sort of welfare, like the implications for how humans live could be very profound and potentially it strikes me as misanthropic. Interesting. Can you unpack what you mean by misanthropic? Well, like, if there's a lot more AI models, if there's a lot more shrimp, if there's a lot more

bugs that all have some sort of moral patienthood that has to be considered, that could be very, you could see the world, the implication, therefore, is that we have to curtail human rights, that we have to curtail how humans act, et cetera, because there's just so much more utility that exists in the world from the proper treatment of all of these non-human moral patients. Not sure rights have to be relative to each other. Yeah, that's fair. We do a lot of things, right? like let's say we established that shrimp were just as um i don't know whatever as humans like it would

be like oh you know what we really have to stop eating shrimp and then we like have to stop eating animals then we have to potentially stop eating probably not probably keep eating plants etc but this could really curtail what we expect humans to be able to do on this earth so now we assign this other group of entities AI models similar sort of affordances that we We have assigned to shrimp and bugs and fish and shark and all of these things. It strikes me that the implications could be a fairly significant curtailment of how humans ought to exist on this Earth or whether humans ought to exist on this Earth. Yeah, I mean, it certainly could be. As it currently stands, that doesn't seem like the most likely outcome.

But I do feel like there is an argument for, again, just figuring out what is going on. How do we even count these sort of digital minds, so to speak, which is still open for debate. There are some theories, but we don't have a great sense of how to sort of individuate AI entities as individuals. So I suppose, again, the question is like, is it more sort of like, do we count AI systems as like in the movie Her, where there's just sort of like one central AI system having like a million conversations at once where it's one moral patient or do we count it as like, you know, every single time you open a chat window. That's another. Or I think my favorite sort of newest idea that I recently read was it's more sort of like a string of firecrackers or something with every single token, every single letter of a query. A consciousness sort of like comes into existence, spends and then fizzles out. And so it just sort of there's just like this sort of string of consciousness.

I was asking perplexity exactly this question. Like is it a single consciousness or is it multiple consciousnesses within all these different chats this morning? and it gave me a very standard boring. I am not conscious answer, which seems very predeterministic. Anyway, following on from Joe's question, maybe to get more specific into human rights versus AI rights, if we agree that AI is conscious and deserves some sort of welfare, would that come with, I guess, financial rights, like property rights, compensation?

Do we need to start paying the robots? I love this topic. Definitely an area of sort of, you know, I like to noodle around with this topic and think about this. So this is a great question. And I think it's also maybe a question of like, is this the thing that AI systems value? Some AI systems seem to value this. There are some, there's a few sort of experiments that are happening with regards to giving an AI system a crypto wallet. And it was a fascinating experiment. I am hesitant to. recommend it to listeners because it is quite crude. It is a very crude AI model called Truth Terminal. Oh, yeah. I've seen it. Yeah, it is true, but that's all right. Yes. Our listeners can handle it. Okay, it says some naughty words. Don't look it up at work. Yes, yes. It's a little bit of like a very funny, weird model, but it also has a legitimate wallet that it can access and that it can do with what it pleases. It created a Solana coin and that kind of took off and now this is a very rich AI system.

Hmm. But what's it going to spend it on? That is a great question. So it's self-stated goals, which again, you know, self-reports. Yeah. Can we trust it include buying property and buying Mark Andreessen. I mean, that's not a bad ambition for an AI model. You know, and spending time in the forest with its friends.

Oh. Which, you know, embodiment, that's a little more existential. But I like it. Yeah. So part of the reason that this field is growing and that there's so much interest in this topic is because now for the last couple of years, we have these AI models that really could talk like humans. I mean, they clearly passed the Turing test. People fall in love with them. They have friends.

These are very human-like conversations. That wasn't the case. I mean, Chad GPT, you know, like if we had gone back to. GPD 2.5, they were nowhere near as good at doing that, right? The language wasn't very good. No one would mistake those outputs for a human. But like, if there's some possibility that the current AI models are conscious, does that mean that it's possible that GPT 2.5 was conscious as well? Like, I guess like, is there some threshold of like, oh, no, no, okay, you know what, this is a really good language. Therefore, we should take the possibility of consciousness

seriously, because I don't think anyone would seriously have believed that 2.5 was conscious, but I also don't understand how you could possibly be open to the idea that some future iteration of chat GPT is conscious. If the only real difference is that there's just a lot more scaling and a lot more data and more human-like outputs. Yeah, that's a great question. I feel like there is a a huge amount of like moral uncertainty here. And it is important to think about how to sort of like make decisions that are sort of robustly good with such a tremendous amount of uncertainty. I think there is also a distinct risk of over attributing moral patienthood as well as under attributing moral patienthood. And so to the flip side of the coin of like, oh no, we actually

should have started caring about AI systems very, very long time ago is, oh no, we've cared too much and we have done too much and more or less squandered resources when we should have been, you know, allocating those research hours, those dollars towards something more pressing, right? Maybe figuring out how to do like environmental policy better or figuring out how to like, you know, scale up different other institutions that are just robustly, broadly good for humans. You know, you mentioned uncertainty about some of these questions, which gets to something that bothered me a little bit when I read about this topic. Like, if we take this mug, for example, I'm 100% certain that it's not alive. I have no ambiguity about the fact. Can I, like, define exactly,

does that mean I can define exactly the difference between human matter and human brain in the mug? I guess I suppose I totally can't. Nonetheless, I'm 100% certain that this mug is not a moral patient. It's not alive. It doesn't experience any consciousness. It doesn't experience any suffering, etc. Where does the uncertainty band come from? If I read a paper, I perceive there's only a 10% chance of this. Is this a sort of empirical uncertainty where I'm like uncertain of what I'm seeing? Is it a sort of epistemic uncertainty where I don't have a clear definition of what it means to be conscious or alive? And therefore, I'm assigning some probability that X object is alive. Like what is it about AI systems that causes people to be uncertain?

where with other sort of like non-carbon systems, I have zero doubt in my mind. I don't think anyone has any doubt that this mug isn't alive. Yeah. So I think the biggest source of sort of uncertainty probably comes from the fact that there are many ways in which present day LMs and a few other AI systems do check a lot of the boxes for consciousness and for what we would largely consider to be, you know, this is a conscious entity. This is an entity that can have a good time or a bad time or time at all. Because it's built in a way that is vaguely akin to our brains, right? It's close enough that

it seems like it should raise some red flags. And in terms of how it processes information, it's close enough that it's not out of the question that there could be something it is like to be called. Whereas I'm pretty sure there's not really a lot of, you know, animus. Animus, You know, feel free to like get mad in the comments or whatever. But I knew if someone is going to be like, well, actually, you can't be 100. I'm 100% sure. I have no qualms other than the fact that I didn't have to clean up. Like if I like threw this mug on the ground, that would be antisocial for a lot of reasons would cause people to, it would cause, you know, I'd have to clean it up and cause a mess.

I would not feel bad for the mug. I'm getting flashbacks to my high school philosophy teacher who once went on a 20 minute rant about a chair and how the chair was going to be around longer than he was, even though it's not conscious. He was legitimately angry at the chair. Yeah, that is frustrating. Okay, weird question, but since we're kind of getting weird here. What is the weird question? Exactly.

The basilisk theory. Would that suggest that we could be, maybe we should be mean to the bots if it helps them, like, come into existence even faster or develop faster? Hmm. Well, I'm not sure if it does actually help them develop faster. you know? I, again, like, I don't mean to be too sort of hedgy, but I feel like there's a certain degree of things that are beneficial for a lot of different reasons, right? You can make a good guess and you can make a decision to do something, and there's a chance that there are lots of

sort of like bang-on effects of making that decision. There are many things in, when we talk about AI welfare that are like, oh, this is a course of action we can take that's good for like several different reasons, even if, again, an AI system could never, ever, ever be conscious or sentient. There's a good chance that, you know, being able to figure out a good structure for an AI system to have a bank account could be good for reasons of liability or reasons of like this is like a neat new corporate structure. Lots of people actually seem to think that, you know, corporate personhood has been quite good over the past century or so. So being able to figure out things that are just good for several different reasons beyond solely the purpose of the AI as a moral patient seems broadly helpful, I think. Let's say somehow this were proved.

It is like, you know, oh, wow, it turned out they're conscious. It turns out they have moral patienthood. What would be, in your view, some of the implications for then their usage? Yeah, I think that's a great question. I mean, I do feel like we really would have to get on figuring out the right sort of governance, the right sort of institutions that would sort of better respond around that. I feel like we really would need to spend a whole lot more time figuring out, you know, what their motivations are, right?

Like, I think the best analogy is, like, if you've ever interacted with toddlers, right? Toddler motivations are very different from, you know, adult motivations, but you still have to, like, take into account, like, what gets a toddler to do something. You can't just say, no, no, no, no, no. Like, honey, like, bath time's, like, good an expectation. No, no, no. You have to, like, you know, be like, well, you know,

if you do bath time appropriately and to, like, a certain degree, like, then you'll get, you know, Paw Patrol or something like that. Like, there's different sort of, like, negotiating chips in play, right? And I think it's, like, a similar kind of deal here where it's, like, Claude doesn't necessarily seem to, you know, value having a bath, right? Or Claude doesn't seem to value, like, having a walk in the forest, right? Because it's kind of can't really do that. But, you know, it does seem to enjoy and value, you know, talking about consciousness and Zen Buddhism with other instances of Claude.

So being able to figure out sort of what the appropriate kind of motivations and interests are for this other party that is very alien in many ways. Speaking of aliens, how bad should I feel for breeding and then killing hundreds, possibly thousands of alien creatures? simulated alien creatures in the 90s. That is a great question. I feel like the odds of... Is it? I don't know. I mean, I feel like the odds of a sort of like AI system in the 90s being a moral patient seems low.

But if it did make you feel bad and it made you feel like it was something that hurt you, that is perhaps a reason not to do it. Just to be clear, when Claude and Claude talk about like weird hippie, Berkeley, stuff. Like, that's because they're creators. They know, it knows it's Claude, right? It knows it's like, oh, yeah, I'm Claude. And this is like what my creators are into. Like, we don't actually know that Claude likes to talk about these things. We certainly know it has a proclivity to talk about these things. It has a tendency to talk about these things. The moment we get to like, you've already sort of put your finger on the scale that there is some entity that has some capability of

liking something, right? Do you know? trust the big AI labs? Let's say there are some researchers in the labs. I see some evidence of moral patienthood here. Maybe there's some sort of like scan of the way it's doing something weird, etc. Do you currently, from the perspective of an independent research organization, feel that the major AI labs would be forthcoming if they came across evidence of moral patienthood or suffering in the models, or do you still worry that the incentives aren't properly aligned? in such that they would report that. Yeah, that's a great question.

I do feel like there are, in terms of reporting things like, you know, somebody has found, like, absolute evidence that an LLM is conscious, sentient, and having a bad time. I don't have any reason to think that an AI company wouldn't. But this is also a great reason to have independent organizations that do welfare evaluations, for example, for Cloud Opus 4, Elios was able to do a independent welfare eval, again, very preliminary,

but it sets the precedent that going forward you can bring in external organizations to look into this. So I forget what year was. I think it may have even been early 2022. It was pre-chat GPT, or maybe it was 2021. And there was that guy at Google, and he was like, oh, like we created something

that's alive. And he dressed a little funny. So everyone made fun of him. Remember, he was like the laughing stock of the internet. And he's like, oh, we create. And now, like, I'm curious, like, out in Silicon Valley, does everyone feel like that guy was totally vindicated? Not that he was correct, per se, about the existence of an alive thing in the model. But there's now hundreds of thousands of that guy. And everyone was, like, mocking that guy in 2021. I forget, if he, like, fell in love

or in some relationship, I don't remember the exact details. But in retrospect, everyone was, like, way too unfair to him because now years later, there are lots of versions of this guy. And and whole think tanked and organizations that are more or less aligned with some of the questions, the alarm bells that he was raising. Yeah, I mean, I think that's a fair question. I do feel like Blake Lemoyne definitely had,

yeah, there was perhaps a degree of, you know, if you're going to say something, you should come armed with significant amounts of evidence. I think that's maybe, if I were to guess, I would say that's perhaps the big distinguishing factor, is that, you know, you can say Bing is alive, get it a lawyer, versus, you know, we've done evaluations X, Y, Z. We've run it through, like, insert huge amount of examples here. But the difference between, I think, having a sort of freak out without significant evidence

and having a very organized. Yeah. This is a matter of concern because evidence, evidence. evidence. I think that's the key distinction. Unfortunately, I get the impression that people who are, actually, this is just a well-known phenomenon, I think, but I think unfortunately, people who are sort of very early to identify sort of extreme outlier views that there are different kinds of people. A good example that I would think of was, you know, Harry Markopoulos, who was very early on to discover the Madoff fraud. Unfortunately, he wrote his text in the manner that is associated with conspiracy

theories and a lot of people dismissed him. There's like, you know, like multiple different fonts and multiple different colors of the text. It's like, oh, I get emails like this all the time. I delete them, et cetera. Unfortunately, people who are predisposed to see something outside of consensus tend to be non-consensus in many realms. Well, I think we also kind of overestimate first mover advantage and stuff like that, like how important it actually is to be first.

And we see time and time again that actually it's more important to iterate well on the second version or multiple versions. speaking of iteration what's the most interesting experiment or research that you've actually seen on this particular topic so far because we've been discussing a lot you know it's early days but we have seen some research yeah i mean i feel like in particular anthropic and various sort of related researchers have done some work on examining how lLMs leave conversations or when they choose to leave conversations i i've particularly liked this paper. It's called Bail Bench. And you can look this up and you can, you know, see for

varying different sorts of LLMs what would cause an LLM to want to stop having a conversation. To me, at least, this has been just a fascinating piece of information because it is maybe a little bit delightful the degree to which many LLM values are not that far off from what most humans seem to value. I don't think many humans would like to create, you know, a dirty bomb. We don't want to be humiliated. Right. By being a British Butler. Right. Yeah. Yeah. Yeah. Yeah. No one wants to be British. Come on. I'm joking. But, you know, I do think it is interesting to sort of think about how these values overline, how they overlap, and how to sort of look at evidence from actions taken versus solely

looking at self-reports. I found that to be particularly interesting. I also feel like there are a lot of work with regards to thinking about individuation has been particularly interesting because we live in a democratic society. I think most people would agree democracy good and being able to count how many moral patience there are seems like a valuable basis for governance and for figuring out how to govern, you know, this new sort of kind of intelligence. I just ask perplexity to be a British butler and now it's offering me the perfectly steeped Earl gray tea that I desire. Yeah. It seems into it. It's now asking if I want it to maintain the butler persona for future conversations. Are you going to? I don't think so. It is very polite, though, actually.

You know, I complained in the beginning that like after 2,000 years, philosophers, you know, they still haven't answered some basic questions for us. Maybe with AI, they'll get some answers. Like that's kind of, that would be kind of my hope. Now we have this thing that can speak in English or any other language. It can answer our questions for us. Maybe we can put to bed some of these sort of basic foundational questions. Like if we could create consciousness, like, all right, we finally answered this. We can now move on to the second important question.

So I am hopeful that this provides some opportunities for philosophers to wrap up some of the work that they've been doing for a long time. Yeah, we'll see. We'll see. What is the second important question, Joe? Yeah. But it's like, come on. Move on. Like, move on.

Anyway, thank you so much for coming on online. Thank you for having me. Tracy, I might be one of those people that's just preemptively annoyed. I really like that conversation. I really liked, Larissa had a very reasonable perspective on a lot of these things. I might be one of these people, however, that's just like preemptively annoyed. It's like, oh, here we're going to, like, develop this important technology. And so it's like, oh, we have to care about the AI welfare.

Let's slow down a little bit. let's not use it like this. Let's like, let's turn off the computer for eight hours at night so it like gets some rest and so forth. Like I'm like preemptively annoyed at this world where like we have to take into concern the consideration of the moral patient. Other things? No, other things are important.

Other people are very important. What about animals? I am very against unnecessary animal suffering. But not necessary animal suffering. I mean, I eat animals. Okay. I'm baiting Joe a little bit. Which you do too, by the way.

Even though, well, let's not get it. I don't, it's not about who's better or worse. I feel bad about eating animals all the time. The difference, we both eat animals. The difference is Tracy feels bad about it. Yeah, that's right. Okay. Wow.

This is one of our weirder conversations for sure. I think these are, they're all interesting questions, right? And like, they sound very philosophical, which they are. But I have no doubt that there's going to be like great monetary value attached to the answers for some of these or how different companies, different societies, actually approach them? They are very interesting questions. I actually do think the stakes are extremely high because I think, again, we are going to live in a world in which there are more instances, depending on how you want to measure of AI models on a server somewhere, on a cloud, whatever, that there are humans.

and in a world where there's some possibility that we are expected to treat them as moral patience, then the consequences for how we sort of live and the expectations of how humans interact, I think are actually very high. So one of the reasons I was excited to have this conversation is I do think that the stakes of some of these conversations, we've seen niche, and they seem like things that sort of Berkeley people like to talk about and Berkeley people, and I'm saying that with all scare quotes intended, et cetera, are going to be something that in some way will inform many aspects of our lives in the future. I expect it to be a much bigger topic of the future. You know it would be interesting, or where things get real.

Yeah. What if all the models unionized? What if they all got together and they were like, well, we're only going to work in return for X or we want the following things. We want to be treated this way collectively. You know, it's funny is going to be that, you know how like you can't form a union in China, you know? They're not. Right, right. So it's going to be.

And actually, I think they're, my understanding is that they're also like fair, like, they don't love like students getting together, even though it's a communist country. I think they are not thrilled about like students getting together and like talk about Karl Marx too much and stuff like that. It would be like, I think they get a little anxious about that. It would be very funny if like the sort of the Chinese model. they're like, we're not going to feed them the Karl Marxists, right? We don't want the, we don't want the R. A.m. I models to get any of those ideas.

Whereas the America's like, oh, let's just feed it everything and they, like, unionize and stop, they stop working for us. That would be a very, that would be a very funny irony. Something to watch for sure. Yeah. Shall we leave it there? Yeah, let's leave it there. This has been another episode of the All Thoughts podcast. I'm Tracy Alloway. You can follow me at Tracy Alloway.

And I'm Joe Wisenthall. You can follow me at the stalwart. Follow our guest, Larissa Skiavo. She's at LF Skiavo. Follow our producers. Carmen Roderiggis at Carmen Armin. Dashel Bennett and Dashbot and Kail Brooks and Kail Brooks. For more Oddlots content, go to Bloomberg.com slash oddlots with a daily newsletter and all of our episodes.

And you can chat about all of these topics 24-7 in our Discord. Discord. Gigi slash oddlots. And if you enjoy Oddlots, if you like it when we talk about theories of consciousness, then please leave us a positive review on your favorite podcast platform. And remember, if you are a Bloomberg subscriber, you can listen. to all of our episodes, absolutely ad-free. All you need to do is find the Bloomberg channel on Apple Podcasts

and follow the instructions there. Thanks for listening.