Short Wave - Can you spot AI-written stories? Listen and find out

Episode Date: August 12, 2026

Can you tell the difference between a human-written short story and an AI-generated one? Most people can’t. In a series of experiments led by a team at Villanova University, researchers found two things: First, most people said they preferred human written stories, but couldn’t tell the difference between AI and human writing; and second, when presented with two similar short stories, one written by a human and one generated by AI, readers tended to prefer the AI-generated piece – especially if they had been told the AI story was written by a human.We excerpted two of the short stories on our website for you to guess which is AI-generated and which is human-generated. Check out the full set of stories here.Interested in more science and technology news? Email us your question at [e-mail] public media with NPR+ and enjoy perks for over 25 podcasts like this one. This show’s perks include sponsor-free listening. Learn more at plus.npr.org. See pcm.adswizz.com for information about our collection and use of personal data for sponsorship and to manage your podcast sponsorship preferences.NPR Privacy Policy

You're listening to Shortwave from NPR. Hey, short wavers, Regina Barber here with producer Burley McCoy. Hey, Gina. We are back with our biweekly science news roundup with Friend of the Pod, host of all things considered, super awesome lady, Mary Louise Kelly. Hey, hoo! Happy to be here. Hi. Hey, so our first topic is AI.

And Mary Louise, we're going to read two different short story excerpts to you, one written by a human. one written by AI. I can't wait. I'm going to nail it. Let's do it. Let's see who goes free. Regina, go first. Okay, okay.

In the stillness of the early morning, I find myself at the edge of a coy pond, a relic from my childhood. Watching the fish swirl and patterns, only they understand. Their vibrant colors, a mosaic of oranges, whites, and blacks, mirror the autumn leaves that dance on the water's surface.

Each coy, with its graceful, unhurried movement, seems a silent sentinel guarding the mysteries of life and death. The pond nestled in the heart of my mother's garden has been a constant in my life. As a child, I would spend hours by its side, my mother sitting quietly beside me, her eyes always carrying a soft, distant gaze. She would tell me stories of her youth, of her dreams and fears, her voice weaving tales that made the past seem like a vivid tapestry,

full of color and life. I feel like that's a hard act to follow, Gina. Like, I've got to put on a voice, too. You really put yourself into that, Gina. Burling. Okay, story number two. Something is wrong with the fish in the koi pond outside my mother's Florida salient bedroom window.

There are three of them, small golden orange, medium marmalade, and large vermilion with black markings. Last time I visited for the weekend, I named them. in reverse order of their descriptions and with no assignment of gender intended, Kurt Russell, Goldie Hawn, and Kate Hudson. My mother's husband says he thinks one of them is about to give birth. Then he says the big ones usually eat the young, but that they're pretty good at hiding among the hydrangeas growing hydroponically in one end of the pond.

I haven't seen a new fish born here in some 10 years, but perhaps that's why. Maybe they're not so good at hiding after all. What do you think? Okay. Okay, so reviewing the assignment. We've just had two co-pond readings. One is AI. One is written by a human.

And I'm guessing which. I'm torn because I liked the second one better. I just got like an image in my head. I see Goldie Hawn swimming around. So I want to go with that. I'm wondering a little bit the no assignment of gender intended. It feels like something AI would have been programmed to be politically correct about. So I wonder if I'm wrong.

But I'm going with number. Number two, Burley, the one that you read. Is AI or is human? Is my favorite and is human written. Okay. All right. Well, you will find out after the break. Wait, I don't get to know.

I was such a tease. Well, just to tease you a little more, today on the show, you'll also hear about a newly discovered wandering supermassive black hole and how Pumas could make roads safer for humans. You're listening to Shortwave, the science podcast from NPR. Okay, Mary Louise. earlier we read you two stories and you guessed which was written by AI. You guessed the second one was the human and the first one was AI.

And now I'm going to reveal to you. This is like a reality show. Yeah. I feel like I'm about to get like dumped into a shark tank. You were right. You should have seen her face. She's so happy. I am so happy.

I have a question though, if I'm allowed to ask. Of course. Those were excerpts. Yeah. Just like, I don't know, a couple paragraphs. She'll just read me. Like not a whole story. What was the fuller content that participants read? So we're talking short stories. So this paper that was recently published in the journal Judgment and Decision Making included a few experiments. In the first one, participants read one of several short stories, some written by people, some of them AI generated. Then they rated them. But here's the catch. Participants were either correctly or incorrectly told that their story was written by human or AI.

In the other experiments that these researchers did, participants got. a pair of stories. One was human written and one was AI generated and they were asked to identify which was which. And these researchers found some key things. First, most participants couldn't tell which stories were AI generated and which were human written. And second, they tended to prefer AI stories especially if they thought those AI stories were written by humans. Ah, okay. So what people said they preferred human writing and what they actually preferred was often different. Yeah, yeah. So this fits with a growing body of research suggesting that people's beliefs about air involvement in a message influences how they feel about that message.

Expectation bias. Exactly. So Dina Weisberg says that it's a sign that large language models have gotten a lot better over the past few years. I personally feel confident in saying the time has passed when it is simple to just be able to tell what a human has written and what an AI has written. Dina's an associate professor of psychological and brain sciences at Villanova University and the senior author of this study. And now she did caveat that these results might be because of the length of these stories. So they were all around a thousand words each. And when we talked to some other experts in the field who have done similar research, they emphasized that length is often a key factor in our being able to detect AI generated content.

We also talked to Mohit Ayer, an assistant computer science professor at the University of of Maryland. And he wasn't involved in the study, but he told us that because AI-generated content tends to be pretty formulaic, the longer they are, the worse their quality is. AI got nothing on Tolstoy. Okay, but no. Go back to the study because I'm trying to figure out why. Why do people prefer AI-generated content? Yeah, Dina had this hypothesis. The AI-written stories were very obvious and I think more predictable. in their flow, which isn't surprising given how AI generates text. And the human written stories were more difficult to read.

They were more subtle. They didn't spoon feed you the themes in quite the same way. In a way, they're easier stories to read. Dina and the rest of our research team have made those stories available so you can read them yourself. The human written ones and the AI generated ones. We'll link them in the show notes. Okay, so brilliant, Gina. You said, topic two is wandering supermassive black holes.

What even is that? Yeah. So super massive black holes are these objects in space that are millions of times more massive than our sun. And all the ones, except for two that have been found, are in the center of galaxies, hence wandering. This is one of those two. Okay, but don't worry, Mary Louise, this wandering black hole is 750 million light years away from us. Okay, so we got a little bit of time. But if it's that far away, how did they find it?

It ate a star. It ate a star? Yeah. Poor star. Yeah. And now I'm nervous again. No, no, no, no. Don't be.

It's fascinating, really. Because it's most likely not really like wandering, like you say. It's not going to come towards us. It's really probably orbiting a very, very far away galaxy. Usually black holes are invisible unless they're feeding on something. So the way scientists detected this one was by observing a flash of light created when the black hole ripped apart and ate a star. It's like this little black hole holds up a flashlight. And it's like, here I am. Look at me.

So that's Jillian Rastinashad, an author on the paper that came out in the astrophysical journal letters about this black hole. And her colleague Robert Stein at the University of Maryland trained an algorithm to recognize the pattern of these special light signatures that indicate that a supermassive black hole ate a star. They are very rare, but we started to find them in the centers of galaxies. And what we did was we trained the AI algorithm to recognize this very characteristic pattern of light. Regina, stay with that point he just made, that these are very rare. Why have scientists only found two of these black holes wandering? Wandering. Wandering.

Yeah. So that flash of light is rare, but on top of that, we were only looking for those flashes of light in the center of galaxies. And it kind of blew my mind that astrophysicist, we never thought to look at anywhere else until Yale theoretical astrophysicist, Prim Vada and Natarajan and our team proposed the possibility of wandering supermassive black holes. and how to find them with this special light signature. And she says this discovery is a huge leap forward to learning more about black holes, which are still pretty mysterious. But it looks super exciting, right? So you could find dozens and dozens per year.

That's our prediction. Let me drag us back to Earth. Yeah. Topic three, which is how Pumas could make roads safer for human drivers. And we should say a Puma, that's what? It's a mountain lion, right? Right. Or they're called cougars sometimes too. Okay.

So here, the researchers looked at Puma movement in the Olympic Peninsula in Washington State. And they found where these big cats were most active, the number of vehicle deer collisions were 76% lower than in areas where Pumas were least active. 76% lower. That's huge. Okay. Vehicle deer collision, 76% lower. Is that because why? Because Pumas are eating the deer? There's fewer to hit. So I thought that too, but it's not that. So researchers controlled for deer numbers and still found this huge difference in collisions.

And the team says it's because the Pumas are likely changing deer behavior, both by prompting deer to be more active during the daytime, which makes them more visible to people driving versus in the nighttime, and by pushing them to more remote places away from roads. That's interesting. Why are Pumas pushing them away from roads? So the researchers think is because roads break up the landscape and make for really effective hunting ground. grounds for pumas. So think about like a road by a forest. The road is essentially a place the deer would be out in the open. Meanwhile, a puma could be hiding in the forest waiting to ambush them. So the deer avoid it, opting for more secluded areas. And this study was recently published in the journal Current Biology. I'm thinking about applications. I'm guessing y'all are not about

to tell me we should just release a bunch of mountain lions to make our road safer. Yeah, not quite yet. or maybe ever. But lead author Justin Sarachi from Conservation Science Partners said it could add to the conversation some people are having in places where Pumas are starting to come back naturally. And, you know, one response to that could be, it's a predator, get it out of here. But another response could be to allow that natural recolonization to occur. And I think this at least provides an argument in favor of that. Otherwise, Puma's coming back could be a hard sell to folks who may be worried about safety or hunting competitors.

Well, with that, I will say thanks to you both so much for having me, Gina and Burley. Thank you. Congratulations on being right. It is my honor. I wear my crown with humility. You'll never forget this. You're going to hold on. Never. You can hear more of Mary Louise Kelly on Consider This and PR's Afternoon podcast about what the news means for you. And for more stories, just like this one, follow a shortwave on whatever app you're listening to. This episode was produced by Hannah Chin, Juan Diego Ramirez.

and Jordan Marie Smith. Han also contributed reporting. It was edited by Rebecca Ramirez and Patrick Jaron Watananin. Tyler Jones, check the facts. Ted Mebane and Kwayne were the audio engineers. I'm Burley McCoy. And I'm Regina Barber. Thank you for listening to Shorewave,

the science podcast from NPR.