99% Invisible - RoboUmp Hits the Big Leagues

Episode Date: April 14, 2026

One study from 2018 found that Major League Baseball umpires blow about 14 calls every game. That’s 34,000 bad calls every year. And it makes a difference. A blown strike call can decide a win or a loss, a championship or six months at home, wondering what could have been. And while umpires are about 97% accurate in calling balls and strikes, Major League Baseball has been considering something drastic. Something to take us up to 100% accuracy. They have a plan to replace human umpires with robots. Now, with an update! Subscribe to SiriusXM Podcasts+ to listen to new episodes of 99% Invisible ad-free and a whole week early. Start a free trial now on Apple Podcasts or by visiting siriusxm.com/podcastsplus.  Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

This is 99% Invisible. I'm Roman Mars. And it's baseball season. If you haven't watched a game in a while, it can be a little confusing to jump back in. Major League Baseball has introduced a lot of rule changes in the last few years, including a countdown clock for pitchers to make the game go faster. They've also made it a rule that pitchers have to show their hands to the umpire between innings to make sure that the players aren't using any gooey substances to get an extra strong grip on the ball.

But this season, the one rule change that everybody is talking about is the introduction of robot umpires. Starting in 2026, if a player doesn't like a ball or a strike call, they can appeal to a robot arbitrator called the ABS System, who can overturn the call. And that one's going to be challenged. Body gets it right. Reverse from a strike to a ball. You may remember the name ABS System, because.

because we talked about it back in 2023. Today, we're going to replay that episode, and then I'll be joined by producer Chris Perube, for an update on how the robot umpires are doing now that they've reached the major leagues. All right, play ball. If you're a baseball fan, you might remember the 1997 playoffs.

That's when pitcher Levan Hernandez was unstoppable. Can Levan Hernandez get out of it? Yes, he can. Hernandez was a rookie for the Florida Marlins, and his masterpiece was game five of the National League Championship Series against Atlanta. That's producer Chris Barubei. Levan Hernandez struck out 15 batters that game, which for context, is so many batters. Strike three is 11. It was an incredible night, but a lot of his strikes, they weren't actually strikes.

Hernandez was pretty consistently missing the zone. The three-two pitch. Punches him out on what McGriff thought was born for. It's his 15th. This pitch is, I would say, a foot, two feet outside of the strike zone. Not close called a strike. That's baseball analyst Katie Nolan. She vividly remembers that game because it really was not good.

I mean. Okay, second pitch, way outside called a strike. Egregious. Egregious. Katie and I rewatched video from that game with, let's call it, a perverse fascination. Almost none of the batters actually swung at his pitches. You see, Hernandez just winding up and throwing ball after ball like a foot outside the strike zone. And then, inexplicably, the umpire Eric Gregg, he just kept making the hand signal for a strike.

It was so bad. It was probably the worst umpiring I can remember. The outside of this strike zone, it just didn't end. It was like a never-ending strike zone. In case you don't know anything about baseball, in the major leagues, there are four umpires on the field, one behind each base and one behind home plate. The home plate umpire has the most important job, which is calling balls and strikes. A strike is basically any hitable pitch, something over the plate between the batter's chest and his knees, and a ball is everything else.

I remember watching games as a kid, and whenever an umpire blew it, I would say, I could do better than that. And so I tried. I was a little league in high school umpire from the age of 14 until my early 20s. And I think I could have gone pro if it weren't for my poor eyesight, my aversion to getting yelled at, and the time I was hit in the throat by a baseball. My point is, even in the Little League's, getting calls right is a lot harder than it looks. And at the pro level, the baseball is moving at like 95 miles an hour.

It's kind of incredible that on average, umpires get it right about 94 to 97% of the time on strike calls. And umpires are getting better. The worst umpire today would have been upper tier in 1997. But the crazy high speed of the baseball means sometimes umpires are going to get it wrong. It just feels to me like it's asking a lot of the umpire to be able to recognize if it nickedy inside of the strike zone on its way over the plate or if it didn't. And I know we all make fun of the egregious calls, but I feel like some of them, you're not standing back there. You're not having to do it entirely with your eye.

It's got to be really difficult. One study from 2018 found that umpires blow about 14 calls every game. That's 34,000 bad calls every year. And it makes a difference, like in the Levan Hernandez game. The Florida Marlins came out on top, and a few weeks later, they won the World Series. These calls can make all the difference between a win and a loss, a championship, and sitting at home for six months, just wondering what could have been if he'd only made the right call. Given the human fallibility of umpires,

Major League Baseball has been considering something drastic, something that would take us up to 100% accuracy. They have a plan to replace human umpires with robots. Like any scenario where a human being is being replaced by a robot, there is the question of whether robots can do a better, more accurate job. And in baseball, a sport that is legendary for its quirks and its general human imperfection, there's another trickier question.

Is more accurate what we actually want? The idea of replacing an umpire with a machine isn't new. In the 1950s, the Brooklyn Dodgers tested a robot umpire designed by General Electric. The GE umpire was a big machine. It kind of looked like a barbecue hooked up to a specially tricked out home plate. If the ball cast a shadow over the plate, the machine would light up a big red button indicating a strike. The trouble is the machine didn't work very well.

It made a lot of bad calls. And if it was a night game, the robot umpire just didn't work at all. In the 1950s, the technology just wasn't ready. And the robot umpire went nowhere. For years, the idea seemed like a non-starter. But a few years ago, the commissioner of Major League Baseball, Rob Manfred, said he was considering robot umps for the big leagues. The robot umpires of the 21st century are a lot more sophisticated than a moment. barbecue. But modern robot umpires, they aren't technically robots.

That's what a lot of people picture is, is like, you know, beep, boop, boop, kind of a metallic thing behind home plate. What it really is, it's this system. That's Zach Helfend. He's an editor and sports writer at The New Yorker. And today's version of the robot umpire is actually a series of HD cameras. But for some reason, the name integrated camera baseball tracking system has never caught on. So for this story, we're just going to keep calling them robots. I prefer robo. I just think robo sounds better.

Baseball won't be the first sport to use robos to referee games. In tennis, there's a tracking system called Hawkeye that can pinpoint whether a ball is in or out of bounds. And it was good. What a shot. And in soccer, they use motion tracking cameras to help determine off sides and whether the ball has crossed the goal line. And the goal has been disallowed. In fact, most Major League Baseball stadiums already have a sophisticated ball tracking system in place. Those were installed in the early 2000s for TV broadcasts to give fans a clear picture of what happened during every pitch.

To track things like exit velocity off the bat, how fast the ball is moving off the bat, spin rate. It counts every single revolution of a baseball from when it leaves a pitcher's hand to when it gets to the plate. So they have these very sophisticated missile tracking systems, essentially, in ballparks. By the way, he is not exaggerating. This is based on missile tracking technology. If you've watched a baseball game on TV, you've seen this tracking system in action. In replays, broadcasters will show you charts and scatter plots to lay out where the ball landed inside the strike zone. But the umpires, the people actually making the calls, don't have access to this information.

Only viewers do. which creates some awkward moments for fans. Wait, if we know that's a strike, why is he calling it a ball? It just doesn't make sense. Why doesn't he have the information I have? He should make the right call. But look, baseball is a pretty conservative sport. It's slow to embrace change.

So for now, robot umpires are being tested on the minor leagues to work out some of the kinks and to help fans get used to the whole concept. Since 2019, robot ump technology has been working its way through the minor leagues, where it's called ABS for automatic balls and strikes. Last year, the ABS made its way to AAA, the highest level of minor league baseball. I wanted to see this robot umpire, okay, ABS system, in action. So I bought a ticket to watch some minor league baseball last summer

in fabulous El Paso, Texas, where the hometown chihuahuas were taking on the Albuquerque isotopes. But I got COVID, so I had to watch the game at home. The field and the isotopes. in position. Riley Smith will start his last eight final warm-up tosses before we get underway with tonight's game. Coming into the game, I was worried the baseball experience would feel totally different without the umpires, because for me, they're essential to the fabric of the game. But actually, I didn't miss the human umpires, because they were still there. Fans here are tonight's umpires. Behind home plate is Dylan Wilson.

down the first baseline, Cody. For those you worried about robots coming for human jobs, at least in this case, the humans are safe. Baseball still needs humans for lots of important jobs, like calling timeouts or cleaning home plate with those tiny, adorable brooms. This robot umpire was actually a collaboration between the ABS system that made the call and the human umpire who said it out loud. I listened with the earpiece along to a minor league game, and it's more or less instantaneous.

The ball hits the glove. You kind of hear the smack of the ball in the glove, and a split second later, you hear strike or ball. And it's funny, the strike is very peppy and, you know, sounds very encouraging, and the ball is ball, kind of disappointed. It's a man's voice just saying ball or strike. ball strike ball strike that's Fred DeHesus he was actually the first umpire to use the ABS system in 2019

fun fact his earpiece is now part of the collection at the baseball Hall of Fame I obviously couldn't get there as a player so I made my earpiece made it my joke is six Puerto Ricans have made it and one Puerto Rican's earpiece has made it

Fred says at first he was wary about the ABS, but he came around pretty quickly. You know, when in Rome you do what the Romans want, they wanted you to follow the system, you call it. I know this collaboration sounds, you know, a little ridiculous, but watching the game, I was pleasantly surprised. It was pretty smooth. It didn't look like a game umpired by a sophisticated missile robot.

It just looked like a regular afternoon at the ballpark. And so here's Bernard, right-handed hitter against the lefty groom from the full wind-up first pitch. There are no publicly available statistics on the accuracy of the ABS system. But anecdotally, Fred DeHeses says it was pretty damn good. It was very accurate. There were times where you would go, ooh, but again, you did what that machine wanted. There's no dispute here. The ABS is more accurate.

than a human umpire. Fred says there were some minor glitches when he used it, but nothing that can't be worked out by the time the system reaches the major leagues. The accuracy thing is huge because there's just so much money on the line. A bad call at the wrong time can ruin a player's career. And sports betting is such a huge industry now. I get why the major leagues want a more accurate system. But a few days after watching the robot umpire in action, my doubt started to be. to creep back in.

Because accuracy isn't everything. Here's Zach Helphund. I don't think most people watch sports to see the fairest or most accurate outcome. For me, the argument comes down to efficiency and accuracy versus charm and drama and dialogue. The thing is, for more than 100 years, Baseball has been played by humans and umpired by other humans.

And in that process, we've introduced lots of small quirks and inefficiencies. For example, baseball stadiums don't have standard dimensions. So a home run at Fenway Park might just be a long flyball at Dodger Stadium. Baseball just has all these unstandardized things. One of them is the application of the strike zone. Again, the textbook strike zone is supposed to be the player's chess. to their knees over the plate. But most human umpires don't exactly follow those guidelines.

There's lots of pitches that are considered hitable that don't land inside the textbook strike zone. And human umpires usually call those strikes. But the robot umpire, they've been slower to pick that up. In 2019, the ABS system was introduced to the Atlantic League, and it was programmed to call the textbook strike zone. But most fans and players thought the system felt, Off. The robot was calling a lot of hitable pitches as balls. So when the strike zone is, you know, so coldly unchanging, that sometimes presented some problems. When the strike zone is smaller than what you're used to, games can drag on.

Zach Elfin says the league needed to reprogram the ABS to be less accurate in how it called balls and strikes. They expanded it to about maybe an inch or an inch and a half off of the play. counts as a strike, and that better represented what the real strike zone is. You can program the ABS to call a less accurate game, but you can't program it to do all these other things that human umpires just do instinctively. So I'm going to let you in on a little dirty baseball secret. Umpires are constantly changing the strike zone based on context. It's raining. Let's move this along. Let's get this over with. Or one team is up by a lot. Let's just go. home, when a pitcher is struggling, there's a demonstrable effect that the umpire zone gets bigger.

Sometimes it gets as much as 50% bigger. That's what they call the compassionate umpire effect. So a pitcher's having a really tough time, we're going to help them out. And they don't do this consciously. When you leave it up to the machines to decide balls and strikes, you're ignoring years of training and experience and intuition that every great umpire has. And you're taking away one of those small imperfections that makes, basically, baseball kind of romantic.

There is a trade-off because you do lose this discussion. You do lose these quirks, these injustices, these twists of fate where someone blinks or gets dirt in their eye and they make a bad call and that changes everything. I want to see how people react to that. We watch baseball to feel something to divert ourselves. And sometimes it's nice to feel righteously mad against an empire or to feel like you got away with something. Okay, but let's talk about righteous anger for a minute. Because Zach is totally right. Yelling at the umpire is a part of the game.

Umpires get yelled at by fans and players and mascots pretty much nonstop. Because unlike other sports, baseball centers the umpire. The umpire is right behind the plate making judgment calls on every play. And usually the yelling is fun and cathartic and professional empires can handle it, but it sucks to experience that. Here's Katie Nolan. Imagine going to work knowing you could get a shard of wood directly into your face, or you could get hit by a 100 mile per hour projectile in the face on a bad day.

And on a good ceiling of this job is like you make calls that get people to tell you that you suck at your job and you're the worst and you ruined the game. You know, I've got a video on Instagram right now that's got over 3 million views where the player is saying, Freddie, you're the worst umpire in the league. How did they mic you up? Now, he's obviously joking, but this is what the world wants to hear. They want umpires to be ridicule. And it's not just ridicule.

There are stories of umpires receiving death threats or even being physically assaulted by fans. Tonight is Staten Island parent coach is accused of punching the empire so hard it left him with a broken jaw. CBS2's Lisa Rosner spoke with friends of the umpire. People get carried. away and it can get scary.

I remembered this Littly game where I made a really bad third strike call and after the game, a coach was waiting to yell at me in the parking lot. The abuse is actually the primary reason that I stopped umpiring. And it's why my favorite thing

about the robot umps isn't their accuracy. It's their ability to bring down the temperature. Zach Helton noticed this too when he saw a robot umpire in person. fans were a lot less likely to get into arguments when they knew it was a machine making the calls. Some fans who, as fans do,

and as is part of the pleasure of baseball, were heckling the umpire when I was out there. At one point, one of the fans who did know that they were using robo umpires this season in that league pointed up at the hardware above home plate and said, you know, it's not the umpire. This is just the strike zone. and the fan was humiliated in a certain way, very humbled,

and was like, you know what, it's actually calling a pretty good game. Watching the Isotopes Chihuahua's game, I remember this one at bat. So the Isotopes third baseman Taylor Snyder was at the plate. Count still one in two. Bases juiced. Here's the pitch. Takes a cult third strike breaking ball inside corner. That ends the inning.

Isotopes do not score. So the batter, Snyder, he disagreed with the call. He thought it was inside, and he was clearly furious. He starts to turn towards the ump, and it looks like he is ready to yell. But then he didn't. He stopped himself. And he walked back to the dugout. I'd never seen that before.

And for me, that's a big plus for the idea of robot umbs. Ultimately, the robot umpires are coming. They're going to be used in all AAA games this season. Some games will use a full robot umpire system, while others will use the robot umpire as an appeal system if the player doesn't like a call. Robot umpires are probably going to show up in the major leagues in the next couple of years. And I know baseball purists are going to be really mad. I get that.

I don't love the general idea of robots muscling in on human jobs. But I think I can live with this new technology. Because I'm in favor of anything that makes us see, umpires as people. Even if that thing is a robot. That story was first broadcast in 2023. Today, the robot umpire has officially been introduced at the major league level. And after the break, Chris Barube will tell us how it's going.

So we're back with Chris Barube. Hey, Chris. Hey, Roman. So Chris, what has changed since we first broadcast that story? So the robot umpire has arrived in major league baseball, but not in the way that I was predicting, certainly that a lot of people were predicting. So remember, in that story, we talked about a system where the robot umpire makes all the calls and then the human umpire says it out loud, right?

Like they were trying that at the AAA level. So Major League Baseball is not doing that. Instead, they are using the robot umpire as a kind of challenge system. So right now, human umpires are still making every ball and strike call. But if a player disagrees, they can challenge the call. And then the automated balls and strikes will determine who was correct. So they're using the robot umpires, but humans are still doing most of the work. So this was not what I was expecting either.

So how does that end up working during the game? So, Roman, I've actually sent you a link with an ABS challenge from a game this week between Atlanta and the Kansas City Royals. So I want you to pull it up and narrate through. Like, what is it that you see? Okay, the pitch comes in. It looks a little low. Okay, the catcher taps his helmet. Okay, and then the umpire goes to the side to check what's happening.

happening. All right. All right. And then they show a graphic of the ball. Just sort of hitting the bottom of the strike zone. As you can see, this is very close call. Yeah. I mean, the Empire does a really good job, actually. Yeah. That's a really close borderline call. So the fact the catcher knew to challenge that one is impressive. But yeah, yeah. Yeah. So that's what it looks like. It's like the ball comes in and then either the catcher or the hitter or the pitcher taps their head, taps their helmet with their hat. Wow. And then the umpire says, okay, there's a challenge on the field.

And then everybody looks up at the scoreboard and they show the graphic. Like within about 10 seconds, you know what happened. And then it shows the graphic. And then the ABS decides, okay, that just graced the strike zone. So that's a strike or, you know, that was actually outside. So that's a ball. So that's how it works. Okay.

So my question is this, okay? How much does this slow the game down? Like baseball is not like a lightning fast game to begin with. You know what I mean? Because I would imagine. in, like, why wouldn't you challenge everything? You know, like, and therefore slow everything down even more. This is a really good point.

And I think this is what people were worried about with the introduction of the challenge system. But they've put in a rule to limit how often this happens. So each team only gets two challenges a game. But if you make a successful challenge, you get to keep it. So basically, each team gets the right to be wrong twice. So if you, you know, if you challenge and you're wrong, you lose that challenge for the rest of game. So it incentivizes really only picking your spots, like really only doing it on important plays or plays where you're like, I'm pretty sure I'm right about this one. That's right. You have to

save your challenges for when it really matters. That's it. Yeah. So are people happy with this so far? I'd say for the most part, yes. So for players, like, this has been pretty good because, you know, before, if there was a bad call, they had to kind of grin and bear it and go back to the dugout and be angry. But now a few times a game, you know, they can get a call overturned. Like they have some recourse. So that's cool for them. The fans, for the most part, they seem pretty into it, which has been kind of a surprise. So this is anecdotal, like at the time of recording, we're one week into this experiment. But all the baseball fans in my life that I've spoken to have been like, huh, this is actually kind of fun. I'm surprised. What's fun about it? Well, I think what a lot of people

were worried about was how this would affect the drama of the game, right? Because instead of having a human make calls, you'd have, you know, a clinical robot making the calls. But it's actually creating more drama, which is because a couple times a game now, a player will say, oh, you thought that was a strike. I think that was a ball. And then everybody looks up at the scoreboard together. And then, like, fans go crazy when their team is right, you know, in their challenges. So it's not what I was worried. about, which was, oh, it's going to be a human versus robot. That's kind of a boring conflict. Instead, it is this moment of player versus umpire with the robot as the arbitrator. And that's actually kind of fun. It's kind of fun to watch that happen. I mean, actually does sound fun. So how are the umpires

taking it? Well, that is a different story. So obviously, no active major league umpires have said anything about it yet. I'm sure they're still in, you know, kind of a wait and see phase. And, you know, you don't want to risk your job by saying something about Major League Baseball. But a couple of retired umpires have given interviews. And basically what they've said is like, I feel terrible for the umpires right now. Because if you think about this from an umpire's perspective, this cannot be good for your self-esteem, right? Or for your reputation? Because if you're an umpire, several times a game now, someone questions your judgment, and then everybody looks up at a giant scoreboard to see if you blew it.

And if you blew it, everyone cheers. Like 30,000 people are cheering for a mistake that you made. And for the accurate umpires, like, doesn't really matter that much. But for the ones who make mistakes more often, like, there's a couple of umpires who have been genuinely humiliated by the ABS system. And we're only a week into this so far. Yeah, that doesn't sound fun at all. Yeah. So I want to show you an example of this, Roman.

I want to turn your attention to the case of CB Buckner versus Augenio Suarez. So C.B. Buckner is an umpire. He's been in the league for about 26 years. And A. Eugenio Suarez is the third baseman for the Cincinnati Reds. And earlier this week, when we're recording this, they had a bit of a notorious dustup involving ABS. Okay. Let's watch it. So here we are. We are in Cincinnati. And this pitch is clearly low.

A. Eugenio Suarez taps his helmet to indicate he wants a challenge. Buckner looks out to the audience to say the call is being challenged. Then we look up with the scoreboard. And it's pretty obvious that this one was like a little bit outside the zone. So Suarez was right. Call gets overturned. This was a third strike call too. So this would have ended the inning, but instead he gets another pitch.

And then it happens again, a pitch later. So there's another pitch outside the zone. Suarez challenges again. Buckner's looking pretty upset. He talks to the audience. He's like, all right, I guess we're doing this again. And it's pretty obvious he was even more wrong this time. And now the crowd's losing it.

They're like, yeah. Now, obviously, this is great drama, right? And for the fans, this is cool because it's like, you get to challenge. And it's like, we were right. The umpire's wrong. Are you blind? You know, all this stuff. Yeah, yeah, sure.

But for this umpire, you know, he had a whole stadium full of people cheering because he did a bad job at work. Right. Yeah. And this clip went super viral. I did not know the name C.B. Buckner before this week. And now everybody in baseball knows who he is, right? There have been all these articles that are like, here's another thing. CB Buckner did.

And this probably would not have happened without the ABS system challenging his calls. Yeah. Yeah. That sounds very rough for him. My heart goes out to CB. Yeah, it's about to get worse. So CB Buckner in that game got six calls overturned. So this happened to him six times.

And now basically anytime he does something wrong, there's people on social media and people in the baseball media being like, C.B. Buckner screws up again. And a contextual note, CB Buckner is one of the few. few black umpires in baseball. So this is obviously not ideal that this is the person coming in for so much criticism. I should know he is a below average umpire. Like anybody who does the like tracking of umpire accuracy says like C.B. Buckner doesn't have a great eye, but like you gotta feel bad for this guy.

And then later in the week he got hitting the mask by a foul tip. Like there was a foul ball that hit him in the face and he had to leave a game. So, you know, when it reads it pores for this guy, like you got to feel for this guy on a human level. Absolutely. He's just having the worst week ever. My sympathies, for sure. Yeah, me too. And when we did this original story, I came out softly on the side of robot umpires because I thought it would help people see umpires, like the humans as human beings, right? Because it would take out some of this, like vitriol against umpires in the game.

But in this particular case, it's kind of doing the opposite. Like it has put this person under a microscope. It has empowered people to, you know, yell at this guy and be. mean to him on social media and stuff. And, you know, C.B. Buckner's a grown man. You know, comes with the territory. But, like, it has kind of done the opposite of what I was thinking the robot umpires would accomplish. Yeah. Yeah. So I guess we'll just have to watch the rest of season and see how it goes for all the other umpires. But I'm totally intrigued. Thanks for the primer. Of course. And I hope we've given everybody some extra context when you're watching a baseball game

this summer. Awesome. Well, thank you, Chris. Thanks, Roman. 99% Invisible was produced this week by Chris Barrupe. The 2023 episode was edited by Kelly Prime and fact-checked by Graham Hesha, mixed by Martin Gonzalez, music by Swan Rial. You can find Zach Halfen's article about robot umpires in The New Yorker. His latest feature story is about airport lounges. It is a must read for fans of 99PIA. You're going to love it. Katie Nolan has recently launched a new sports podcast called Casuals.

You can find it wherever you get your podcasts. Our executive producer is Kathy, too. Our senior editor is Delaney Hall. Kurt Kolstad is the digital director. The rest of the team includes Jason DeLeon, Emmett Fitzgerald, Christopher Johnson, Vivian Lay, Lashma Don, Jacob Medina Gleason, Talon and Rain Stradley, Joe Rosenberg, and me, Roman Mars.

The 99% of visible logo was created by Stefan Lawrence. We are part of the SiriusXM podcast family, now headquartered six blocks north in the Pandora building. And beautiful, uptown, Oakland, California. You can find us on blue sky as well as our own Discord server. There's a link to that as well as every past episode of 99PI at 99PI.org.