Effectively Wild: A FanGraphs Baseball Podcast - Effectively Wild Episode 1058: A Better Bullpen Tool

Episode Date: May 16, 2017

Ben Lindbergh and Jeff Sullivan banter about Bryce Harper, Mike Trout, Jason Vargas, and the evolution of Matt Albers’ save celebration, then talk to writer Sky Kalkman about his new bullpen stat, Win Probability Added Over Replacement, the problems with Win Probability Added as a tool for reliever evaluation, and the “right” way to evaluate […]

I think it was effectively wild. I threw enough strikes and moved the ball around enough to keep him off balance. Lord, from up to heaven He's a mighty good leader All the way Hello and welcome to episode 1058 of Effectively Wild, a baseball podcast from Fangraphs Presented by our Patreon supporters I am Ben Lindberg of The Ringer, joined by Jeff Sullivan of Fangraphs Hello

Hi I was just navigating to the Fangraphs leaderboard To see where things stand in this sport that we talk about And Bryce Harper right on top, 2.8 war Mike Trout, second, 2.5 war I derive such satisfaction from seeing those two guys At the top of the leaderboard It just makes the whole world feel orderly And predictable and exciting at the same time.

It's great for baseball that these two guys are the best players in the sport. It's great for us because we get to feel like we understand baseball on the most basic level that we know who the best players are, who they were supposed to be. I just hope that they finish the season 1-2 in some order again, because I like this. I hope that this could just be the way that baseball is every year for the next decade or so. Right. If it could just stay normal at the top. And then if you look at there's a combined war leaderboard on Fangraphs combining position players and pitchers. And if you look at that just outside of the top 10, you get these names in order. Corey Dickerson, Aaron Hicks, Eugenio suarez zach kozar jason vargas so baseball's still weird

it's still plenty weird but it is perfectly normal at the top assuming that you figure bryce harper bouncing back is perfectly normal and it makes sense miguel sono is there that kind of makes some sense aaron judge because he's basically miguel sono paul goldschmidt because he's just an older version of the same two players. It all feels pretty good. Still no Fangraphs post on Jason Vargas. That's pretty amazing. He's been like one of the two or three best pitchers in baseball

for a month and a half now and with the volume of posts that you produce and that everyone else at Fangraphs produce, you'd think that by now someone would have gotten desperate enough to throw a token post jason vargas's way but we're this far into the season just so great is the resistance to writing about jason vargas as if he's interesting there's not anything different there's nothing

different his numbers are different but he's pitching not any different it's jason vargas is the real life version of what people thought the reds were on effectively wild just i at this point not only am i not writing about jason vargas i think that other people are following online and just avoiding him because there's just what's the appetite for it no one's going to come up to the trade deadline be like you know what we need to trade for an ace have you seen what jason vargas is doing on the royals there is nothing as long as jason vargas is throwing like he's throwing, like his pitches, and he still is and resembles and is as old as Jason Vargas, there might be quite genuinely nothing he could do on the field of play to convince other teams

by July, he is a guy you need to trade for for the playoffs. Yeah. I mean, he's got the low Babbitt going. He's got the 2% home run per fly ball rate going. So none of his fly balls have been home runs practically. He is not really striking out guys more. I guess his control is very slightly better, but it's always been pretty good. He's not getting more grounders or anything like that. I haven't looked at his stuff, but I assume you're saying that it's not dramatically different he did not wait wait wait this is a trick harder now we're talking about jason we're talking about jason okay well okay well whatever as long as we're talking about jason vargas i will point out let's see so here are here are the top four pitchers this season qualified pitchers this season in contact rate within the strike zone number one jacob de grom number two chris sale number three jason vargas number four max scherzer it doesn't make any sense there's he is one of the most interesting pitchers in baseball this year and i couldn't give less of a crap i'm not gonna write about jason vargas until probably like

wednesday it's inevitable yeah it's just your optimism at the beginning of the week you probably have topics stored up for today or maybe tomorrow and then you'll run out of gas well jason vargas sitting right there three of the top 20 names this year at qualified pitchers in zone contact right jason vargas nathan karnes and danny duffy so maybe the royals have just been facing strike up run opponents that's going to be my null hypothesis. Jason Vargas, not better. Royals facing bad opponents, but still losing. Who knows? All right.

So later in this episode, we're going to have a guest. We're going to talk about relievers in general, but you want to talk about one reliever in specific, one who is very close to our heart? Yeah, I think you and I both came into this with a little bit of potential Matt Albers banter, and we'll see if it's the same Matt Albers banter. But as everybody, including probably the entire US government, understands by now, a week and a half ago, Matt Albers got his first career save in a game against the Phillies. Just yesterday, in the nightcap of a doubleheader, Matt Albers got his second career save in a game against the Phillies, because in the first game of the doubleheader in which by the way matt albers also pitched look at matt albers

did you know he could pitch twice in a day i didn't matt albers appeared but he did not get a safe situation because i believe it was sean kelly who did and he treated it horribly and the nationals lost so they called upon matt albers to save the second game. He successfully did it with a 1-2-3 inning. Anyway, after his first save on May 5th, you tweeted about, wrote about, we talked about Matt Albers' reaction. He just had like a big old smile. He was enjoying it. I think he recognized the silliness of it all a little bit, being that he is a 34-year-old large man who had no business closing for the best team in baseball in the first place, but there he was, got a save, and he just enjoyed it. He skipped a little bit.

He had a big old smile. He just looked like a goofy guy who was in the right place at the right time. Yesterday, fast forward, he closes out the game, the night cap against the Phillies. Circumstances are very similar to the game on May 5th. He gets a ground ball out and matt albers is pumped up thumps his chest he's yelling he gives like the world's most aggressive high five to ryan zimmerman at first base and he has full-on closer intensity

so i don't know what was going on in matt albers head differently on may 5th and may 14th but it took him it looks like one save and two save situations to really just get that aggressiveness and like hostility going in his blood matt albers now looks like a closer when he gets a save yeah that's pretty fascinating i don't know whether someone gave him a note like hey matt just a tip from a more experienced closer we don't smile we don't look satisfied with ourselves after we just like inducting him into the fraternity hey it was your first time we forgive you for just looking like a happy-go-lucky guy who looked into a safe situation but you get that opportunity again you better look fired up i I wonder, like, it's just kind of acting the part, dress for the job you want, not the one you have or whatever, or act like you've been there before.

And now he had been there before one time. And so that was enough, apparently, to look like a real closer. So I like that. And the Nationals Twitter account tweeted a gif of Albers also. I don't know if you saw it. I did. every team it has the player doing something dramatic and looking up with his cap hiding his eyes and then suddenly you see his eyes or maybe turning around looking back over his shoulder or folding his arms and there are lasers and lights flashing all over the place and because it's Matt Albers it doesn't really look as intimidating as it's probably designed to look. But I love it, and I will link to it.

People should check out that GIF. And I hope that this becomes a regular occurrence for Matt Albers to be coming in with save situations. But it's funny how everyone has written their wow, the Nationals bullpen is bad articles in the last few days, which I wonder has something to do with the fact that Albers is now saving games because it's not really a pressing situation. They have the best record in the National League, the second best record in baseball tied for the biggest division lead.

So yes, they have a not very good bullpen. Yes, that's probably something they should address at some point before the end of the season, but not a pressing concern right now. address at some point before the end of the season. But not a pressing concern right now. They have built up this cushion. But I guess when you see Matt Albers getting the call over and over again, it kind of drives home how few other options there are. But maybe now that he reacts like a real closer, people will accept him as one. I know the whole point of any sort of closer introduction in the ballpark is supposed to be like it's entertainment and it's supposed to be intimidating. You're supposed to intimidate the other team. But I feel like showing the other team

what Matt Albers looks like kind of has the opposite effect because Matt Albers has the body of someone who comes in after the ninth inning to be a groundskeeper when the game's over. And yet he comes out, he gets two grand balls and a strikeout yesterday and he is just pumped the hell up.

It reminds me that little, that scoreboard thing that you were talking about the gif of him turning around or looking up at the camera or whatever just because you can't have static player photos anymore. They have to be doing something. That's like something about this past decade of sports.

I remember how weird it was when they started doing football player introductions on TV and you have like all the offensive linemen moving their heads. It's weird. Anyway, I'm reminded of a story I might have told this a long time ago before on this podcast. I don't know. But at Petco Park years ago, this is back in the Khalil Green days, I remember they would kind of experiment with different ways to introduce players on the scoreboard because you need to distract Padres fans from the on-field product with whatever that you can that's electronic.

And there was one year, maybe a couple years, where they tried to do one thing in particular where they would introduce, say, Khalil Green, and they would put his numbers and his image up on the scoreboard. But it would begin with Khalil Green and every other player looking 90 degrees away from the camera. So you get a full profile. And then just unsettlingly

slowly the player or the camera would rotate such that the player would start to face the camera but there was no change of expression it was just it was kind of serial killer-y you know especially when you would have a player like Kaleo Green he was not up there like with a big old grin you know he was just up there like please stop photographing my Green, he was not up there like with a big old grin, you know, he's just up there like, please stop photographing my face. And he was just turn and just turn very slowly. And I can't really do this, you would need to whoever's listening to this or Ben, you can just ask Jesse to do this, just talk to someone and have that person look away from you. And then slowly rotate to face you with no change of expression. it's really it's really phenomenally uncomfortable and it's actually even worse if they're smiling if they're just holding like a steady grin then it

is just that's like a murdery clown that's all that's a horror movie staple that sort of thing yeah right all the players every single player for like a year at least at petco park that's how they were introduced was terrible do we know what Matt Albers' entrance song is? I just searched it quickly and I found a tweet from 2011 that says it was I Believe in a Thing Called Love, which if it's still that, that's probably another note he'll be getting now about the proper entrance music for

a closer, but I don't actually know what it is now. Yeah, that's interesting because his first save was on the road so he wouldn't have had anything yesterday he got a save at home so mate yeah the nationals fans if you are out there and listening let us know yeah we're about to get a hundred tweets about this but thanks to all of you in advance all right so we've talked about one reliever that we love a lot we're going to talk now about how to evaluate all relievers or any reliever. Who can say what brought us to this miracle? We found there are those who'll bet. There are those who'll bet Love comes but once and yet

I'm oh so glad we met The second time around. Okay, so this is probably going to be a slightly wonky conversation even for us, but most recently the Hardball Times, for which he wrote an article last month about a new way to look at reliever evaluation instead of when probability added, when probability added above replacement. And we will explain in a minute what that means. Hi, Sky. Hi there.

Thanks for having me. Yeah. So bullpen evaluation, reliever evaluation, obviously always a hot topic in the sabermetric world. And win probability added is often cited, and we cited it, I think, on our most recent episode, even on a team level. And for those of you who aren't familiar with win probability added, it's pretty simple, at least conceptually there is a certain win expectancy to every situation in a game that is based on all of the many thousands of games that have been played before so two teams based on what inning it is how many outs are left what the score is we know from many many other previous games which team is likely to win and what the likelihood of that team winning is so we

can apply that to any game that we're watching. And so if you go to fan graphs, or even if you watch some broadcasts now, you can get a live win probability graph that will tell you what the odds are. And so win probability added is just looking at the win expectancy before something happens and then comparing it to the win expectancy after something happens. So if a reliever comes in, his team has a certain win expectancy. He then pitches an inning or two. He leaves the game with a certain win expectancy. And so you basically just do subtraction and you get win probability added, which is a good story stat as it's been described. It sort of describes the pressure of the game and how you feel when you're watching a game. If a guy comes into a close and late situation and does a good job, it feels like he has improved his team's win probability added quite a bit. It's a pressure packed situation, but you do run into problems when you use this as less of a descriptive story stat to recount what happened in a game

and more to evaluate how good a player is. So Sky, could you lay out some of the issues with using win probability added or WPA to save ourselves some time as an evaluation tool? You bet. First of all, thank you for giving that description. I was really not looking forward to trying to explain what the heck WPA is.

So I started thinking about this last fall when Zach Britton was getting talked about a lot as MVP. And I really have two big issues with using when probably outed for relievers in particular. I, too, kind of like it as a stat. It takes into account clutchness, which a linear weights-based metric isn't doing. So it's kind of nice to have that included. But for relievers, the two big problems to me are that you're not comparing it to replacement level. I mean, I think replacement level is a pretty commonly accepted thing that we need to include for our value stats. And when probably added is above average and not above replacement. So let's build that in. And the second is that it's including leverage for relievers. So

closers in general are put into the game in really important situations. And I think they do deserve some credit for that. But just like we do or Fangraphs does in its war, at least, we don't need to give them full credit for that. And we can go into it a little bit more if you want. But there's this idea of bullpen chaining, where if you took a closer out of the closer role, and you brought up a guy from AAA and plug them in, you're not going to plug them into the closer role, you're going to plug them in as far down the line as you can and bump everybody up a little bit. And you're not going to lose, you know, four or five wins from kicking out Zach Britton, you're going to lose more like two thirds or three quarters of that.

Yeah. So if I can clarify the problem with using leverage as an evaluation tool is that basically the leverage increases as the game goes on, all else being equal, which mirrors the fans feeling, at least if it's a close game, you feel like if it's close and it's late, there's more pressure, the leverage is higher. But the situation is essentially the same. You could have a bases loaded one out situation in the third inning, and you could have a bases loaded one out situation in the ninth inning. And by leverage, by win probability added, that same situation in the ninth inning is going to count more, even though you could make the case that it's just as important to get an out there in the third inning or whatever. And so if your reliever is coming in in the eighth inning, the ninth inning every time, and coming into these tight situations, every time and coming into these tight situations, he's going to get a lot of credit from leverage and from WPA just for being in those situations. And it's not entirely his doing that he is in

those situations. Obviously, he had to pitch well enough to be entrusted with those situations. But you could say that it's giving a closer too much credit because leverage has that relationship with the game state and increases as the game goes on. Yeah, I would agree with that. It's the epitome of one of these, it's not all and it's not nothing things and finding how much credit to give them is the tricky one. I mean, one thing that's nice about having relief pitchers, and I generally don't like relief pitchers, but you do get to pick and choose when they come into the game. So if you're a relief pitcher who's really, really darn good, that does have value to your team above and beyond if you're just going to show up in the first inning every week because you know all your innings are going to pop up when they are important.

Finding the middle point is the tricky part. So one of the issues when you were dealing with trying to, I guess, reset where closers belong on the win probability added spectrum, if you're trying to adjust, is that very concept of replacement level and how replacement level can be different for a starting pitcher, for example, versus a closer. So in the article that you wrote, you talked about one of the big issues that you have to tackle when you are trying to reset a closer or high leverage reliever is the concept of chaining. And this is a very complicated subject that is not a delight to talk about explicitly, but I'm going to challenge you, I guess, if you could briefly, moderately briefly explain the concept of chaining, give it your best chat. Excellent. I will try briefly and I will try not to bring too much math into it. So the idea of chaining is you have a bullpen that consists of seven, eight people, who knows how many people we're going to have down the road. When somebody goes

down, you're going to bring in someone who is theoretically replacement level. This is freely available talent. You don't want to spend too much money on your ninth bullpen guy. So you either call them up from AAA or you make a simple trade. And if your closer goes down, you don't plug that new guy into the closer role. If you did, you'd pull out Zach Britton, who's say a three or four win player, and you plug in a replacement player and you get no extra wins above replacement. But what you're really going to do is put that replacement player down in the eighth slot and bump everybody up a little bit. So the guy who was pitching in blowouts in the fifth inning gets bumped up to pitch in blowouts in the seventh or eighth inning. Your setup guy becomes your new closer. So their average leverage

might have been 1.4 over the season, say, and now their leverage as a closer is going to be more in the 1.8 role. And that setup man as a closer is going to be pretty good. He's not going to be as good as Zach Britton is, but he's still going to be pretty good. And he might be more of a, say, a two-win pitcher as a closer over the course of the year. So when Zach Britton went down, you don't replace him with a zero war player. You're really replacing him with a two war player. Now you have to carry that down the line a little bit. Somebody has to take the innings that your setup guy was pitching. So your seventh inning guy becomes your eighth inning guy, and you compare your old eighth inning guy to that seventh inning guy. So maybe that's a loss of

making up numbers here, half a win. And the term chaining is applied here because you do that all the way down the line, and you add up all the little differences in war. And that, under these assumptions, is the value that Zach Britton provided, not just him compared to the replacement level pitcher, but him compared to the whole new bullpen with the roles changed. Was that all right? Did you realize when you were writing about win probability added that it is like the most difficult thing to explain in words, like spoken? So, no, well, a little bit. So I was, I tweeted about this last fall and I got asked to write this article and it took me about five, six months to write it because of all this

crap. So, so yeah, I had an inkling. It's the hardest thing whenever, I don't know, Ben probably experiences the same thing, but whenever we write an article that cites WPA, I always roll my eyes like, okay, now I need the two paragraphs that tried to explain what this concept is. And I, I still haven't figured out a good way to do it. Yeah, yeah. So the upshot of this different new method that you came up with here is that the elite relievers who come in in the ninth inning, the Zach Brittons of the world, will be a little devalued relative to WPA because we are accounting for the fact that if they weren't in there, someone else really good would be in there. It's not like no one would be pitching those innings. It's not even like the worst reliever in the bullpen would be pitching those

innings. Someone would be pitching those innings. And if Zach Britton isn't there, then it'll be Brad Brock, who has been there when Zach Britton has been on the disabled list lately. So it's still going to be a good pitcher. And so the net effect of that is that these guys look worse than pure WPA would say that they were. And what's the magnitude of the effect for most guys? So there's two things going on here. For the bullpen chaining part of it, it's for the top guys, like for example, last year, Zach Britton lost about one and a half wins. So he had 6.1 win probability added and he got bumped down to about 4.8. So it's maybe a win, win and a half. And yet that is still like a win and a half more than his war for last season would have been

and i guess that's because even the revised version of wpa that you've come up with here is giving him credit for everything that happens on the field when he is pitching whereas we know that fielders had to field his balls and he gave up, you know, 80% ground balls or something. But as good as that is, those all would have been hits if not for the fielders standing behind him. But WPA currently makes no attempt to credit them for anything. Sure. And with a stat cast, it'd be a great thing to be able to bring into it down the road. But yeah, for now, there's much more to the story than just Zach Burton's pitching. the road. But yeah, for now, there's much more to the story than just Zach Britton's pitching.

No metric can ever claim to really identify who is and is not the most valuable. There are always considerations. But have you encountered any sort of pushback along the lines of, say, some closers might have better in-house replacements than others, and you're still not entirely capturing the value of, like, Zach Britton to Brad Brock versus Zach Britton to, I don't know, Jason Aquino, let's say, could be one team's replacement. Too bad for them. No, I mean, that's a fair point. We start getting into the issue here of how much do we really care about who the most valuable player is. For me, these things are interesting because they help assess relief pitcher value in general or relief pitcher value relative to starting pitchers and batters in general.

Sure, there's a lot of nitty-gritty stuff here, and we have very large error bars, more than most people like to pretend we do. So what about on the team level? Is this above replacement necessary if you want to evaluate how good a bullpen has been on the whole? I would say that's a good question. On the one hand, a bullpen is a bullpen. And one thing you don't have to account for there is replacement level. Every team has a bullpen and you can use above average stats, which is fine. What's going to throw it off is when one team sees more

high leverage situations than another team has that might be either due to a really good starting rotation or a really bad offense low scoring teams that play low scoring games tend to have a lot more late inning high leverage situations and so you start you might start giving credit to a bullpen just because they have more of those situations and really that's just that the offense is is not very good something to consider yeah i can tell you right now from the current fangraphs team bullpen win probability leaderboards the rockies are nearly two wins above second place but looking at the uh the game the average game leverage index when rockies relievers enter a game they are at 1.22, implying their average

situation is about 22% more, I don't know, intense than average. Leverage, yeah. And then the Reds, and the Reds have the number three bullpen in win probability added, which by the way, that sounds funny. Their game leverage is actually 0.84. So they have been encountering far fewer high leverage situations than the Rockies have. So just another, I guess, thing to keep in mind. And maybe more remarkable that the Reds are number three as a team, considering they have not really been throwing that many important innings. Definitely matters too, which of the leverage metrics you look at. You just mentioned when you enter a game, say the Reds are making a lot more mid-inning pitching changes. The first guy gets into a jam. If he gets out of it, that's not affecting their LI

of when relievers enter the game. But if you do make a pitching change or bring someone in, then all of a sudden you have a high leverage situation where a relief pitcher entered the game. Just a weird fluke of the stats. So in your article, we'll link to it. People can go check out the differences between when probability added and win probability added above replacement for all the relievers from last season can look up the full results. I guess there's nowhere that people can get this constantly updated in season currently, unfortunately. Maybe David Appelman will get on that one of these days. But how do you evaluate relievers then? Do you use either of these metrics or would you if they were available or do you just go with pure context-free, look at how effective they were on a better per better basis and not dwell that much on the situations in which they were called upon? So that's going to depend a bit on the question.

If we're tackling the MVP question, I do think it's fair to bring in this question of clutchness. Are you performing well when it matters? So I would look at this when probability added adjusted for replacement level. If I'm looking at anything predictive, I'm still going to bring leverage into it, but instead of seeing how they've done in specific situations, I'm going to take how good a reliever are they, and then what role in the bullpen do they deserve based on how good they are. And if they are a

closer of Britain's stature, I'll give them the full 1.8 leverage that they deserve, but then just run that through Fangraff's war calculation rather than trying to bring in any win probability stuff. Maybe there is some predictive nature to win probability added, but I haven't really seen that. So I would ignore it in predictive senses. This could be either a very difficult question or a very easy question, depending. We have seen very few reliever seasons quite like what Zach Britton did last season. In your article, based on your method, you're able to adjust him down so that he's not number one in win probability added above replacement anymore when you run through all the math. So at least in your own estimation, would it pretty much ever be possible for a relief

pitcher in your eyes to deserve to win the Cy Young or the MVP? I think they could. I think it's going to depend a lot less on how good the reliever is. I think it depends on is it just a year on how good the reliever is. I think it depends on, is it just a year where there's a dearth of starting pitcher talent? Did the top three pitchers get hurt and not finish the year? And were there just a bunch of 3.25 ERA guys instead of one guy who has the two and a half ERA? I would say generally not, but technically, yes. And does this have any implications for reliever salaries? I mean, would you pay a reliever solely based on war? Would you factor in the leverage and pay him according to what, when probability added

above replacement, would say he deserves, which would be many millions of dollars more? No, I would go with more of a war-like metric. And my guess is there's not any team that's going to pay based on win probability added either. Mostly because, sure, Zach Britton had a great number last year, but he's not going to do that three years in a row. And so his win probability added metrics or his win probability added above replacement over the past three years when he starts signing free agent contract, that's not going to be too much higher than what war says he is anyway. So I would go with war and I imagine most people would go with some number near war or at least a predictive war. Stepping away from the topic just a little bit, but you said something about, I don't know, 10, 15 minutes ago that I wanted to ask you about.

You said that you don't like relief pitchers. Is that just a personal thing? Is that because you think it's too easy or what's your mindset? Should I just say they're fungible and move on and we'll let everyone react to that? No, I don't like them. I like the good relievers because they're doing something different. I think the 3.50 ERA reliever is a dime a dozen these days and people tend to treat them like they're hot stuff and valuable. It's not that hard to find good relievers to fill out your second, third, and fourth spots in the bullpen. And maybe I'm also just biased by so many of them appearing in games these days. I'm just sick of relievers entering the game. So that's my emotional reaction to that.

What about relievers who might be named something kind of like Matt Albers appearing in innings kind of like the ninth? I haven't heard much about that. All right. Well, have we left anything out? has done more to discount the value of the reliever in some ways than the standard stats, or if you count saves among the standard stats, then that would sort of inflate the reputation of the reliever, whereas something like war that is context-free would say, oh, relievers aren't that great, basically what you just said. And yet there is also WPA, which has kind of burnished the value of the reliever. But I guess that's more about it being misunderstood or misapplied than what the stat is actually

saying about the reliever's worth. I'm inclined to believe that relievers have a bit or can have a bit more value than war has traditionally said. It might be a compounding effect where a bullpen from one to five is just great. It might be just that some years you have the Zach Britton who has so many opportunities in high leverage situations where he actually is better than his war says. He's just not nearly as good as his win probably added says. One more thing I would like to add is the second piece to this above replacement part is just the fact that starting pitchers pitch so many more innings than relievers. And when you look at win probability added, that's not included here. So if you look at the delta on the chart in my

article, full-time starters are gaining two, two and a half wins while the relievers are losing one and a half wins. So relievers are losing one and a half wins. So when probability added, the big part is that it's, it's above average right now. And a starting pitcher with a league average ERA is, is much better than a relief pitcher with a league average ERA. And, and both of these adjustments, both the bullpen chaining adjustment and the starters and relievers having a different replacement level, both of those are already built into wins above replacement. We already commonly accept those. So in that sense, my adjustments are not groundbreaking. It's just that we need to do them to win probability added. Right. So you could say, I guess, that it's

less that you are necessarily trying to do something novel as it is maybe bring more attention to how something like win probability is actually calculated so then people can understand where its strengths and weaknesses actually are. Yes, good point. I don't – people like the idea of win probability added. I'm not sure a ton of people really understand what's going on under the hood. Yeah. Yeah. Yeah. Well, and does this tell us anything about the trend, what seems to be a trend perhaps toward teams wanting to just stack lights out relievers in the last several innings of a game? Like, is there an effect like you might see, say you have the best left fielder

in the world and the best center fielder in the world and the best center fielder in the world and the best right fielder in the world and each of them kind of robs a little bit of the other's value because they can all cover that ground and there's overlap between them or at least that's a theory is there the same sort of thing with relievers at a certain point where you lose some of the value that a reliever can offer because you have three other great relievers who could come in at that high leverage moment? Do you think there are diminishing returns? Or today when you have starters going less deep into games and the relievers are out there, and you can adjust how you use them in the playoffs, do you think that there are not diminishing returns that the more great relievers you can get, the better?

Yeah, if anything, I would say there are, to make up a word, compounding returns on it. I remember Tommy Bennett brought up an idea a lot of years ago about leverage is not just something that you get. Like if you're Zach Britton, you have a high leverage, but pitchers previously in the game have given that to you. And there might be this idea of leverage created. I mean, leverage is higher in close games.

And if you have a pitching staff, including relievers who are really good, they're going to create more high leverage opportunities for later pitchers. So having a lights out seventh, eighth, ninth might be creating more opportunity to have more wins and more war and win probability added. All right. Well, bullpen evaluation, reliever evaluation is complicated, and hopefully we were able to navigate through this complex subject and not lose too many listeners along the way. So we will link to Sky's piece if you want to digest it slowly and we thank you for coming on people can find you on twitter at sky underscore kalkman

yes you bet you bet hope it wasn't too dry me too thank you very much you can support the podcast on patreon by going to patreon.com slash effectively wild five listeners who've already done so include Damian Masterson, Nicholas Rapp, Terry Spencer, Brian Fitzgerald, and Nathan Bodnar.

Thanks to all of you. You can join our Facebook group at facebook.com slash groups slash Effectively Wild, and you can rate and review and subscribe to Effectively Wild on iTunes. Thanks to Dylan Higgins for editing assistance. If you're looking for something else to listen to, Michael Bauman and I have a new episode of the Ringer MLB show up. We talk to beat writers for the Cubs, Brewers, ands about the tightly compressed and topsy turvy NL Central. Keep your questions and comments for me and Jeff coming via email at podcast at fangraphs.com or via the Patreon messaging system. We will talk to you soon. I'll be there. I'll be there.

Everything will be explained.