Sorting out the Bullpen | The Process Report

Sorting out the Bullpen

Bullpens are always tricky to evaluate because there are so many moving parts, and the inherent sample sizes usually make a poor or great performance skew the numbers more than we would like. However, the toughest nuts usually have the juiciest fruit so this endeavour is worth embarking upon with the hopes that we can make some sense of the senseless. Where are the Rays acting right and where are they falling short?

For this analysis I wanted to be able to compare relievers against their ilk. Every team has at least one good reliever. Well, almost every team. Some even have two to three great relievers while the well dries up shy of that for many. For these reasons I wanted to compare each reliever by their role. To distinguish that role I used our good friend gmLI, which tells us the leverage at the exact point that our reliever entered the game. It makes sense that over a long enough timeline a manager will be using his best reliever in the highest leverage moment and then cascading down from there. This gives us a good idea of how these guys rank. Here is how the Rays broke out:


The Rays have not been shy about using Erasmo Ramirez in high leverage. In fact, he is in the 99th percentile for gmLI for all 353 relievers that have thrown a pitch this year. We slot down from there with Xavier Cedeno also seeing very high leverage (89th percentile) on down through Colome, Romero and the rest. I have also looked up the players SIERA and have built a few things off of this starting point. You can see their SIERA on the year, and I should note that this does not include yesterday’s (5-23) latest debacle.

I ran the z-scores for each reliever to get a sense of who is above and who is below average and then multiplied this figure by the number of batters faced to bring in the volume component with the output being the zTotal column. I flipped the sign so that bigger is better as this seems more intuitive to folks. Lastly, I have converted these to a percentile so that you can have some context for where a guy ranks.

What we find is that Erasmo and Colome have been absolute beasts this year with the latter placing in the 99th percentile for zTotal and the former just cutting into the top-10%. Cedeno ranks third in the 61st percentile, which is a dramatic drop down and then you can see another shelf down to Enny Romero. We see that Ryan Webb hasn’t been good, but he hasn’t been as bad as maybe thought and certainly nowhere near as useless as Geltz or Eveland. Lord Farquhar is getting his brains beat in down in Durham, but wasn’t awful in his stay with the Rays, but the real gem was Jhan Marinez whom the Rays let go after determining that above average performance against 13 batters wasn’t good enough for this vaunted pen.

Right away we can start to question why Geltz and Eveland have seen so much medium to high leverage action. If anything, Ryan Webb should probably have been the go to there as he has been a bit better even if not great. Here’s the fun part. Very few teams have five or six very good relievers. The Rays have two elite guys, a pretty good one, and then someone that has pitched well at times and not so well at others. This quartet should be the backbone of the bullpen with the other three spots mostly being fungible to allow other guys to work in low leverage. How do the Rays compare to the rest of the league:


As a team, the Rays rank square in the middle of the pack driven by an acceptable SIERA, but tempered by a relatively small number of batters faced. That’s mostly pretty good, but not great. As a unit they have been around 5% better than average, which would seem to indicate that while not a great strength the pen has also not been a tremendous weakness. A large reason for this has been Mr. Cash’s lack of aversion to throwing his very good relievers out there when the situation calls for it…


Note that only the top-nine relievers for each team made this cut for brevity. The total percentage gives you an idea of how much these nine slots are contributing for each team.

No team uses their relief ace more than the Rays. Erasmo has accounted for more than 20% of the batters faced by the bullpen, and their three best relievers have faced a full 44% of all batters. That’s a good thing. You want your best relievers facing as many guys as their health will allow. The problem, and you’ve probably known where this is going if you have watched this team, is that they’re getting very little from the other 56%. Let’s look at this in another way:

Highest Lev

Erasmo has contributed 52 units of zTotal, which places him firmly in 7th best contribution for each team’s highest leverage reliever. The best reliever on the Rays turns out to be one of the better ones in all of the game, but let’s see how this falls off:

2nd Highest

Moving over to our second highest leverage reliever, Xavier Cedeno, you can see that the Rays now sit in the back half of the league, but are still getting positive contributions from this slot. And on to the third highest leverage reliever:

3rd Highest

Colome has been a stud out of this position with the highest zTotal on the team placing the Rays third for this role. To this point we see very few teams that have huge negative values indicating that while the Rays are mostly in a good place with their top-three relievers so is the rest of the league. Most teams have three quality guys that they can trust to usually come through. It is the back half of the pen that you hope to hide from high leverage, but almost assuredly find that to be a fool’s errand during the grind of 162 games in 180 days.

4th Highest

We see the Rays cross the average boundary here into the negative realm. Many teams have that fourth quality reliever, but the Rays do not, and it only gets worse.

5th Highest

Now we get to the recently demoted Geltz and it’s pretty clear why. While this role isn’t the most important, the fifth best reliever needs to be a bridge between the starters and the back end studs. That hasn’t been the case for the Rays, and this looks like an area that should be relatively easy to improve upon provided the team calls up one of their actual good arms instead of Tyler Sturdevant. Speaking of opportunities to upgrade:

6th Highest

Eveland hasn’t been as bad as Geltz relative to the demands of the role as we start to see more and more teams falter to find a reliable guy that they can routinely call upon, but the team still places in the bottom third of teams. It should be stressed that these are relatively smaller roles, but with the level of parity in the American League any weakness will be exposed. I’m going to skip the 7th role since Farq isn’t here right now and go right to the 8th role where we see Ryan Webb:

8th Highest

Webb has been fine for his role, though if you flip him into the fifth spot he’s still not looking shiny or anything. With this in mind I don’t think it’s as simple as replacing Geltz and Eveland and then moving Webb and the new guys up in leverage, because you don’t really even want Webb out there when it matters.

Getting Brad Boxberger back in the next week or so will go a long way in replacing a lemon, but even if he’s something like a mid-point between his excellent 2014 and trying 2015 this team still needs at least one more guy. It feels like the Rays could use one more guy out there so here’s what I would do. Sturdevant is likely only a placeholder until Boxberger comes back. When that happens Boxxxy should probably slot right into the fourth highest leverage role, which will slide Enny Romero and everyone else behind him down a role.

Then I would get wacky with it. Kudos to my friend @KevinGengler for suggesting this first, but I think it makes a ton of sense at this juncture. Per his advice the team should call up Blake Snell for the rotation and move Matt Moore into that Dana Eveland role. The problem for Eveland is that he is a LOOGY that is being asked to be a longman. Moore would provide the ability to go through a lineup one or more times, as needed, while still being a weapon over shorter stints. Snell in the rotation gives the team another dynamic lefty to replace Moore. He still has some undercoat and polish to put on, but it is beginning to feel like this season is slipping away. Snell for Moore and Boxberger for Geltz/Sturdevant immediately plugs that dyke. Here would be the new pecking order:

  1. Erasmo Ramirez – Relief ace that can face very tough righties and give length when needed
  2. Xavier Cedeno – High leverage LOOGY that could face righties as that leverage lowers
  3. Alex Colome – Face all righties late in the game and don’t be afraid to work around good lefties
  4. Brad Boxberger – Can get both types of batters in any kind of leverage for one innings stints
  5. Enny Romero – Use him when you need a strikeout and a walk won’t kill you, more when he’s feeling it
  6. Matt Moore – Lefty longman that can go once or more through a lineup as needed, don’t be afraid to get big outs
  7. Ryan Webb – Groundball getter that can also be used to soak up low leverage innings in the event of a blow out

This will replace not only the Rays two largest weaknesses in the pen, but also potentially provide an upgrade in the rotation with uberprospect Snell getting a chance to sink or swim. There’s nothing wrong with his stuff, and while I think he would take his lumps against big leaguers there is no better way for him to learn his craft than at the Show. This would be an easy fix that wouldn’t cost the team a player other than Dana Eveland and would go a long way in letting Mr. Cash prep his bullpen to achieve maximum success.


  1. rb3 wrote:

    Potentially meaningless impression (looking at the teams arranged by zTotals through the top three top leverage roles): TB and NY and Bsn seem to covary (if that’s the word) somewhat positively in their leverage patterns of use; Balto. a little less, Toronto not so much. At role #5, as one might predict, TB becomes the divisional outlier. At #s 6 and 7, we’ve got company in the negative end of the spectrum with Balto. and Bsn.

    NYY seems most consistently high in those tables overall, within the division.

    Some Boxxy-induced shuffling could indeed help (as the offense continues to get better, especially).

    I watched Sturdevant today on the TV and he looked like he Actually Had Some Idea of what to do. He may be back soon even if he is a placeholder right now.

    • mattc286 wrote:

      Worry not, rb3, covary is indeed a word (
      I think Strudevant looked quite comfortable out there yesterday and would love to see him stick around over Geltz. His K rate in Durham this year is ridiculous. It’s interesting that you note that the differences between the Rays and their ALE opponents seem to come only deep in the bullpen. Those are the relievers that (shouldn’t) matter as much, as hopefully they’re not getting as many high leverage innings.

      I think you can pretty much draw a line after Romero in that first table and say any of those guys below it are pretty much garbage. I’m still interesting in seeing if Webb can turn it around, but if he can be assigned to Durham, I much prefer Gengler’s suggestion going forward.

  2. Baltar wrote:

    Outstanding article!
    It would be great if the Rays took your Snell/Moore/Eveland advice.

Leave a Reply

#layout { padding-left:20px; }