A couple weeks ago, Statcast unveiled a new catch probability statistic. While this metric is currently only used for outfielders (it will be expanded to infielders soon), it gives us a new way to evaluate players defensively.
The main metrics used in defensive evaluation are Ultimate Zone Rating (UZR) and Defensive Runs Saved (DRS). Each is composed of run values saved or lost due to a given player’s defensive performance. Today, I am looking to see how Catch Probability compares to these metrics, as well as Range Runs (RngR), and what it says about the St. Louis Cardinals new look outfield.
To get these metrics on a rate basis, I scaled DRS and RngR to 1,000 innings, and used UZR/150 instead of UZR. I then standardized the data by converting the statistics to z-scores.
Catch probability divides plays into five categories. In these categories, five star plays are the most difficult while one star plays are the most routine. I used the Catch Probability Leaderboard to calculate a Catch Probability Rating (CPR) by multiplying the players’ success rate in each category by that category’s star rating. Then, again, I converted CPR to z-scores.
Next, I compared the zCPR in 2016 to zDRS/1000, zRngR/1000, and zUZR/150 in 2016 to measure the degree to which the statistics agree, using a sample of players with 750+ innings and 50+ catch opportunities. zCPR had an r-squared value near 50% with each metric, which is fairly strong. As an evaluative tool, CPR is essentially as useful as the other metrics. That means that, within a given season, Catch Probability will generally lead us to the same conclusions as UZR and DRS.
Where CPR really improves upon UZR and DRS is in its predictive ability (at least based on the past two seasons). In the top row of the table below, I calculated the r-squared value between 2015 zCPR and zUZR/150, zDRS/1000, and zRngR/1000. In the bottom row, I calculated the r-squared value for each stat from 2015 to 2016. For example, the r-square value of .2589 compares 2015 zUZR/150 to 2016 zUZR/150.
Traditionally, metrics like UZR and DRS vary from year-to-year due their own limitations, especially with regard to sample size. Thus, they have very little predictive value from one year to the next. This is reflected in their r-squared values, which range from .25 to .35. Using a previous season’s zCPR to predict that player’s zUZR/150, zDRS/1000, and zRngR/1000 the next season improves this range across the board by 8%, resulting in a window from .33 to .43. Keep in mind that zCPR does not include a positional adjustment, which might improve the model even more.
Additionally, using a player’s 2015 zCPR to predict that player’s 2016 zCPR results in an r-square value of nearly 65%. If we accept that CPR is approximately as good at measuring defensive value, then we now have a defensive metric that is very predictable from one season to the next.
The St. Louis Cardinals outfield defense ranked 19th by DRS and 21st by UZR/150 in the 2016 season. While the overall outfield picture was dragged down by the defense in left field, both DRS and UZR/150 liked Randal Grichuk and Stephen Piscotty. Additionally, Dexter Fowler set a career high in both categories.
By CPR, however, the picture isn’t so rosy. I calculated a stat I’ll call CPR+, which scales CPR so that the average equals 100, and every +1 or -1 is a percentage point better or worse (think wRC+). While CPR+ had Grichuk as approximately an average outfielder at 99, Stephen Piscotty came in at 89 and Dexter Fowler at 84. Sure, the combination of these three is better than the combination of Grichuk in center, Piscotty in right, and Holliday, Moss, and others in left, but it will still likely be below average.
Additionally, while Fowler set career highs in UZR/150 and DRS last year, he actually got worse by CPR+ from 2015 to 2016. In 2015, he owned a 97 mark, but dropped to 84 in 2016, as mentioned above. That’s a much less promising picture than the one the St. Louis Cardinals bought into this offseason.
Granted, the pool of players who met my criteria was fairly small and the minimum threshold of 50 opportunities is also low. If CPR takes a few years to steady like DRS and UZR, then this analysis might not mean much, if anything at all. It does, however, raise a concern for the St. Louis Cardinals outfield heading into 2017, and fans should hope that the starting trio plays more like how they were measured by UZR and DRS than by CPR.