Over the past few weeks I’ve been digging into the stats at Sports Info Solutions in an attempt to learn more about which advanced stats hold some predictive value. Most stats hold some correlation from one year to the next and appear to be skill-based, but some of the more interesting numbers are the ones that appear to be pure luck.
One luck-based stat I recently noticed was catchable target rate allowed by cornerbacks. This isn’t surprising. Opposing defensive backs might factor into the play calling and they might factor into the quarterback’s decision making, but once he chooses his target, the defensive back is no longer a part of that equation.
I did wonder, however, if maybe defensive backs could influence a quarterback’s accuracy on throws downfield. For example, are some quarterbacks maybe trying too hard to throw the perfect pass against Jalen Ramsey or Patrick Peterson? The answer is no. On throws 10 or more yards downfield, the correlation from Year N to Year N+1 is .012 (rough translation: the previous year’s catchable target rate explains 1.2 percent of the following year’s rate. In other words, it’s purely random.)
Here’s what that data looks like in chart form:
Learning that cornerbacks have no control over whether or not their assignments are seeing catchable targets doesn’t mean this is a useless stat, however. Much like a stat such as BABIP in baseball, we can use this to determine how much good or bad luck is reflected in the player’s stats.
To learn more about how much catchable target rate might affect our perception of players, I compared their PFF coverage grades from one year to the next and discovered that cornerbacks who benefit from an unrealistically low catchable target rate are also seeing a huge spike in their grade.
For this study, I analyzed 43 cornerbacks who had at least 30 targets of 10+ yards in consecutive years during the 2016-18 seasons. Of this group, 10 cornerbacks saw a catchable target rate of 50 percent or lower in Year N. Of those 10, six experienced an increase of at least 14 percentage points in catchable target rate in Year N+1.
The significant increase for these players isn’t surprising, because we already established that catchable target rate is random from the cornerback’s perspective. The fascinating discovery here is that of those six players who saw a huge rise in their catchable target rate, it was matched by drop in their PFF grade. These players generated a strong 76.4 PFF coverage grade in Year N, but fell to an average grade of 63.9 in Year N+1. All six players experienced a drop in grade, five of them by at least nine points.
Here’s a look at the specifics for a few notable players in the sample:
I started out by focusing on the players who obviously experienced good luck. But if there is actually a strong correlation between catchable target rate and PFF grades, then we should expect to see a corresponding shift any time there is a significant change in catchable target rate. That could be a shift from good luck to bad luck, bad luck to good luck, bad to luck to worse luck, etc.
In this sample of 43 qualified cornerbacks, there are 16 who experienced a shift of at least 10 percentage points in catchable target rate either in a positive or negative direction. When shifting from a low catchable target rate to a high rate, we should expect to see the PFF coverage grade drop from one year to the next. When shifting from a high catchable target rate to a low rate, we should expect to see an improved PFF coverage grade.
And sure enough, the results play out as expected. 14 of the 16 players shifted in the expected direction by an average of 9.3 PFF grade points (that’s HUGE!).
But it gets better, because we have an obvious outlier in 49ers cornerback Ahkello Witherspoon, who somehow managed to experience an improved catchable target rate (by 11.8 percentage points) but saw a massive 33.9 point drop in his PFF grade. If we remove this outlier, the remaining 15 players shift by an average of 12.1 PFF grade points in the expected direction.
So I’ve thrown a lot of data at you in a short amount of time, so let’s sum up some of the key points here:
- 43 cornerbacks who saw at least 30 targets at 10+ yards downfield in consecutive seasons were analyzed
- Catchable target rate proved to be purely random (correlation from Year N to Year N+1 is .012)
- Of those 43 cornerbacks, 16 (37.2 percent) experienced a shift in catchable target rate of at least 10 percentage points in either a positive or negative direction
- The 16 players who experienced a significant shift in catchable target rate, also saw a corresponding shift in their PFF grade of an average of 9.3 grade points
So what conclusions can we draw from this?
Let’s start out by acknowledging that with just two separate two-year samples (the 2016-17 seasons and the 2017-18 seasons) we have a relatively small sample size. We have to remain open to the possibility that these findings will be proven wrong with more data.
That said, the extremely low correlation between catchable target rate from one year to the next is convincing. It’s highly unlikely that this stat is capturing a cornerback skill. It appears to be pure luck.
Additionally, the random variation in catchable target rate appears to have a consistent and significant impact on PFF grades. With 14 of 16 players experiencing the expected shift when dramatically affected by a change in catchable target rate, that’s some pretty damning evidence even if the sample size is small.
As we add more data to this study after the 2019 season, the strength of the correlations may shift, but I do not expect it to change our conclusions. We (as collective football viewers, PFF analysts included), appear to be unfairly crediting/blaming cornerbacks for the accuracy of passes they face in coverage.
And yes, I know your next question. Who are the players likely to experience a significant in luck in 2019?
According to Sports Info Solutions, these five players experienced the lowest catchable target rate (meaning our perception of their performance is more likely to regress in 2019): J.C. Jackson, A.J. Bouye, Joe Haden, T.J. Carrie, Tramon Williams.
And these five players experienced the highest catchable target rate (meaning they are more likely to “improve” in 2019): Gareon Conley, Chris Harris, Ken Crawley, Rashaan Melvin, Coty Sensabaugh.
