Judging quarterback performance is a tricky business. They are executing plays selected by someone else as part of an offense designed by others and relying on 10 other players to do what they are assigned. The long list of confounding factors renders just about every raw QB stat a team stat in actuality. One of the ways that we can approach teasing these things apart is to look at different measurements in terms of year to year stability. As things around the QB change how much does the metric we are looking at change? Does changing teams, hiring a new coach, or signing new players change the stat in question or is it “sticky” from QB to QB? Today we are going to look at this question from the point of view of sacks and sack rates.

What Are the Contributors to Sacks?

Intuitively there are several things one would expect to raise or lower the probability of getting sacked on any individual play. Some examples include:

• Indecision on the part of the quarterback causing them to hold on to the ball too long and giving pass rushers more time to get home.
• Play design. Long developing plays mean the quarterback and his pass protection have to survive longer. Conversely, plays based around quick passes make can make life easier for blockers.
• Protection schemes. Is there personnel on the field that are assigned to help block like a running back or a tight end? Are teams asking rookie linemen to go against the Myles Garretts of the world without help?
• Personnel fit. Is the design of the play such that it heavily relies upon the quarterback’s mobility to evade pressure and does that match the player’s skill set? Taking a play designed for Lamar Jackson and telling Payton Manning to run it probably isn’t going to go so hot.
• Does the QB have leeway to change things up if he sees an unfavorable box count or a defensive alignment he doesn’t like? Depending on the player’s ability to read what he sees being able to do this can either fix problems or create them if they get fooled by deceptive looks.

Looking at the Data

To see what the numbers can tell us about the relationship between quarterbacks, sacks, and their year to year stability I will be pulling data from nflfastr covering the seasons 2014-2024. This gives us a sample size of 222,198 dropbacks. I have also limited the results to players with at least 100 dropbacks in a given season. If we don’t do that we get a long tail of trick plays like Penei Sewell (an offensive lineman) getting sacked during the Lion’s 2024 Thanksgiving game:

Penei Sewell tried it 😅

📺: #CHIvsDET on CBS/Paramount+
📱: Stream on #NFLPlus pic.twitter.com/tPb5KRWNCr

— NFL (@NFL) November 28, 2024

When we apply all of this and look at how many rows of season vs. season-1 we have for each QB we end up with 323 total.

An Important Caveat

One problem that we need to keep at the forefront as we examine this is that there are selection effects at work here. One potential consequence of a quarterback getting sacked too much is that they may wind up getting benched. A recent example of this that comes to mind is Matt Ryan’s final season when he was on the Colts and getting absolutely destroyed in every game he played. What this means is that our sample is composed of players who were either good enough to keep their job regardless of how they did with regard to sack rates OR the team had no better option to replace them with. Having the dropback minimum filters out a lot of backups who have extremely high sack rates in a small sample size. Their high sack rate influences the fact that their sample size is unlikely to grow because no coach would feel good about starting them and even if they did would lean heavily towards run plays.

What Does The Distribution Of Sacks And Sack Rates Look Like?

Over our sample period all of the QB seasons that meet the eligibility criteria have a mean of 29.4 sacks and a mean sack rate of 6.18%. Here is what the distributions look like:

You see more dispersion in the count than the rate. This is to be expected given that our minimum dropback count for a given season is 100. The median for a full season is 523 (and certain pass happy offenses can end up with numbers far higher). Injuries, suspensions, benchings, and being kept out after a playoff berth have been secured will create a lot more variance in a counting stat than in a rate stat (although as we already talked about if your rate stat is bad enough you don’t get the opportunity to rack up further counting stats).

How Sticky Is Sack Rate From Year To Year?

It turns out that the answer is “somewhat”. With an $R^{2}$ of about 0.477 we see that a decent chunk of the sack rate follows quarterbacks around but a lot of it is contributed by some combination of other factors. One of those “other factors” that we should keep in mind is changes in the quarterbacks themselves. Older players slowing down and becoming less elusive and younger players learning and improving are certainly things you would expect to happen. Here is a scatter plot showing the relationship over our ten year sample. As you might have guessed based on the $R^2$ value there is a messy but discernible relationship between $YEAR and $YEAR-1 sack rates.

What Does This Mean For Projecting Future Performance?

Something that might be instructive here is to look at a couple of players whose careers span the sample period so we can get an idea of what “somewhat stable” looks like at the level of individuals. First, lets look at the career of Aaron Rodgers.

Aaron Rodger’s Career Stats Breakdown

In 2023 he only played four snaps before tearing his Achilles so we can ignore that. He also missed significant time in 2017 which is why he logs only 261 dropbacks instead of the ~620 he usually records. If we look at the rest we see that he generally bounces around a 6% sack rate with a couple of bad years in the final two seasons of former Green Bay head coach Mike McCarthy (to be precise McCarthy was fired with 4 games left in the 2018 season and Joe Philbin served as the interim for the remainder of the year). Contextualizing this a little bit by looking at how he compares to his peers we see that his ~6% career sack puts him in the 42nd percentile of QBs in our sample range. Compared against the distributions above we see that his sack totals are above average but his unusually high passing volume puts his rate below the average. Given that there is generally a trade off between volume and efficiency this leads me to the uncontroversial conclusion that the future hall of famer Aaron Rodgers does in fact appear to be good at his job based on these numbers. From the lens of predictive value it’s notable here that this time period spans 3 head coaches (5 if you count interims) and two teams yet that would be hard to tell from the stats alone. Even his 2024 season with the Jets when he was coming back from a major injury looks roughly in line with his career numbers in Green Bay. Next, lets look at a very stylistically different player: Russell Wilson.

Russell Wilson’s Career Stats Breakdown

Russell Wilson takes sacks. A lot. Why is that? There are multiple overlapping reasons:

1. At least in the early part of his career he was doing a lot of running. A lot of the RPOs, designed QB runs, etc. that were exploding in popularity in the mid 2010s can end up with the play being graded as a sack even though the designation of the QB as a passer rather than a runner can be somewhat murky and counter to intuition.
2. Scrambling ability has been one of Wilson’s strengths since he first entered the league. While this can help avoid sacks there is a tendency to sometimes scramble instead of simply throwing the ball away. Overconfidence in this ability can also lead to seeking the big play over taking what the defense is giving you ending in sacks. Wilson has done a lot of this over the years.
3. One of the things that Wilson has been very good at over the years is deep shots. A lot of his success as a passer has come from downfield routes that take a while to develop. These plays can leave you vulnerable in pass protection because the defense has that extra time to try to get to the QB.
4. The Seattle Seahawks front office has never really prioritized the offensive line. As a consequence they have graded out poorly for almost every year that Wilson was there. This is exacerbated be Wilson’s scrambling. It makes the assignment of the linemen much more difficult if they don’t know where the QB is because he’s running around rather than staying in the pocket.

Looking at these numbers in terms of year to year consistency we see that he is going to have around 52 sacks in every year that he plays the full season (which he did not do in 2021 due to a finger injury). The rate is somewhat stable with a base line of about 8.6%. The fact that he has been above that for the last 3 straight years might be cause for concern for the New York Giants that acquired him in free agency this year. One thing that should be commented on is that although simply looking at the teams column would imply a lot of stability that’s not really totally true. While Pete Carroll was the head coach during this entire time period he was more of a defensive guy. In terms of offensive play callers Wilson had 5 total over the sample period (3 in Seattle, 2 in Denver, and 1 in Pittsburgh). We once again see that a lot of things can change around a player but their stats tend to stay within a certain band (with the occasional outlier year).

What Can We Conclude From All Of This?

I think the big overarching message here is that we should generally think about sacks and sack rates for players in terms of decently sticky ranges. If you have a multi-year sample for a given player you should believe in the averages rather than try to extrapolate from recent results and the inherent noise therein. When asking questions like “should my team sign this guy?” you should apply the heuristic that players who get sacked a lot will likely continue to get sacked a lot and vice versa. This also means you can’t entirely wave away bad sack stats as the product of being on a bad team although you can apply some reasonable discount.

Looking At The Rookies

In the spirit of irresponsibly making grand conclusions from small sample sizes let’s look at the qualifying rookies from the 2024 season and interpret their stats through the point of view of what we’ve just discussed.

Assuming a moderate degree of stickiness with these numbers implies that Bears and Patriot fans should be a little nervous here. The dropback numbers here make me feel a little bit better about Williams and a little worse about Maye. 633 pass plays is a lot to put on a rookies plate so you can chalk some of the high sack total up to volume/efficiency trade offs. For Maye it’s the opposite. He only played in 13 games with fairly run heavy game scripts so he accrued a high sack rate with only 59% of the dropbacks that Williams had. This is counterbalanced somewhat by the fact that both teams had a weak roster and poor coaching so they do get some benefit of a doubt. That being said based on the available evidence one has to provisionally conclude that both players will end their careers with averages on the higher end of the range. Rattler is a similar statistical story but I think the circumstances matter here. It was never really the intention for Rattler to start and the Saint’s original plan was to develop him for a while. Injuries forced Rattler on to the field before he was probably ready to be out there. This is not an excuse but simply something to think about when considering whatever circumstantial discount you want to apply. In the case of Daniels these numbers are not as sky high as Williams but when you watch his games he does have a distinct lack of self-preservation instinct so these numbers are not surprising. One thing to keep in mind here is if these numbers hold and he ends his career with Russell Wilson-esque sack numbers that does not preclude having a lot of success. Wilson has a ring and multiple Pro Bowl appearances so sacks are far from the full story. I don’t think there is much to say about Michael Penix at this point. He only started 3 games and is just barely above the snaps threshold to qualify for inclusion. For what it’s worth I thought what I saw from those games was pretty encouraging but we will need to wait until next offseason to examine actual numbers. The fans of the Denver Broncos should feel a lot of optimism about Bo Nix based on these numbers. Even with a lot of snaps he had an outlier low sack rate. Even if we expect some level of regression towards the mean going forward the reasonable expectation at this point is that he will measure out on the happy side of the distribution over the course of his career.

Future Areas Of Investigation

• An examination of age curves for sack rates. Do we see some sort of a bathtub curve where rookies and washed up players have higher rates than mid-career guys?
• Sacks per dropback is more confounded than something like sacks per allowed pressure would be. If play level pressure data was available it would be interesting to see how that alters the results.
• What are the sack rates for different types of pass plays and does the risk/reward pencil out?
• What is the relationship between sacks and other bad outcomes like interceptions? Are there behavioral things that can be measured here (e.g. a QB preferring to throw up a prayer ball that sometimes gets picked rather than take a sack)?