Carl Lawson exemplifies why sacks are a poor evaluation statistic

Despite being the go-to pass-rushing stat for decades, sack totals are not a good tool for evaluating players.

Recently-signed New York Jets edge rusher Carl Lawson – who came to Florham Park on a three-year, $45 million pact – collected a modest 5.5 sacks for the Cincinnati Bengals last year. He only has 11.5 sacks over the past three seasons combined. So, how on Earth is he worthy of being paid like a top-tier pass rusher?

Those are the words of a close-minded football observer – one who is attached to a shallow and decades-old statistic that has been misleading fans for years. Anybody who has watched Lawson’s film knows the man is an elite-level beast as a pass rusher, but one glance at his sack totals would suggest he is just another dude.

It’s time for us to move on from this archaic stat as a means of evaluation. While sacks are eye-popping plays that have a massive effect on a game, they cannot be used as a method of deducing how good a player is at pass rushing. There are a few key reasons why:

• When you look solely at sacks, you ignore other forms of pressure (hits and hurries), which are also highly impactful and are far more common than sacks
• Most sacks are relatively unimpressive as they are usually credited to a player who was not the primary reason for the sack occurring
• Players have no control over whether they get a chance to convert their pass-rush wins into sacks

Let’s dig into these reasons to get an idea of why we need to stop using sacks to evaluate pass rushers.

Sacks are a small part of the story

Sacks occur far too infrequently to be considered the only measure of what a defender accomplishes as a pass rusher.

If someone tallies 15 sacks in a season, he is considered a monster. Take T.J. Watt, for example. (We’re not saying Watt isn’t a monster, he definitely is; we’re just using his numbers as a model to lay this out.) Watt led the NFL with 15 sacks last year. He picked up those 15 sacks over 479 snaps in which he rushed the passer. That means Watt sacked the quarterback on 3.1% of his pass-rush snaps or once every 32 pass-rush snaps.

The best sack artist in the sport only picked up a sack 3.1% of the time. What about the other 96.9% of Watt’s pass-rush snaps? Are they all meaningless because they didn’t result in sacks?

Valuing only sacks is a very black-and-white method of analysis. Looking at things through this lens means you only consider a rep successful if it ends in a sack, while every sack-less rep is a loss.

Anyone who watches football should know that sacking the quarterback is not the only way a pass rusher can make a positive impact on the game. In fact, sacks are merely a tiny piece of the pie.

Three forms of pressure are tracked by most outlets: sacks, hits, and hurries. The latter two matter just as immensely as sacks, if not more so.

Simply applying pressure on the quarterback has an enormously positive effect on the defense. In 2020, the league-average passer rating on passes thrown while under pressure was 65.5, a 38.3-point drop compared to the league-average passer rating of 103.8 on passes that were not thrown under pressure. That’s essentially the equivalent of turning Russell Wilson (101.3 career rating) into Josh Rosen (65.5 career rating) just by pressuring him.

Not only are hits and hurries valuable, but they are far more common than sacks, so ignoring them means that you’re ignoring the majority of a player’s impact. In 2020, 83.2% of all pressures accumulated by edge rushers were hits or hurries while only 16.8% were sacks. Only 12.5% of pressures by interior defensive linemen were sacks, while 87.5% were hits or hurries.

Since over 80% of the impactful plays made by pass rushers are either hits or hurries rather than sacks, it is essential to consider what they are doing beyond sacks, or you are overlooking the majority of their work. Looking only at sacks is analyzing less than one-fifth of the pie, leaving players susceptible to being highly overrated or underrated.

Players can post inflated sack totals that oversell their impact. This was Trey Hendrickson in 2020: he ranked second in the NFL with 13.5 sacks but 31st with 49 pressures. Players can also post sack totals that undersell their true impact. This was Carl Lawson last year, as he tied for 52nd with 5.5 sacks but placed sixth with 64 pressures.

Some might argue that sacks are more valuable than other forms of pressure and that they would prefer Hendrickson’s superior sack total to Lawson’s superior pressure total. My counterargument would be as follows: While a sack inarguably has a greater effect on the game than a hit or hurry – a sack is a guaranteed negative result for the offense while a hit or hurry is not – few sacks are actually the handiwork of the sacker himself, so sacks should not be considered more impressive than hurries or hits in regards to what they tell us about a player’s pass-rushing ability.

That brings us to our next reason to stop using sack totals to judge pass rushers.

Most sacks aren’t legitimate wins

The majority of sacks are not the classic play you picture in your head when you think of the term – a rusher destroying his blocker in a mano-a-mano battle before laying a bone-shattering hit on the quarterback. Most of them fit one of these descriptions:

• Player A defeats his blocker in a legitimate fashion to create pressure, forcing the QB to move into the arms of Player B, who did not win his rush but cleans up the sack to steal credit
• Good coverage forces the QB to hold the ball for a long time, buying time for a rusher to get home even after his initial rush was stifled
• The QB is sacked due to his own mistake, either holding the ball too long despite having options or scrambling into a rusher
• A stunt/twist is successful (which is a combined effort between teammates)
• Someone is left unblocked and gets a free sack
• A successful blitz creates an easy sack

None of the above sack varieties entail the sacker quickly beating his blocker one-on-one and then finishing the sack himself – i.e. what most would consider a legitimate sack. All of them involve something/someone else knocking down the first domino and then the sacker merely putting the cherry on top.

Here are a few examples of what most sacks look like.

Trey Hendrickson gets one of his 13.5 sacks on a play where he was beaten. Hendrickson is sent way up the field by veteran left tackle Donovan Smith, getting nowhere near Tom Brady’s pocket. On the opposite edge, Cameron Jordan beats rookie right tackle Tristan Wirfs inside, moving Wirfs directly into Brady’s set point. Brady is forced to scramble and runs right into Hendrickson.

Jordan made that sack happen. Hendrickson was unsuccessful at his job and just happened to be in the right place at the right time.

The definition of a coverage sack (well, this one is more on the QB). Against a four-man rush, Derek Carr gets a pristine pocket and an eternity to throw. All four Saints pass rushers lose their reps. The coverage isn’t actually good, as there are plenty of options for Carr, but he hangs on to the football.

Hendrickson eventually gets the sack, not touching Carr until over six seconds after the snap. Yet again, Hendrickson gets a sack on a play where he failed to beat his man.

The Jets’ offensive line gives Sam Darnold plenty of time and space, and Lawrence Cager is wide open over the middle, but Darnold fails to make the play and gets sacked by Alexander Johnson on a snap where the pass rush was shut down.

This sack happened because Darnold made a bad play, not because Johnson or any other rusher on the Denver defense made a good one. Oftentimes, it takes a bad mistake by the quarterback like this one by Darnold or the one above by Carr for a sack to occur. Most sacks can be avoided by throwing the ball away or getting it out quickly.

Frankie Luvu picks up an easy sack as he is left completely unblocked off the edge and Dwayne Haskins fails to get the ball out. Nice job by Luvu finishing the play, it’s a solid rep, but this tells us nothing about his pass-rushing skill. This sack happened because of a blown protection and a bad play by the quarterback. Luvu did not do anything to put himself in position to make the play.

Again, just like the other plays above, it’s a nice finish by Luvu. You can never knock a guy for finishing the sack – that’s always a good thing.

It’s just that sacks don’t tell us much about a player’s pass-rushing talent when you consider that so many of them are plays like these in which the primary reasons for the sack’s occurrence were something other than the sacker’s ability to beat a blocker. We use sack totals to try and estimate how good players are at pass-rushing, and pass-rushing is the art of defeating blockers, so it doesn’t make sense that we are using a stat that often credits players for not defeating blockers.

Sacks tend to be the finishing touch following a series of preceding events. They are not always the dominant individual effort that they are known to be, which would have to be the case to validate using them as a tool for accurately evaluating pass-rush talent.

Evaluating the sack totals of a team is perfectly fine. It doesn’t matter how a team gets its sacks – if they’re all from blitzes or good coverage, so be it, the impact is still as great as if they were all from Von Miller or Khalil Mack destroying someone – but sack totals are not useful for learning about an individual’s talent and production level since many sacks aren’t impressive plays nor are they attributed to the true creator of the sack.

It’s extremely tough for a player to get a sack without help, and that brings us to our final reason that sacks are a bad stat for individuals.

Players can’t control their opportunities to get sacks

The reason that such a low percentage of pass-rush victories are converted into sacks is that they require a perfect storm of events is required in order for them to happen. To turn his win into a sack after beating his blocker, a rusher needs an ample amount of time to get home, a good angle to the quarterback, and, preferably, for the quarterback to not see him coming.

If a rusher beats his man but the quarterback gets the ball out immediately, there’s nothing he can do about that. He did his job and was unlucky that he didn’t get a chance to make a play. If a rusher beats his man but the quarterback scrambles because another rusher got dominated and gave up a scrambling lane, there once again is nothing the rusher can do about it. If a rusher wins but the quarterback happens to be looking in his direction at the time, the quarterback will probably scramble away to avoid the sack.

These things are just bad luck. There is absolutely nothing a rusher can do to control these factors, and yet they are the primary determining factor in how many sacks he gets.

Just comb through Carl Lawson’s career film. Lawson ranked second in the NFL with 32 quarterback hits last year. Some might be frustrated about having a player who makes a lot of “almost” plays rather than finishing sacks that can be clipped and posted on the team’s Twitter account with an all-caps caption and flame emojis, but when you actually watch Lawson’s hits play-by-play, you will have a hard time finding examples of plays where Lawson should have been able to get a sack but didn’t. The guy was constantly winning reps quickly. Things just haven’t broken his way very often.

What more do you want Lawson to do on those plays? He is obliterating his matchups about as quickly as he possibly can. Quarterbacks getting the football out before he can get home is just up to pure luck.

And that’s just it: sacks are largely based on luck. The rusher’s job is to win and create pressure. Full stop. If he’s lucky enough to get chances to finish sacks, awesome. Then it comes down to his ability to finish sacks, which is something that we certainly can fairly evaluate by keeping track of how many sack opportunities he converts versus how many he botches (for Lawson, this is not a problem as he was credited with only 1 missed tackle in the passing game last season).

But it has to be kept in mind that the number of sack opportunities that are presented to a player is completely out of his control. The only thing that is under a pass rusher’s control is how often he gets into the backfield – and that’s how should we evaluate them, not how often they are lucky enough to hit the lottery and have a sack fall into their lap.

This is why a player’s pressure total (sacks, hits, and hurries) is far more indicative of his true performance level than his sack total: there are a lot of things that can stop a guy from getting sacks, but there are significantly fewer factors that can prevent him from creating pressure.

Conclusions

Remember these simple things when it comes to using sacks as a tool for evaluating a pass rusher:

• Hits and hurries are highly valuable and make up the majority of a pass rusher’s impact plays (over 5 times as common as sacks)
• All a pass rusher can control is beating the man in front of him – whether he is given the chance to turn a win into a sack is largely up to luck
• Most sacks aren’t a special individual effort anyway

The moral of this story is not that you should expect Lawson (or any other pass rusher you are analyzing) to start putting up more sacks in a better situation – the moral is that you shouldn’t care how many sacks a pass rusher puts up.

If a pass rusher is winning battles and racking up pressures at a high level, it doesn’t matter how many sacks he gets (so long as his lack of sacks isn’t due to his own failures in blowing chances to finish them). He’s making a positive impact on the game and controlling what he can control. If he is not winning or creating pressure at an adequate level, then we can criticize him, even if he stumbles into a hustle sack now and then. Sacks do not tell us how well a player is playing.

I would advise football fans to begin evaluating pass rushers using statistics such as quarterback hits, quarterback hurries, total pressures, pressure rate, and pass-rush win rate instead of sacks. Some of these metrics can be difficult to find, but as it pertains to the Jets, we have you covered throughout the year here at Jets X-Factor with in-depth pressure statistics for defenders across the roster.

It’s time to move forward and leave the days of simple-minded analysis in the past. Kicking the bucket on the utilization of sack totals as a method of evaluation is a great way to start.