You’ve seen the memes. “The Chiefs warming up for today’s game” captioning a video of NFL referees stretching on the field. “Unnecessary roughness for breathing on Patrick Mahomes … 15-yard penalty, automatic first down.”
This year’s AFC Championship was just additional evidence for many NFL fans that the league’s referees are biased towards the Kansas City Chiefs. On a 4th-and-1 attempt early in the fourth quarter, Buffalo Bills quarterback Josh Allen appeared to get the ball over the line, but the officials ruled that it was a turnover on downs.
More from Sportico.com
The internet mob was also up in arms after the Chiefs’ Divisional Round game against the Houston Texans during which two defenders were flagged for personal fouls on Patrick Mahomes. “We knew it was going to be us versus the refs,” Texans defensive end Will Anderson Jr. said. He was flagged for a roughing the passer penalty during the contest.
Gripes with the referees’ treatment towards the Chiefs goes further back. Earlier this season, a crucial fourth down defensive pass interference on the Cincinnati Bengals kept Kansas City’s hopes alive in that game. Two postseasons ago, the Chiefs closed out the Super Bowl after a critical holding call against the Philadelphia Eagles’ secondary in the final minutes.
An ESPN graphic showing that the Chiefs have had fewer penalties than their opponents in 10 of their 11 playoff games since 2021 circulated on social media last week, but commissioner Roger Goodell called the idea that the refs favor the Chiefs “a ridiculous theory” at a press conference on Monday.
Let’s look at the data and see if he’s right.
For starters, “since 2021” is a nonsensical end point carefully selected to make a particular argument. The Chiefs had more penalties than their opponents in two of their three 2020 postseason games, so someone pushing the narrative of Chiefs favoritism would intentionally exclude that year.
There are, however, two time frames with legitimate justification: “since 2020,” beginning after Mahomes won his first championship, or “since 2023,” beginning after Taylor Swift became associated with the Chiefs.
Over the past two regular seasons, there is no evidence of the Chiefs getting any benefit from the referees. In 2023, the difference between Kansas City’s and its opponents’ penalty yards was -241, the second lowest in the league. This season, the Chiefs were flagged for only 20 more yards than their opponents.
Chiefs haters will cry out that total penalty numbers are irrelevant, because the refs’ bias truly shows in the game’s most important moments. This isn’t backed up by data, either. We can tally the net change in the Chiefs’ win probability on all plays resulting in a penalty, filtering out several types of plays that would otherwise skew the data, such as long gains from scrimmage with a smaller chunk of yardage tacked on for a face mask or unsportsmanlike conduct penalty, for instance.
The results show that the Chiefs have actually had their expected win probability decreased by plays involving penalties over the past two regular seasons, ranking 22nd in the league in net impact of those plays. This conclusion also holds when expanding the dataset to include the past five years. Even if we limit the sample to fourth quarters and overtime periods, the data still show the Chiefs narrowly disadvantaged by plays on which the refs threw a flag.
The stats begin to swing in support of the conspiracy theorists in the playoffs. In the 2023 and 2024 postseasons combined, the Chiefs have been called for 16 fewer penalties for 160 fewer yards than their opponents. Their total win probability gained over those six games is +27%, just behind the Green Bay Packers for tops among all teams.
In the five postseasons since winning their first ring, the Chiefs have definitely experienced a streak of good fortune. Across 15 playoff games, their opponents have committed 21 more penalties for 144 more yards, and the Chiefs’ cumulative win probability added from plays with penalties has been +45%, more than any other franchise.
This difference in between the regular season and postseason trends could be used to argue either side of the issue. On one hand, the discrepancy may indicate a selectively applied bias by refs in big games—the Chiefs are generally not an especially disciplined team when it comes to penalties, so their lower penalty totals in the playoffs should raise an eyebrow. On the other hand, perhaps we should place less emphasis on a small handful of January games and trust the much larger sample of 17 regular-season games, which show no favoritism at all.
The Chiefs’ postseason penalty numbers must also be put into context. They are not extreme outliers in recent NFL history, as plenty of teams simply have a stretch of years in which calls go their way.
From 2011 to 2015, the Denver Broncos’ postseason opponents drew 16 more penalties for 122 more yards across 10 games. From 2013 to 2017, the New England Patriots got even more lopsided treatment than the current Chiefs, getting called for 22 fewer flags for 276 fewer yards than their foes.
You might notice that both of those teams featured Hall-of-Fame quarterbacks whose advancement in the playoffs might be a desired outcome for the league. Indeed, there may be a slight league-wide bias towards giving superstar quarterbacks the benefit of the doubt. But consider the 2015 to 2019 Minnesota Vikings (15 fewer penalties for 171 fewer yards), helmed by a potpourri of second and third-tier QBs, or the Andy Dalton-led Cincinnati Bengals from 2011 to 2015 (10 fewer penalties for 140 fewer yards), who went 0-5 despite advantageous officiating.
And while the similar penalty advantages for Peyton Manning’s Broncos, Tom Brady’s Pats and Mahomes’ Chiefs may appear suspicious, it is also possible that the causality works in the opposite direction. Perhaps those teams were dominant and successful precisely because they executed well and did not commit penalties.
Furthermore, the win probability methodology makes Mahomes’ Chiefs look like even less of an anomaly. Their +45% total has been topped by several franchises over a five-year stretch at some point during the past 20 seasons, including the aforementioned Patriots and Broncos runs, along with periods for the Philadelphia Eagles, Pittsburgh Steelers and Baltimore Ravens.
Sportico found that just one of the 12 “high-impact penalties” (+5% change in win probability) during the 2024 postseason was against a Chiefs opponent (Anderson’s roughing on Mahomes). In the 2023 playoffs, three of the eight such flags by the officials benefited Kansas City, but two others went against them.
In fact, the single highest-impact penalty on win probability over the past 10 postseasons—a defensive holding on cornerback Trent McDuffie that extended a 49ers field-goal drive in overtime of Super Bowl LVIII—went against Kansas City. It’s hard to argue the fix is in when a single subjective call issued by the refs made the Chiefs 20% less likely to win the Super Bowl.
Of course, this data analysis is incomplete. The nflfastR win probability model is just that—a model—and does not definitively determine referees’ impact. More generally, looking at penalties alone excludes crucial no-calls and any plays not involving penalties altogether, such as Xavier Worthy’s controversial simultaneous catch in the second quarter of the AFC Championship.
All the more reason that anyone claiming to have data proving that refs favor the Chiefs is being disingenuous.
One last note on the notion that Mahomes, specifically, gets preferential treatment. His 6.3 roughing-the-passer penalties drawn per 1,000 passing attempts ranks 28th out of 73 qualifying quarterbacks since 2009. This season, he ranked behind Justin Fields. In the playoffs, Mahomes’ rate has indeed escalated to eight flags across 724 passing attempts (11.1 per 1,000), but that sample size is far too small from which to draw any conclusions, and his frequency is still lower than Ryan Fitzpatrick’s over his entire 17-year career (11.4 per 1,000).
Also, just watch the plays. They’re almost all no-brainer penalties.
The Chiefs have absolutely gotten some generous breaks from the zebras. But if you suggest that there’s something fishy here, you must also think that Ryan Fitzpatrick and the early-2010s Bengals were the beneficiaries of a conspiracy.
Our brains are remarkably good at connecting dots. We evolved that way because it’s a biological advantage. There was a predator behind that bush the last two times a caveman from my tribe walked by it—maybe I shouldn’t walk that way. If homo sapiens didn’t possess this superpower, we likely wouldn’t have survived and advanced to the point where we can now sit on our couches and watch moving images of people bashing their helmets together sent to our screens by machines orbiting the planet.
But this also means we often seek patterns where there are none. One of the main ways we do this is confirmation bias—searching for, noticing and recalling only information that supports what we already believe. That bias is on full display among those who roll their eyes with every penalty called on a Chiefs opponent and disregard those called on the Chiefs. In fact, those folks may not have even made it to the end of this article, and if they have, they’ll probably dismiss it.
Best of Sportico.com
Sign up for Sportico’s Newsletter. For the latest news, follow us on Facebook, Twitter, and Instagram.
Read the full article here