Long, long ago, before Tom Brady and Eli Manning set their head-to-head matchup in the 2012 Super Bowl, and before Aaron Rodgers challenged the NFL record books with his precision passing attack, ESPN blazed a path to restore credibility to the analytical side of their brand as the "Worldwide Leader In Sports." Prior to the start of the 2011-12 NFL season, ESPN boldly declared it to be the "Year of the Quarterback." Putting aside for a second the fact that everyone and their mother already knows that each season in the modern NFL is the Year of the Quarterback, the network's project did hold a intriguing promise behind the slogan.
Drawing inspiration from established NFL sabermetricians like Brian Burke and the staff at Football Outsiders, the Booyahs hired Dean Oliver as their director of production analytics. Baseball has isolated individual events that lend themselves to tidy sabermetric calculation, but Oliver spent his time working on advanced statistics and analytics in basketball, a very cluttered and team-oriented sport. His book, Basketball on Paper, remains the gold standard by which all other detailed rounball analysis is judged, but ESPN immediately put him to work on another complex team sport: NFL football. He and his team emerged with ESPN's QBR.
The conceptual underpinning for the project focused on the incorporation of win probability analysis and a rigorous application to every event that takes place on a professional football field. This means runs, passes, sacks, drops, interceptions, etc. Everything has a value, and it can change based on the game situation. A five-yard completion to midfield on third-and-six means next to nothing. That same play 15 yards closer to the opponent's endzone might mean the difference between a field goal attempt and a pooch punt. The ambitious project featured an analysis of over 60,000 plays to determine estimates of win probability values, and in the end the QBR product emerged. The idea was perfectly legitimate and the execution seemed consistent with a serious scientific inquiry into the game of football, but the follow-through never capitalized on the quality of the product.
There is only one sensible reason to ever create a new statistic: to express a deeper understanding of how players and plays contribute to, or detract from, the ultimate goal of winning. A stat that accounts for context potentially provides a serious upgrade over our flawed and selective memories, because with statistics we perfectly record every discrete event that takes place on the field. The allure is clear to whomever wants to learn more about the game and knows better than to trust their own memory with an accounting of meaningful events.
QBR promised to replace NFL Passer Rating as the new standard in performance analysis. The rough edges and context-neutrality of Passer Rating has irked every NFL at one time or another, but we have learned over time to take the shades of gray and make slight adjustments on the fly. An imperfect method to be sure, but everyone knows a good passer rating when they see one, even on the quirky 158.3 point scale. Anything over 100 is very good, anything below 60 is very bad and anything in between can have a whole to do with the final score garbage time yards gained against prevent defenses. In other words, we make our own impromptu judgments and work in the context of the performance as we see fit. Enter ESPN's QBR:
QBR allocates the points added by every play in an NFL season to each of the players involved, every play. On completed passes, for example, it splits credit among QBs, receivers and blockers, depending on factors such as whether the quarterback was under duress, where he threw the ball, how far it traveled and how many yards the receiver gained after the catch. QBR splits the blame for sacks on quarterbacks and offensive linemen and attributes QB fumbles to QBs. Further, QBR weights every play by its clutch value -- its contribution to a team's chances of winning, given the score of a game, not just to scoring points.
And if you care at all about stats, that's the key: QBR finally brings all the advantages of win probability to football.
With values and credits assigned for every sack, completion, scramble, interception etc. in any possible game situation, it's out of the darkness and into the light, right? Well, not exactly. This is about as in-depth the explanation gets from ESPN. There is no deeper conversation on why certain events are worth more or less. The algorithm is treated like a propriety "secret sauce" fans aren't privy to beyond some cursory explanations like the one above. In essence, ESPN wants to draw on the enthusiasm of the most sophisticated NFL fans, but stop short of answering any of the "why" and "how" questions that fuel their passion for the game. By withholding the details, the opportunity to learn something new about the game all but disappears.
Unfettered access to key information and fundamental assumptions is what makes the baseball sabermetric community so robust and interesting. For example, if you ever wanted to know the run expectancy values for any play in a baseball game, you can easily take that information and toy with it to your heart's content. That's how theories are tested and new insights are uncovered. Looking past the idea that we don't know about how QBR works, let's take a look at the results from the 2011-12 season:
| Rank | ESPN QBR Top 10 | QBR Rating | Rank | NFL Passer Rating Top 10 | Passer Rating | |
|---|---|---|---|---|---|---|
| 1 | Aaron Rodgers | 85.2 | 1 | Aaron Rodgers | 122.5 | |
| 2 | Drew Brees | 84.0 | 2 | Drew Brees | 110.6 | |
| 3 | Tom Brady | 74.2 | 3 | Tom Brady | 105.6 | |
| 4 | Tony Romo | 70.1 | 4 | Tony Romo | 102.5 | |
| 5 | Matt Ryan | 67.5 | 5 | Matthew Stafford | 97.2 | |
| 6 | Matt Schaub | 66.7 | 6 | Matt Schaub | 96.8 | |
| 7 | Matthew Stafford | 65.1 | 7 | Eli Manning | 92.9 | |
| 8 | Philip Rivers | 64.3 | 8 | Matt Ryan | 92.2 | |
| 9 | Ben Roethlisberger | 63.3 | 9 | Alex Smith | 90.7 | |
| 10 | Michael Vick | 63.2 | 10 | Ben Roethlisberger | 90.1 |
Has your football world been turned upside down? Didn't think so. Eight of the top quarterbacks show up on both lists, and the top four names appear in the exact same order. Did you need ESPN insider or a 100 point scale to figure out Aaron Rodgers, Drew Brees, and Tom Brady are some of the best at their position? If anything, the comparison establishes a prima facie case that QBR is dispensable; we get along fine on our own, thank you very much.
The thing is, QBR's approach is still worthy of praise. Context does matter, and ESPN had a golden opportunity in the 2011-12 NFL season to prove the power and significance of their new stat. What could they have done? They could have used QBR to prove Tim Tebow did not deserve the hype he received. Tebow clearly did not have a good season throwing the ball, and his No. 28 ranking in NFL Passer Rating (72.9) demonstrates, but the rhetoric quickly moved beyond passing skill. It shifted to topics typically insulated from scrutiny and beyond the limits of traditional stats like Passer Rating. His running ability and his knack for performing in the clutch (aka- Tebow Time) became part of his legend mean to live somewhere out in the ether.
The best quarterbacks look good under any rating system, but a guy like Tebow is exactly the type of quarterback QBR is designed to shed light upon. As supporters claimed Tebow's value lurked in the shadows, beyond the box score and outside the limits of Passer Rating, QBR formulated direct responses to the questions of his clutch ability and his non-throwing contributions to the offense. Completely unaware and unconcerned with the rhetoric and pseudo-religious undertones of Tebow's role in the NFL, ESPN's QBR definitively undermined the hype and cut the clutch argument off at the knees.
After calculating and allocating "points added by every play" and every player in the NFL 2011-12 season and weighting those plays according to their clutch value, Tim Tebow finished 32nd overall among a pool of 34 qualifying quarterbacks in the 2011-12 season, behind guys like Sam Bradford, Tarvaris Jackson and John Skelton. Furthermore, his clutch rating failed to exceed the marks of Kevin Kolb, Mark Sanchez and Rex Grossman. Finally, his negative rushing rating fell behind stationary quarterbacks like Matt Cassell, Carson Palmer and Matt Hasselbeck. In plain English, QBR confirmed that Tebow is not only a bad quarterback, but that he didn't help really his team win. Nearly every other quarterback in the league did more for their team. The results could have shocked the world and put QBR on the map.
We all know what actually happened, however. Instead of leading an intelligent discussion on how their sophisticated new algorithm refuted the mythology of Tim Tebow the winner, ESPN jumped at the opportunity to dumb down the analysis and ramp up the shameless promotion of Tebow as player, not just as a man. With hard evidence to the contrary in hand, ESPN took the easy way out and pumped him up by developing a brand on their airwaves. Skip Bayless became the network's mouthpiece for the movement. That alone speaks volumes about the lack of depth in their coverage.
If Tim Tebow is where this all ended up, why did I bury his name 1000 words down in the column? Because this story isn't about Tim Tebow. I've written an exhaustive story on him and unburdened myself of so much pent up football analysis already. This article is about the opportunity he presented ESPN to lead the charge in a meaningful mainstream statistical revolution in the nation's most popular sport. Everyone wanted to discuss Tebow and the concept of clutch performance. People wanted to go deeper into the stats and look beyond the facile formulations of NFL Passer Rating. Sensible NFL fans were begging for QBR, even if they didn't know it.
The timing could never have been better to promote QBR's thorough and unbiased answer to those inquiries on Tebow's value, but the evidence didn't jive with the sexy storyline. ESPN made their choice and never looked back. Frankly, I'll take NFL Passer Rating over Skip Bayless' nonsense any day. Year of the Quarterback? Yeah right. Maybe next year...
Steve von Horn is a writer and editor at SB Nation's Milwaukee Bucks blog, Brew Hoop, and is also a writer for SB Nation Chicago. If you have questions, comments or new writing opportunities, he can be reached at steven.vonhorn@gmail.com. Follow him on Twitter: @StevevonHorn.