Penn State And Quantifying Reputations

Every now and again I like to take a look at the Sports-Reference College Football School Rater. By asking users to decide which school is the best in an individual matchup, the site compiles a list of ELO-based ratings, similar to those used by chess federations, Halo’s online rating system, and even the Sagarin rating system which goes into the BCS calculations.

As of this writing, the Penn State Nittany Lions are ranked as the 35th best college football program. Of course, “best” is subjective — you can look at bowl performance, overall season performance, National Championships, whatever you want — but determining what the masses consider “best” is also part of the point of the exercise. And when it comes down to it, with placement in the National Championship Game decided by voters in a poll, subjectivity rules college sports. Reputation, as this exercise attempts to measure, defines much of fan pride in most sports, and college sports in particular. This exercise gives us just one way to quantify it.

Penn State’s 35th overall ranking, purely based on football wins and losses, is insanity. It is one of six programs with 800 all-time victories. It has a .690 winning percentage over 118 seasons. It has two National Championships and three Big Ten titles since entering the conference in 1993. When it comes to winning football games, there aren’t many schools that have done it better than Penn State.

There are maybe five to ten schools with a better overall résumé than Penn State. Alabama and Miami (FL) have been dueling for number one lately, with Florida, Texas, and Nebraska rounding out the top five. Penn State’s pure winning percentage in the fan voting matchups of 75% puts it in third place between Alabama and Florida. And, in fact, when I checked the site back in August, Penn State was tenth in the rankings — I’ve seen them as high as fifth.

Of course, this was all before the Jerry Sandusky case came out. Since then, Penn State has since fallen like a rock in the ratings, even with the winning percentage holding high. This is due to the nature of the ELO rating system. If a good player beats a bad opponent, his rating will hardly change (for example, when I voted for Alabama over North Texas, neither rating changed). If a good player loses to a bad opponent, his rating will drop like a rock. For example, I voted Fresno State over South Carolina, and South Carolina’s rating dropped a full seven points. It’s easy to see, then, how a few “bad losses” could impact a team’s ratings so heavily.

With all the backlash against Penn State since this news has come out, it isn’t difficult to imagine the people voting on this project — over 85,000 total votes — to begin voting for teams like #120 Florida International over Penn State, tanking it’s rating farther than any pure football vote ever could. Unsurprisingly, as a result Penn State has the largest gap between its expected rating based on winning percentage and its actual ELO rating.

Observe:

Click here for a larger, interactive version.

Here we have rating graphed against winning percentage. Above the trend line indicates a better fan rating winning percentage than expected, and vice-versa. We see Penn State is one of the highest above the trendline, but it isn’t quite how obvious just how much higher the Nittany Lions are than the rest of the programs. The next graph takes the guesswork out, graphing the distance from the trend line, or, in real terms, how much higher a team’s fan rating winning percentage is than we would expect based on their overall rating.

Again, click for the larger, interactive version.

Penn State clearly stands above the rest.

There is one simple conclusion to be made here: the voters, whoever the are in this exercise, are taking out their frustrations over the Jerry Sandusky scandal out on the program. Although correlation doesn’t imply causation — one of the first lessons of statistics — there is more than residual calculations behind this. Given the timing, it is the only possibility. Prior to the scandal, Penn State was considered a top-10 team. Now, despite a clear top-10 history, the team has fallen to 35th in these rankings despite a still-elite winning percentage in individual matchups. It is the only way for a team to drop this far this fast in this rating system.

Now, what we can’t say is who these voters are and if they are representative of the college football fan landscape as a whole. Obviously, given the nature of Sports-Reference, they won’t be completely representative. But would it be surprising if a similar backlash were occurring in the hearts and minds of college football fans across the country? To me, not in the slightest. The games Penn State won with Jerry Sandusky and Joe Paterno will never go away. Penn State is clearly one of the nation’s best historical football programs, but acts as horrible as those Jerry Sandusky is charged with are the kind that irreparably damage reputations. It just remains to be seen how deep the damage will go.