Stats are ruining hockey

Updated: October 29, 2012 at 7:23 am by Eric T.

Sabermetrics! Fun!
“It’s a triumph of number-crunching over the human spirit — and it’s about time.”

It’s something I think we’ve all heard, that stats are ruining the game. It always struck me as nonsense.

After all, a fan could easily watch every game on television, read his hometown newspaper every day, talk to the guys at the bar all the time, and never once encounter a non-traditional statistic. If you really hate hearing about non-traditional hockey stats, it seems like your strongest rational claim is that stats are ruining the arguments you like having in certain circles on Twitter and blog comment sections.

Surely they aren’t actually hurting the product on the ice, right?

Driven by incentives


There’s a passage in Nate Silver’s book The Signal and the Noise in which he talks about a basketball bettor who got an edge by observing the behavior of Ricky Davis. With his Cavaliers out of the playoff hunt in 2002, Davis’s incentives to maximize the team’s chances of winning were reduced, and his play was increasingly reflecting a focus on his impending free agency.

Knowing that players are often evaluated in part on statistics such as points per game and assists per game, he was pushing the offense to run at a much higher pace. Regardless of what effect that might have on the team, getting extra possessions would lead to him getting more points and assists in each game, which in turn would presumably make him a more valuable commodity in the upcoming free agency market.

Some might see this as selfish, and I can’t say that they’re wrong. A player in a team sport has a variety of incentives, and sometimes the best action for the team is not the best action for the player; a reasonable definition of a selfish player would be one who chooses the option that benefits him more often than most people would.

However, I am also hard-pressed to criticize every individualistic play. Certainly, some individuals are more selfish than others, but the system of incentives is complex and ever-changing.

There are a lot of things that can reduce the incentive for a player to help his team. Maybe his choice becomes less important to the team because they are out of playoff contention or because the game has gotten out of hand. Maybe he has become less interested in helping them because he wants to see a coach he dislikes fail or because he’s not worried about letting down teammates who have treated him badly. A veteran who knows he is retiring at the end of the year and has never won a championship might be more team-focused than the player whose team has made it clear that they won’t re-sign him and have made him available before the trade deadline.

So let’s take it for granted that many players will sometimes make a play that suits his needs more than the team’s — some might do so much more easily, egregiously, or often, but it is probably unavoidable that it will happen on occasion. What does that have to do with the argument that stats are ruining the game?

Misaligned incentives


Sometimes those selfish plays will have nothing to do with how a player is evaluated. A player gets out of position to deliver a big hit because he just enjoys delivering big hits. Another player takes an untimely retaliatory penalty because the opponent gets under his skin and he loses sight of what his team needs from him. But sometimes the connection to glory and money is strong, and a player, like Davis, takes actions directly aimed at boosting his market value.

Right now, the stats with strongest influence on a player’s perceived value in hockey are his goals and assists. As a result, a selfish player who seeks individual glory might wait for a breakout pass to set up an offensive chance instead of backchecking hard to help his team recover the puck. This kind of play can unquestionably hurt the team — he might not be making the optimal play — but at least when he succeeds in scoring a goal, it is clearly a positive for the team.

One of the primary revolutions in recent statistical analysis has been the discovery that shot differential bears a strong correlation to winning and is one of our best predictors for future wins. As a result, the shot differential with a given player on the ice is widely seen as a measure of his ability to drive play and an important component of his value.

But unlike with goals and assists, nobody thinks shot volume alone is the cause of winning. Shot differential is valuable only because we’ve observed that differences in shot quality from team to team are small, and so the number of shots is a good indicator of how many goals a team is likely to get. That doesn’t always have to be true.

If in those selfish moments, players start taking shots from anywhere on the ice just to boost their shot differential, then this will no longer be a matter of simply focusing on offense over defense. It’s not a big deal if individual incentives lead players to sometimes make choices that are slightly less than optimal, like a point guard pushing the offense a little faster. But when our value metric is not a very direct cause of success, the incentives shift from promoting slightly less optimal sugestions to promoting an active undermining of the team’s goals.

A shifting landscape

A growing number of teams are adopting the idea that shot differential is an important factor in rating a player. Obviously the notion that selfish players would be shooting from the outside to pump up their shot differential for the Corsi nerds is a bit tongue in cheek. But the possibility that new views of the game will result in new behaviors by players is real, and that possibility underscores the need for us as analysts to constantly update our views on the game. After all, the idea that shot quality differences are small from team to team is by no means universal or immutable; it wasn’t that way a generation ago and won’t necessarily be that way a generation from now. 

In fact, the progress of statistical analysis may even propel such changes. In a world where half of the teams adopted a heavy focus on shot differential and half didn’t, we would likely see a growing disparity in shot quality skills as one group targeted players who drive shot differential and the other group would then happily see the shot quality players fall to them. The very act of convincing teams not to focus on shot quality could serve to make shot quality a more important factor in evaluating teams than it had been before.

So what are we as analysts to do? We can continue working to break things down from the high-level results-oriented analysis into components that more directly relate to specific skills, as with the zone entry project. But most importantly, we need to continually update our studies and re-evaluate our understanding. The community of statistical analysts has been quite good so far at adapting to new information uncovered by novel analysis, but we need to make a point of also updating or reconfirming the old information.

Previously by Eric T.