Another Look At How Goalies Age: Can we correct for bias?

Updated: December 10, 2012 at 12:05 pm by Graphic Comments

 Another look at goalie aging

The topic of how goalie age affects on-ice performance came up again last week, first with a post from Steve Burtch over at PPP, followed by a response from Eric T. here at NHLNumbers. Although Steve’s post was about the Leafs’ quest for an average goalie, he made the observation that “the correlation between goaltender performance and age is nil.” Eric’s main critique was that the analysis did not take into account “survivorship bias,” that is: only the better goalies actually manage to stick around into their late 30s, so this will bias the observed performance of older goalies upward.

In an effort to further the debate, I would like to propose a methodology that controls for “survivorship bias,” as well as an alternative way to account for systemic changes in the game that might impact save percentages on the whole.

Controlling For Systemic Impacts

It is clear that over time changes to the game – be it rules, defensive strategies or even goaltending styles – have impacted the observed performance of goalies in the NHL. Overall league save percentage has risen by about 40 points from 0.873 when the NHL started tracking it in the 1983-84 season, to 0.914 in 2011-12. As stated, there are a variety of reasons for this, but that’s not the point of this post. The point is that we can’t just look at a given goalie’s save percentage over his career without adjusting for this baseline increase across the league.

One way to do this is to measure how many standard deviations a particular goalie is away from the average of save percentage (SV%) of all goalies playing that season. This is a fair approach that should be able to weed out some of the statistical noise in averaging individual SV%. However, instead of averaging SV% I prefer to use overall league SV% as my baseline.

To do that, I calculate SV%(NHL) as (Total Shots Against – Total Goals Against) / Total Shots Against. To me, this properly weights the overal SV% in the league and eliminates the statistical noise that comes from averaging individual SV%.

Once this is done, we can simply index each goalie’s SV% in a given year to the NHL SV% as a straight ratio, so for a given year:

SV% Indexed(year) = SV%(year)/NHL SV%(year)

So an index of 1.0 means you were right on the overall league save percentage, while an index greater than 1.0 puts you above the baseline.

I don’t believe using standard deviations would result in dramatically different results. It would likely just tend to cluster the data a bit closer around the baseline. 

Either way, this is the easy part. Lets see if we can tackle the issue of survivorship bias in the data.

Dealing With Survivorship Bias

I propose to deal with this problem by comparing the performance of goalies as they age not to the league SV%, but to their own performance. Survivorship bias arises because the sample of older age goalies essentially self-selects. So if only the better goalies survive to play into their late 30s, of course you would expect them to be better than the baseline NHL SV%.

But what if we compared goalies to themselves when they were younger? That should give us a true look at how performance changes as goalies age. I propose to do this as follows:

  1. Index each goalies’ SV% to the baseline NHL SV% as per above.
  2. For each goalie, find the age at which they hit their peak SV%(index), set that year at 1.0 and index the rest of their career to that peak year. 

So for a given goalie, their index will be 1.0 at the age that they performed the best in relation to the league baseline, and all other ages will show as some fraction < 1.0, unless they managed to hit that peak in multiple seasons. What this means is that even goalies that were never better than the league baseline will have a 1.0 index year. It may not have been stellar compared to the rest of the league, but it was their best year.

What this does is allow us to compare performance over a career regardless of individual skill differences. You can now compare how any goalie changes from their peak year over time.

Data Filters

One final bit of discussion before we get to the analysis. We need to filter some of the data so that we’re looking at a representative sample for our purposes.

First, I want to thank Steve Burtch for the data set, which he sent me months ago when I first started looking a whether we could expect Luongo to maintain his performance level for a few more years. The data includes all NHL goalies going back to the 1990-91 season.

We need to limit the sample to goalies that had some longevity in the league, so I’ve set a minimum of 250 career games played to be considered. In addition, in any given season, I’ve set the minimum number of shots against at 750 in order to (a) ensure the SV% for that given season was based on a large enough sample, and (b) to further filter out the impacts from career back-ups that might have met the 250 GP minimum but didn’t log more than 20 or 30 games in a season.

What this means is that although a given goalie might make the 250 GP cut-off, only their seasons with 750 or more shots against are included in the analysis.

Here’s what the overall data sample looks like in terms of how many “seasons” are counted by goalie age:

Seasons that meet criteria

What this tells us is that we shouldn’t give too much value to the results for ages outside the 21-37 range. In fact, there are only nine seasons by 21 year olds, so even that value is suspect.

So with that in mind, let’s get to the results.

The Impact Of Aging On Goalie Performance

When we plot this meta index by age, it’s pretty clear that career performance peaks at 24-25 for goalies, on average. The decline is pretty steady through the late 20s, but it tends to plateau a bit after 30. The interesting thing for me, and for those wondering about whether Luongo still has a few good seasons left in him, is that this plateau stretches out to 37.

Interestingly enough, that’s the age at which Patrick Roy decided to call it quits. Did he know what was coming?


Indexed goalie performance

I should note that In this case, error bars showing standard deviation would probably be useful. I chose to just go with mean and median, as a rough visual indication of variance, because the error bars would be a bit more distracting, but showing the standard deviation would better illustrate the uncertainty associated with the different sample sizes at each age, as per the bar chart above.

One caveat I would add to this analysis is that despite the indexing to try and make individual career performances comparable, these results are still in aggregate. The results for elite goaltenders will get diluted by the inclusion of everybody else, even with the data filters I’ve used.

So just for interest’s sake, let’s take a look at how some of the game’s top goalies have fared.

Comparing Elite Goaltenders

For this, we’ll go back to just the basic index of SV% to the NHL SV%, i.e. the NHL baseline for each year is 1.0, and if you’re better than the baseline, your index is higher than 1.0.

How do Luongo and Lundqvist compare to Roy, Hasek and Brodeur? Pretty much on par:


Performance of top goalies by age

Keep in mind that Hasek didn’t really get a starting job until later in his career, and that Roy’s earlies years aren’t in the data set, which starts in 1990-91. But overall, Luongo and Lundqvist are basically in the Roy and Brodeur territory. Nobody is near Hasek from 29-34, though. He was miles ahead of the rest of the league during that period.

Based on the preceding analysis, I believe it’s clear that goaltender performance does degrade with age and the peak is pretty early in most careers. But on average, most goalies can maintain a decent performance well into their 30s.

Barring any injury troubles, I would expect Luongo to maintain a similar performance plateau for another four years or so.