Sometimes grand ideas come in the form of a “eureka!” moment, a brief instant of intellectual clarity, an epiphany of sorts. We’ve all experienced it — that feeling of instantaneous comprehension of some previously unsolvable notion — and it’s glorious. For just that brief pause, we feel like we’re on top of the world.
As cool as it would have been, unfortunately, that is not at all how Adjusted Goals Saved Above Expected Goals Average (adjGSAxGA/60) was derived. It is simply a slightly different way of looking at things, borne from a slow-drip of new ideas and research in the analytics community, blended with an interest in the methodology behind Goals Saved Above Average (GSAA) metrics, and with just a splash of the insanity it takes to live deep inside “Goalie Twitter.” So, without further adieu, let’s jump into it.
The makings of adjGSAxGA/60 began as I was going on a bit of a rant against using High-Danger Save Percentage as a sole proxy for goaltender quality, when Emmanuel Perry, creator of Corsica, replied with this:
@imfleming16 Do you not think shifting focus to continuous shot quality measure (xG) rather than discrete (zones) is the way forward?
— Emmanuel Perry (@MannyElk) May 24, 2016
It stuck with me, to say the least, but I didn’t know what to do with it at the time.
Fast forward to the beginning of July and this post from @Fooled_By_Grit, in which he details that he believes that missed shots should be incorporated into the evaluation of goaltenders, referencing work from @DTMAboutHeart as a starting point. Both of those pieces are thought-provoking and worth a read, and they were the reason I set out to create a metric in which we could use Fenwick Against as a base statistic instead of only shots on goal.
So How Do We Build a New Stat?
To understand how adjGSAxGA/60 is built, let’s first quickly take a look at its components. It’s based upon a GSAA model, an explanation of which can be found in short form in Nick Mercadante’s article (an absolute must-read for goalie stat nerds) regarding his metric, adjGSAA/60 (“Mercad”):
Basically, take the league average sv% and apply it to the total shots faced by the particular goaltender. Out of that, you get a number of goals that the average goaltender would have given up had he faced the same number of shots as the goaltender in question.
This resulting number can be compared to the number of goals the goaltender in question actually gave up. A plus/minus is the result. If a goalie is in the positive, he is saving more than a league average goaltender might in the same situation. If the goalie is in the minus…well you get the idea.
From there, Mercadante would break down his calculations into subsets by danger zone, with data provided by (the now-defunct) War on Ice, and then recompile them to find total goals saved above average, set to a per-60-minutes rate.
Where adjGSAxGA/60 splits from adjGSAA/60 is in how, instead of weighting by danger zones and using only shots on goal, we’re going to account for missed shots as well and adjust for Emmanuel Perry’s xG model. As Perry explains in his write-up, the xG model accounts for the following:
- Shot type (Wrist shot, slap shot, deflection, etc.)
- Shot distance (Adjusted distance from net)
- Shot angle (Angle in absolute degrees from the central line normal to the goal line)
- Rebounds (Boolean — Whether or not the shot was a rebound)
- Rush shots (Boolean — Whether or not the shot was a rush shot)
- Strength state (Boolean — Whether or not the shot was taken on the powerplay)
Now, gone being the convenience of breaking down by War on Ice’s danger zones, we must find a different way from Mercadante to standardize shot volume and quality against individual goaltenders. We do this by adjusting each goaltender’s average xG against to the league average xG against, using that and the goaltender’s unblocked shot (Fenwick) volume to calculate for an adjusted goals against (adjGA), finding the difference between adjGA and the expected goals against based upon league-wide average expected Fenwick save percentage (xFSv%), and setting it to a per-60-minutes rate. If that all sounds a bit convoluted, it really boils down to this: how well does a goaltender both save shots on goal and influence shooters to miss wide if he faces league-average shot quality?
Taking It from Theory to Practice
To see how this all shakes out in real terms, let’s look at a 3-year 5v5 adjGSAxGA/60 from 2013–2016, with an arbitrary minimum of 3000 unblocked shots against:
As expected, Carey Price and Henrik Lundqvist are head and shoulders above the field; and rightly so, as any conversation about the top goaltender of the current era begins and ends with them, whether your method of evaluation is rooted in analytics or the eye test. After them, much of the rest of the field shakes out as we may expect, with a few unexpected performances dispersed throughout.
Cam Talbot, for one, immediately jumps off the chart as a surprise in how highly he ranks. While his adjustment to new surroundings in Edmonton has not come easy (-0.1915 adjGSAxGA/60 on 1763 unblocked shots in 2015–2016), his superb play in New York the previous two seasons (0.6079 adjGSAxGA/60 on 602 unblocked shots in 2013–2014; 0.4653 on 1244 in 2014–2015) propels him to a spot among the top goaltenders in the league over the last three seasons. A partial explanation can be found when looking at the boost in GSAA Talbot enjoys when jumping from adjGSAA/60 to adjGSAxGA/60, as 2013–2014 and 2014–2015 were two of the most successful seasons since 2007 for any goaltender in that respect — lending itself to the idea that he is consistently able to influence shooters to miss the net at an above average rate.
On the opposite end of the spectrum is Pekka Rinne, Nashville’s uncontested starter of the last 8 seasons. Accounting for Rinne’s time on ice over the previous 3 seasons, his -0.2305 adjGSAxGA/60 stretches to roughly 27.5 more goals allowed during that time than the hypothetical average goaltender. Now compare that to Carey Price, who, given an equal amount ice time and average xG shot quality to Rinne, saves 57.2 goals above the average goaltender, for a total difference of nearly 85 goals between Rinne and Price over that time. Indeed, Rinne’s GSAA rates fall even further when shifting from adjGSAA/60 to adjGSAxGA/60, indicating that, despite being one of the bigger goaltenders in the NHL at his listed height of 6’5″, he is not particularly strong in influencing missed shots. Anecdotally, it is remarkably odd that a team would not scale back a goaltender’s playing time given this type of performance, and the problem only compounds upon itself at the team-building level, where the average annual value of Rinne’s contract is $7 million, leaving the Predators to pay star-level wages to a goaltender who gives them well-below average output.
Where Do We Go from Here? (Limitations and Predictive Ability)
First of all, we need to accept that there are limitations to the specificity of the data. Using Perry’s fantastic xG model is a start toward adjusting for the quality of shots that goaltenders face, but averaging out the xG quality of all shots and deriving a form of GSAA is akin, albeit much more specific, to using an unfiltered GSAA/60 instead of breaking down by danger zone to calculate adjGSAA/60. It is adjustment by brute force, like using a hand grenade to clear out space in the yard for your new garden. It works, but it lacks intricacy, detail.
Second, this model values missed shots equal to saves. Assuming future research is done in this area, whether by myself or others, it is my inclination that the assumption of equal value will not hold. When there is a more definitive weighting of these values, I will happily update the model.
Next, predictivity. This is really the meat of the issue and where things get particularly sticky, if not very interesting. Developing a quality means of predicting the future performance of goaltenders is a sort of holy grail for those of us who have a soft spot for “voodoo.” I intend to write a subsequent article that digs further into the underlying data for adjGSAxGA/60 in an attempt to make incremental gains in predicitivity. “Incremental” is the key word here, but I’m encouraged by early results with Marcel projections, a method of forecasting originally developed for use in baseball analytics. For further reading regarding goaltending Marcels, I advise reading garik16’s work here (with earlier work done by the venerable and oft-cited Eric Tulsky here).
Last, but not least, the name —” adjGSAxGA/60″ is an abomination. I leave the door open to the public to come up with something more fun and accessible, but I have to warn you, Emmanuel Perry already has the inside track:
— Emmanuel Perry (@MannyElk) July 3, 2016
So it goes.