A commonly used measure of goalkeeper performance is the percentage of on-target shots saved called the Saves-to-Shots Ratio or SSR for short. However, as a measure of shot-stopping ability, SSR has a weakness. Goalkeepers playing behind a weak defence potentially face a higher proportion of dangerous shots than those operating behind a solid defensive formation. So goalkeepers in weak teams have a harder job keeping the ball out of the net, and therefore a lower SSR than their ability would suggest.
This post describes a metric that levels the playing field, which I call S3R (Standardized Saves-to-Shots Ratio). S3R provides an improved measure of shot-stopping ability by adjusting for save difficulty.
Modelling save difficulty
Intuitively we know that some shots are more difficult to save than others. For example, shots taken close to goal are more dangerous than those struck from further out because the goalkeeper has less time to react; shots into the top corners are likely to score because the goalkeeper has to move a long way; deflected shots are difficult to save because the trajectory of the ball suddenly alters unpredictably.
But we need to measure the difficulty of making a save more precisely. Fortunately, Opta records detailed information about each shot, so we can use their data to build a model that tells us how difficult it is to save any shot whose features we know. For this analysis I used Opta data on 6,596 on-target shots from the EPL seasons 2010 and 2011.
The shot attributes used in the model are shown below (full details are in my JQAS paper Evaluating Premier League Goalkeepers ). To give an impression of the model, save probabilities are shown for three of the spatial attributes.
Shot Attributes used to Calculate Save Difficulty
The model was quite good and correctly predicted the outcome of 81% of on-target shots.
If the story so far makes sense, we would expect that stronger teams tend to face shots that are easier to save, while weaker teams have to defend shots that are harder to save. I used the model to calculate average save difficulties for every EPL team in the dataset. The correlation between save difficulty and league points was -.51 ( p < .001), confirming that goalkeepers in weaker teams do face on‑target shots that are harder to save. Failing to account for this will lead to a systematic bias; goalkeepers in weak teams will be underestimated and goalkeepers in strong teams will be overestimated.
Using save difficulty as an adjustment factor, I calculated S3R for all goalkeepers in the data sample. The table below shows the unadjusted (SSR) and adjusted (S3R) performances. The table below shows the results for the 15 goalkeepers who played in both seasons.
The colour coding indicates whether a performance was over-rated (red), under-rated (green) or neutral (yellow) by the traditional SSR metric. An interesting example is Szczesny, whose SSR dropped 7 percentage points in 2011, an apparently alarming loss of form. However, his S3R only fell 2 points, suggesting the drop in form was an illusion; the real change was that the Arsenal defence conceded substantially more hard-to-save shots. Similarly, about half of Friedel’s apparent improvement following his move from Aston Villa to Spurs (an uptick of 9 points) disappears when we examine his S3R, which improves by just 4 points.
Season-on-season variations in SSR are quite large, because they are typically based on about 150 shots which is quite a small sample for estimating a proportion with high statistical precision (Colin Trainor has some illustrative examples). However, comparison of mean absolute differences (MAD) between seasons indicates that S3R is more stable than SSR. For the 15 goalkeepers who played in both seasons, the MAD for the SSR was 5.4 percentage points as compared to 3.9 for S3R.
Clearly that is a small sample of keepers, but the indication is that S3R contains less random error than SSR, and is therefore a more reliable indicator of shot-stopping ability.
 Please email firstname.lastname@example.org to request a copy of the full article.