The Best Way To Evaluate A Powerplay

Watching the Penguins' powerplay this season has been really fun. They've been passing the puck well and scoring at a crazy rate. Because of this, they currently sit atop the league with a 34% powerplay conversion rate. But we know that how a team has done in the past isn't always a good predictor of how they'll do in the future.

To make the best predictions possible, we like to look at shot attempts rather than goals. On one level it's counter-intuitive, since games are decided by which team scores more. But using shot attempts has the appeal of giving you more events to analyze, which allows you to be more confident in your evaluation of a team or a player. We've seen this in countless articles, but nearly all of them deal with even strength play. I haven't come across one that documents the data to support focusing on shot attempts rather than goals when a team is on the powerplay. I intend to do that here.

It makes sense to break this problem down by first looking at big samples. This is especially important given how little overall time teams spend on the power play. What I did was pull six years of team powerplay data from War on Ice and divide that data into two buckets. The first bucket was the combined data from the 2008-09 season through the 2010-11 season. As you can guess, the second bucket was the combined data from the 2011-12 season through the 2013-14 season.

I then created scatter plots to analyze the correlation between those two buckets for Corsi For/60, Fenwick For/60, Shots For/60, and Powerplay Shooting Percentage. A higher R-squared indicates more repeatability in that stat, which is what we want. The graphs are below.

This is exactly what we thought would happen. Goals are a poor baseline to use for the powerplay because teams have no long-term control over their powerplay shooting percentage. That last graph drives the point home because it's nearly impossible to get much closer to an R-squared of zero. Looking at shots for per 60 is much better, but you can see that each metric's predictive power gets better as we include a bigger data set. CF/60 has taken an early lead.

I want to check this result another way by looking at in-season correlations. In theory, a lot can change for teams over six years. Personnel, coaches, systems, and other minutiae can all make it impossible to sustain an above average powerplay shooting percentage. However, in the more compact setting of a single season, we might see teams consistently getting higher quality shots while up a man.

To test this, I took every team's powerplay CF/60, FF/ 60, and shooting percentage for the first 41 games of the 2013-14 season and correlated that with the same metric for the last 41 games in the season. War on Ice makes this very easy since you can sort by dates. I didn't bother with SF/60 since Corsi and Fenwick are better predictors when using shot attempts (I confirmed this by running in-season correlations for SF/60). In short, we're looking to see if what teams did on the powerplay through the first half of the season had any effect on what they did in the last half of the season. Graphs follow.

This is the same thing that we saw with our six-year samples above. Teams have very little control over their powerplay shooting percentage, but their shot rates show much more sustainability, with CF/60 again the best. To make sure there aren't any holes here, I did the same in-season correlations for the 2011-12 season.

Same story. Powerplay shooting percentage showed very little predictability while CF/60 continued to reign supreme. I feel comfortable making the following claim:

When evaluating a powerplay, always use CF/60 because it is considerably more sustainable than other metrics, especially powerplay shooting percentage.

I hope Penguins fans and the hockey community more generally can use this as a reference point for evaluating powerplays in the future. Moreover, I want to apply this knowledge to this year's Penguins. The stats are from War on Ice and the ones from this year are current through Thursday evening.

	PP CF/60	PP SH%
2011-12	118.53	12.09%
2012-13	104.1	16.06%
2013-14	112.33	13.89%
2014-15	104.61	21.65%

This is remarkable. People lauded the powerplay early on in the season for doing different things and getting better results. But so far, the Penguins' powerplay actually hasn't been any different than in past seasons. In fact, it's been a bit worse. That sounds crazy because they've scored a lot of goals so far, but that's been driven by an unsustainable shooting percentage.

The metric that we should focus on is CF/60. And that tells us that the Penguins haven't uncovered a secret powerplay elixir. They generated more shot attempts in two of the previous three seasons, and even their lowest numbers in 2012-13 are right in line with what we've seen this year. Going forward, I hope the coaches and the players make getting more shot attempts on the powerplay a goal.

***

I want to head off a misconception when I talk about evaluating powerplays. Goals are what decide the game, so it seems silly to look at the Penguins' powerplay and say anything other than "they've been phenomenal." That's a true statement if our task is simply explaining what happened in the past. However, I'm more interested in the future, and I think most people are too. We want to know where our team is headed and whether they can keep it up.

That's why it's critical to use CF/60 when evaluating a powerplay. The Penguins have done a good job this year generating shots on the man advantage, but their powerplay unit has under-performed relative to its counterparts in earlier years. To maintain their success with the man advantage, they should aim to generate more shot attempts in the future. Given their talent, that seems like a reasonable expectation.