Evaluating Nameless Players

Evaluating hockey players can be difficult. Fans see and remember different things, and we all suffer from an inability to store all of the data that we observe in our memories. One of the best articles to address these points is a piece by Derek Zona written in 2010. Here is my favorite part of that article:

In the world of sports fans, confirmation biases abound. It's impossible for individual fans to record, catalog, process, analyze and interpret the results of hundreds of independent events occurring constantly throughout a game, but it's much easier to pick out those events and sequences of events that support their conclusions. Any hockey fan that has sat silently shaking their head while the crowd piles on an undeserving player recognizes this immediately. It's a powerful psychological force, especially in a setting like sports. Fans can confirm their biases for themselves and immediately fall back on thousands, sometimes hundreds of thousands of fellow fans to confirm what they already know. This is the very foundation of groupthink[.]

Even though we know this, it's still nearly impossible sometimes to break free from these tendencies. Despite a player's bad statistics, there is always the allure of rationalizing his numbers away by referencing amorphous concepts or theories that cannot be falsified.

There is, however, a nifty way to temper some of these judgmental proclivities. There is a Toronto Maple Leafs fan on twitter named Dominic Galamini. He's a very bright guy and a phenomenal hockey follow. What has recently interested me is that he has been putting up statistical comparisons of two players that obscure their identities in the process. Here is an example.

With Dominic's permission, I decided to do these same comparisons for various players on the Penguins (current and former). The tables below contain a litany of stats for two players (A and B) aggregated from the start of the 2011-12 season through the present. I'll tell you who each player is at the end of the article.

Before getting to the tables, two quick notes on the stats. First, dCorsi is a metric developed by Stephen Burtch that adjusts a player's puck possession metrics (corsi) for lots of contextual factors (age, zone start, quality of teammates and competition, team effects, etc). You can read about it here. A negative dCorsi means a player is underperforming relative to a league average player in that same role. A positive dCorsi means he is exceeding expectations.

Second, SC stands for scoring chances. War on Ice recently included scoring chance stats in their database, so we now have all of this data to pull from. SCF% is scoring chance for %, and it's simply the ratio of scoring chances in a player's favor while he's on the ice. The "iSC" stat is the number of individual scoring chances generated by that player.

Let's start off with two forwards who have had somewhat similar usage (all stats are from War on Ice and current through January 2, 2015).

Pick the one you'd want on your team. You can weigh each stat as much (or little) as you like. Don't go on War on Ice to try and figure out who these players are before making your decision!

This next chart compares two forwards who get a lot of powerplay time.

Now we will take a look at two defensemen.

Finally, we look at two goalies.

Make sure at this point that you've picked which player you want from each comparison. Weigh the stats however you'd like. Ready for names?

In the first example, Player A is........Joe Vitale. Player B? Nick Spaling. Spaling is seen as a significant upgrade over Joe Vitale. How many picked Vitale without knowing the names?

Our second example contains two high-scoring wingers with a knack for the powerplay. Player A is James Neal. Player B is........Steven Stamkos! I think this comparison is truly fascinating. Both players put up similar individual stats as to shots, goals, and points. It was remarkable to me how even they were in powerplay points.

Where there is a massive gap is in terms of puck possession. Neal wasn't being carried by Malkin; his dCorsi numbers take account of quality of teammates. What this means is that while Stamkos is a better scorer than Neal, he's a much bigger drag on possession. Knowing what you do now, do you still stand by your player A vs. B selection?

Our third example is two defensemen. This was also interesting because one of them has better individual stats in most areas of play, but the other one has had a more measurably positive impact on puck possession the last 3 and a half years given his usage. Player A is Matt Niskanen. Player B is Team Canada starter Duncan Keith. I don't think Niskanen is as good as Duncan Keith, but my guess is that there's a pretty big talent gap in the general hockey fan's mind between these two. This suggests that that gap actually isn't all that big.

Finally we get to our goalies. I picked this comparison because it puts you in a tough situation. On the one hand, we know that over the long run (which we have here), it's best to evaluate goalies based on even strength save percentage. Here, however, one of these goalies has a much better shorthanded save percentage that allows him to pull ahead in terms of overall numbers. But given what we know about even strength save percentage, should this matter?

Player A is Marc-Andre Fleury. Player B? Devan Dubnyk. Dubnyk has been a journeyman the last few years and is not considered a serious NHL goalie. I found it interesting that he was able to put up a better adjusted even strength save percentage than Marc-Andre Fleury.

***

The most important thing I want to be clear on is that I don't think you should choose players based on only what you see on paper. The point of this exercise is to illustrate how our preconceptions of certain players bias us in evaluating them. It's also interesting because it lets us highlight on-ice performance to the exclusion of the extraneous stuff that sometimes swamps debate. If the numbers don't match up with what you see/remember on the ice, that's perfectly fine. It's just important to try and articulate what exactly it is that you're seeing that the numbers are failing to capture.

I've included a poll below on who surprised you the most!