NBA Draft Lottery Study: Part One - PER

Over the course of the next week or so, Mitchell Maurer and I are releasing a study on the relative productivity of NBA Draft lottery prospects and the draft pick slots associated with those picks for the past decade. For the sake of simplicity and ease of inter-positional player comparisons, we opted to use John Hollinger's Player Efficiency Rating (PER) as our primary vehicle to evaluate player performance in our study. Our ultimate goal is to help demystify the process of reasonably projecting the careers of draft lottery talents. We hope the study will be entertaining, informative, and most of all, the start of a great conversation on the draft, so please think of PER as a convenient tool or a proxy for performance evaluation rather than our final word on the true value of each player.

Our study will be released in three parts. Part One (this article) will introduce PER and raise the most salient issues regarding the strengths and weaknesses of the metric. Part Two will introduce our methodology by analyzing the productivity of the current Bucks roster. Finally, our study will culminate in the massive Part Three NBA Draft Lottery analysis that applies the information and methodology set forth in the initial two posts. Enjoy!

In order to get the most out of our study it is important to understand what PER stat is trying to tell you and what PER cannot tell you. I have compiled the five most important things you need to understand about the strengths and weaknesses of PER as a performance evaluation metric, so let's get the discussion started.

(1) PER takes the events recorded in a traditional box score and attempts to assign value to each of these events to summarize a player’s statistical accomplishments in a single number. In Hollinger’s own words, PER accounts for "positive accomplishments such as field goals, free throws, 3-pointers, assists, rebounds, blocks and steals, and negative ones such as missed shots, turnovers and personal fouls." Although the actual formula is exceedingly complex, the basic point is that the value of these events are informed by Hollinger’s instincts (and probably a proprietary regression model) and then ultimately weighted against corresponding league averages.

(2) The resulting PER value is calculated on a per-minute basis and adjusted for team and league pace (the number of possessions used per game), purportedly allowing for apples to apples comparisons between players with otherwise unequal opportunities. Hollinger normalizes PER values such that the league average in PER is always 15. From there the values can vary significantly. For example, among players with at least 500 minutes in 2009-10, the highest rating was LeBron James (31.19), and the lowest rating was Quinton Ross (3.04). Here is an informal and un-official Wikipedia guide to interpreting PER values:

35.0 - A Year For the Ages

30.0 - Runaway MVP Candidate

27.5 - Strong MVP Candidate

25.0 - Weak MVP Candidate

22.5 - Bona fide All-Star

20.0 - Borderline All-Star

18.0 - Solid 2nd option

16.5 - 3rd Banana

15.0 - Pretty good player

13.0 - In the rotation

11.0 - Scrounging for minutes

9.0 - Definitely renting

5.0 - The Next Stop: D-League

(3) PER values below 15 are not necessarily bad, because sheer durability often has value in its own right in professional sports. For those of you baseball fans out there familiar with the concept of replacement level (in the case of basketball, it is the level of talent readily available to be signed out of the D-league), Hollinger has done an additional multi-season analysis of players who played less than 500 minutes in a season and determined positional replacement levels according to PER. The league average replacement level PER is 11.0, and the position-specific values are:

11.0 for PGs

10.5 for SGs/SFs

11.5 for PFs

10.6 for Cs

(4) PER most certainly does not value defensive contributions fully or properly, because it only takes inputs from traditional box scores (which only record blocks and steals). Hollinger has admitted this is a significant limitation of his model, saying this:

"Bear in mind that this rating is not the final, once-and-for-all answer for a player's accomplishments during the season. This is especially true for players such as Bruce Bowen and Trenton Hassell who are defensive specialists but don't get many blocks or steals… What PER can do, however, is summarize a player's statistical accomplishments in a single number. That allows us to unify the disparate data on each player we try to track in our heads so that we can move on to evaluating what might be missing from the stats."

Therefore, please keep in mind that PER is largely an offensive metric meant to be the start of a discussion on player value, not the final arbiter on the subject. My impression from the comments I excerpted above is that after seeing a player’s PER rating, further discussion and analysis is not merely recommended, it is mandatory. If you like PER for measuring offense, one way to make more complete observations is to use the information available at http://www.82games.com/. The site provides data on the PER of a player’s positional counterpart during their time on the court, so you can get a rough idea of how effective a player is on limiting the offensive production of opponents. However, this counterpart metric is also far from perfect, as it makes an initial assumption in assigning positions to players on the court (ex: who is the Center when Ilyasova and Sanders are on the floor without Bogut or Gooden?), it makes an additional assumption that a PG guards the opposing PG (and so forth), and it cannot account for zone defensive assignments, switches on picks, or the impact of help defense. A small consolation is that the site also provides other valuable counterpart stats such as eFG% and the percentage of shots the counterpart takes close to the hoop.

(5) There is a debate about whether PER inherently rewards shooting. This criticism has been made by David Berri, the author of Wages of Wins, who said the following:

"Hollinger argues that each two point field goal made is worth about 1.65 points. A three point field goal made is worth 2.65 points. A missed field goal, though, costs a team 0.72 points. Given these values, with a bit of math we can show that a player will break even on his two point field goal attempts if he hits on 30.4% of these shots. On three pointers the break-even point is 21.4%. If a player exceeds these thresholds, and virtually every NBA player does so with respect to two-point shots, the more he shoots the higher his value in PERs. So a player can be an inefficient scorer and simply inflate his value by taking a large number of shots."

This feature of PER should not simply be dismissed off hand. Berri believes that PER inherently rewards shooting, and based on those low thresholds the players with higher usage percentage (an estimate of the percentage of team possessions used by a player while he was on the floor) benefit merely from taking more shots than their teammates. For a more detailed study on the issue, check this out.

However, Hollinger has a pretty convincing counter-argument...by which I mean he says Berri has no idea what he is talking about.

Berri leads off with a huge misunderstanding of PER -- that the credits and debits it gives for making and missing shots equate to a "break-even" shooting mark of 30.4% on 2-point shots. He made this assumption because he forgot that PER is calibrated against the rest of the league at the end of the formula.

Actually, if we took a player was completely average in every other respect for the 2006-07 season -- rebounds, free throws, assists, turnovers, etc. -- and gave him a league-average rate of shots, and all of them were 2-pointers, and he shot 30.4%, he'd end up with a PER of 7.18. As long-time PER fans know, that would make him considerably worse than nearly every player in the league.

To end up with a league-average PER of 15,00, the actual break-even mark in this case is 48.5%, which is exactly what the league average is on 2-point shots this season.

So Hollinger says that his PER formula is actually calibrated against the rest of the league to create a break-even point exactly at the actual league average each season. I am inclined to agree with Hollinger, seeing as he created the formula and certainly understands its intricacies better than Berri. It seems pretty clear to me that Hollinger has a reasonable response to Berri, but I felt I should present the debate to you and let you make up your own mind because some people have sided with Berri on this issue, and even the Wikipedia page for PER presents Berri's argument as a valid criticism in a section called "Problems with PER." 

In any case, the question ultimately raised by this final point is how shot-creation should be valued, but I must emphasize that PER is unequipped to answer this question in any meaningful way. A player who creates a decent shot from a desired spot on the court while the shot clock is ticking down to zero has helped his team by doing so, but PER cannot and does not know when shots are taken in such a context. PER gets information exclusively from traditional box scores, but they make no record of context for shots. Box scores do not care when a player gets off a good shot at the end of the shot clock, or when a player mistakenly passes up a layup for a sub-optimal fade away jumper, or even when a player throws up a desperation half-court heave at the end of the half. Because box scores never record this information, PER simply cannot take the context of a shot into account.

Thanks for reading, make sure to check back for parts two and three of our PERfect Draft Study (we're pretty punny, aren't we?), and ignore these five points about PER at your own risk.