Note: The NBA has suspended the 2019/2020 season as a precaution given the COVID-19 pandemic. However, the analytical question remains just as interesting.
By Mike Armson and Manoj Raheja
Flexing our analytics muscles means approaching and solving problems with a scientific and actionable perspective. Our process – called ‘Write the Speech’ – involves four steps:
- Think carefully about the desired outcome,
- Identify the decision we’re trying to inform,
- Generate hypotheses, and
- Do the research to confirm or deny the hypotheses.
By following these steps and starting with the end in mind to help focus research, we avoid analysis paralysis and arrive at insights that are not only interesting but also actionable.
We are researchers and analytics experts, and we are also huge sports and Toronto Raptors fans. That means we love solving data problems particularly when they take advantage of freely available statistical datasets that emerge from sports. As we head into the homestretch of the NBA season, we decided to give ourselves a hypothetical analytics project. We imagined a problem that Nick Nurse, the Head Coach of the Toronto Raptors basketball team, might bring to us:
‘So far this season, it feels like the referees are calling fouls on the Raptors more often than for other teams. It seems that we get fouls called against us when we didn’t commit them, and our opponents aren’t getting called for obvious fouls. We’ve seen other top 10 teams get the benefit of the doubt from referees on split-second, could-go-either-way calls and as defending NBA champions, the Raptors should too. Before going into the playoffs, I want to understand whether my perceptions are accurate and make sure we’re not getting penalized more often than we should. If we are more likely to get unfavourable treatment from referees, I’d like to submit a formal complaint to the league. And if I’m going to do that, I’ll need to back up my argument with some data. Can you help me out?’
We began by reframing Nick Nurse’s request in terms of our ‘Write the Speech’ process.
- Think carefully about the desired outcome
The desired outcome is to test Nick Nurse’s theory that the Raptors receive unfavourable treatment from referees compared to other top teams, and if the data support that conclusion, to arm him with compelling statistical evidence to influence the league into having referees call fouls in the same way they do for other teams in the playoffs.
- Identify the decision we’re trying to inform
Nick Nurse wants to know if the Raptors are being unfairly treated by the referees in terms of foul calls. To look at this, we compared the total number of fouls called against the Raptors over the last several seasons to the total number of fouls called against their opponents. After subtracting opponent fouls from Raptors fouls, we created a metric we will refer to as the ‘fouls plus-minus’ score. That is, using the ‘plus-minus’ score, the Raptors could either end up with more fouls called against their opponents (a ‘plus’) or more fouls called against the Raptors (a ‘minus’). We compared the Raptors’ fouls plus-minus scores from this season to that of other top teams in the league to see how the referees’ foul calling behaviour compares.
- Generate hypotheses
The Raptors have been one of the NBA’s best teams over the last decade. Traditionally, top teams and players tend to get the benefit of the doubt from referees and receive fewer foul calls against them than lower ranked teams and players. Simply put, when making a split-second could-go-either-way foul call in the spur of the moment, referees fall back on a team’s or player’s reputation as a heuristic to help make the call. Based on this convention, one might expect the Raptors as a leading team to receive favourable treatment from NBA referees. However, despite their success, the Raptors don’t seem to be receiving the respect they’ve earned from opposing teams, the media, and most importantly, referees. With this in mind (and with an awareness of our own bias as Raptors fans), we predicted that the Raptors would have a lower foul plus-minus score than other top ten teams this season, i.e., that referees would lean more toward foul calls against the Raptors whereas other top teams would see more foul calls against their opponents.
- Do the research to confirm or deny the hypothesis
Here are the steps we followed in conducting our analysis:
- First, we found the right data to answer our question, i.e., the freely available database of team statistics for each season at basketball-reference.com.
- Since we were specifically interested in looking at foul-calling against the Toronto Raptors and other top teams, we pulled data for the Raptors as well as the other top 10 NBA teams.
- For each of the top 10 teams this season, we compiled the total number of fouls called against them as well as the total number of fouls called against their opponents.
- To calculate the fouls plus-minus score, we subtracted the number of fouls called against opponents from the number of fouls called against each of the top 10 teams.
Here’s what we found.
Over the 55 games played before this season’s All-Star Game (which falls about midway through the season), the Raptors were called for 91 more fouls than their opponents. This translates into a fouls plus-minus score of -91 and an average of almost 2 more foul calls per game against the Raptors than against their opponents. For comparison, the Milwaukee Bucks, the NBA’s top team so far this season, had a fouls plus-minus score of +124 before this season’s All-Star Game, or 2 fewer fouls per game called on the Bucks than against their opponents.
With a small sample size of just two teams, it is, of course, possible that the Bucks’ simply commit fewer fouls and the Raptors commit more fouls than their opponents. However, by increasing the sample size to include all of the top 10 teams so far this season, a clearer pattern can emerge. As you see in the chart, compared to all of the top 10 teams, the Raptors have the worst plus-minus score. In fact, it is twice as bad as how the team with the second lowest plus-minus score, the Boston Celtics, are treated.
Image source: Team logos from https://www.nba.com/teams
These results show that the Raptors consistently get called for more fouls relative to other top teams in the NBA. While it is difficult to know what proportion of the fouls called against the Raptors were legitimate and what proportion of non-calls against their opponents should have been foul calls, these data suggest that referees are less likely to give the benefit of the doubt to the Raptors on split-second decisions. Clearly, referees are treating the Raptors differently than other top teams. The statistical evidence suggest that Coach Nurse should indeed submit a formal complaint to the NBA regarding how referees are calling fouls against the Raptors.
Write the Speech Solves Problems
This is a prime example of how to tackle an analytics problem to get an actionable result. Think about the desired outcome of finding out whether there is evidence to support the claim that the Raptors get unfavourable treatment by NBA referees (and to support a formal complaint to the league about this). Identify the decision to see if the Raptors get called for more fouls compared to other top teams. Generate the hypothesis that the Raptors have gotten called for relatively more fouls this season compared to the top 10 other teams in the league. Confirm or deny the hypotheses to land on an actionable recommendation of sending a formal complaint to the NBA that the Raptors are getting treated unfavourably when it comes to referees’ foul calling behaviour.
Ready to learn more? Learn how brands and marketers can use AI to better meet the needs of consumers and customers in our 2018 white paper, The Marketer’s Guide to Artificial Intelligence. Or, learn how we helped our health and beauty client identify key target groups, determine product positioning, and predict the size of the potential audience.