Thursday, August 22, 2013

Is the NFL Betting Market Getting More Efficient?

Bookie's Office sign - - 485093Advances in computing power and data collection, along with smarter algorithms to make sense of it all, has led to improvements in prediction accuracy across an array of fields, from weather forecasting to what kind of movies you might like. The purpose of this post is to see whether similar gains have been made in the prediction of NFL game outcomes, where I am using the Vegas point spread as a proxy for overall advances in predictive accuracy.

Are the Bookies Getting Smarter?

To be honest, I know very little about how the sports books set their point spreads in the "old days", but my mental image of how it worked involves some cigar-chomping guy named "Lefty" or "Ace", poring over the sports pages and taking in tips from his network of informants (in other words, I have watched Casino several times). Surely the army of stat-crunching nerds that has amassed in that time and the wealth of data they now have at their disposal would lead to a more efficient market? One that is demonstrably better at picking winners and predicting margin of victory? Even the analytical muscle of Wall Street is now in the game, with Cantor Gaming (a division of Cantor Fitzgerald) providing point spreads for various major sports books.

Thanks to, we can see if this is actually the case, as they have archived game results and point spreads for the NFL, going as far back as the 1989 season. Here are the results.

66.8% of the time, we're right all the time

The graph below charts the predictive accuracy of the point spread season by season. In other words, how often the Vegas favorite actually won.

If you can spot a trend in that scatter of data points, you may have a future in reading palms. What the data seems to show is that Vegas can pick winners correctly in 2 out of 3 games, and while they have some up years and down years, they don't seem to be getting any better or worse. If you split the data in half, the point spread accuracy was 66.7% from 1989-2000 and 66.9% from 2001-2012.

Margin of Victory

Binary predictions like win/loss can be noisy, so maybe a clearer signal will emerge by focusing on how closely the point spread predicts margin of victory? The graph below charts, by season, the Mean Absolute Error (MAE) of the point spread (i.e. the absolute value of the difference between the point spread and the actual margin of victory).

Once again, there doesn't appear to be any improvement, with the average miss at around 10.3 points.  The average miss from 1989-2000 was 10.2 points and 10.5 points from 2001-2012.

Uncertainty is a feature, not a bug

Maybe it's just not possible to consistently predict the outcome of NFL games more than the long term average of 66.8%. The NFL (and all professional sports) owe their popularity to a proper combination of drama and athleticism. Unpredictability is there by design. The rules of the game, both on the field (two minute warning, clock stoppages late in games) and off the field (draft position, salary caps) are crafted with that need for drama in mind. A league in which 80% of games have outcomes that can be predicted accurately ahead of time would simply not be as successful.

Bonus graph: accuracy by week of the season

From the same dataset, I was also able to summarize accuracy of the Vegas point spread by each week of the season. Once again, the results surprised me. Accuracy doesn't seem to improve as the season goes on. From my data-centered view of things, I would have expected better results in the second half of the season. There's only so much one could learn (one would assume) by analyzing roster moves, practices, and pre-season games, so I expected early season games would be more of a crapshoot. Not the case:


  1. Perhaps teams are more evenly matched now than they were 20 years ago. In that case the bookies might be getting better, but it wouldn't be reflected in the point spreads.

  2. That's a good point, but you would expect the mean absolute error to shrink in that case. It seems to be getting slightly larger, but I have a feeling that is due to the massive increase in overall scoring over the last couple decades.

    1. My thoughts as well. Scoring margins are more volatile now, as one would expect with an overall increase in scoring. From 1989-2006, the standard deviation of the scoring margin was 14.3 points. From 2007-2012, it was 15.5 points.

    2. Interesting article. Love your analysis as usual, Michael.

      I did wonder about the same thing as anonymous. Don't have sufficient expertise in statistics to decare that although MAE had been flatlining the last few 2-3 decades. the bookies were actually getting more accurate because as NFL becomes increasingly higher scoring, it becomes more difficult to maintain the same MAE. I doubt it somehow, though.

    3. Increased scoring doesn't necessarily mean increased variance. Consider the following two scenarios:
      50% chance of 0 points, and 50% chance of 7 points
      50% chance of 3 points, and 50% chance of 7 points

      Clearly, the second scenario is going to have more scoring, but lower variance. There are other high-variance plays, but I wonder how well mean absolute spread error corresponds with mean absolute turnover difference.

  3. Let's assume - for the sake of discussion - that a turnover is worth net about 7 points, and that they happen randomly at a rate of about 3 per game. Then we'd expect the average absolute swing due to turnovers to be (roughly) 1/4*21+3/4*7=10.5.

    Surely, there are other factors and turnovers probably aren't quite that random, but this does illustrate that 10.5 could be a small average absolute error considering the rate of high-variance events in football games.

    If I'm feeling industrious, I may look at how absolute error matches up with turnover count later.

    1. I ran the regression, and the average error against the spread per turnover is only 4.1 points, so that accounts for about 60% of what you're seeing.

    2. I think that's a good way to look at it and allows one to set a lower bound on the theoretical minimum level of error one could expect when predicting NFL scoring margins (assuming turnovers are largely unpredictable).

    3. The more I think about it, the more it seems that identifying and quantifying error contributions would call for some sophisticated math. For example, naively throwing the approximate 1.5 points of home field advantage error that seems to be present into the mix doesn't seem to affect the average absolute error.

      The question of how much of an edge can be had is, of course, quite salient, so I'll be thinking more about it.

  4. Huh, no nested replies in blogger. It turns out that turnovers are at least little predictable: For example, the New England Patriots, at home, have a turn over differential of +0.7 per game, and the league average for home teams is 0.092 or so.

  5. Option adjusted spread trading exchanging is basically open doors when bookmaker's costs contrast by an edge enough to permit danger free speculations.

  6. Thanks for making such a cool post which is really very well written.will be referring a lot of friends about this.Keep blogging

  7. It is therefore recommended that you place your bets with the best betting sites so that you know that you are dealing with the reliable and genuine people who would make all your possible timely payments, in case you win the bet. zcode system

  8. To Ivan

    What can you tell me about best cheltenham bookmaker offers
    ? Do you agree with these suggestions? If you got more questions, please email me and realize what else we can do. Thanks!

  9. This is a great article. It gave me a lot of useful information. thank you very much..This is a great article. It gave me a lot of useful information. thank you very betting