Saturday, January 12, 2013

4th and 1 Conversion Rate and Team Strength

This week at Advanced NFL Stats, Jack Moore published a post on passive 4th down decisions in last week's Wild Card games.  According to Jack (and his slick visualizations), coaches forfeited a total of 0.24 in win probability as a result of choosing field goals and punts on fourth down instead of an attempted conversion.

A common objection I see from fourth down skeptics has to do with the probability of a team converting on fourth down.  The skeptic will argue that the probability of conversion is based on league averages, but a true expected conversion rate should vary by team strength (good teams should have a higher probability of converting than poor teams).

While I don't doubt this is directionally true, my assumption has been that the conversion rate simply doesn't vary that much by team strength, and so most fourth down analyses are still valid when using league average rates.

I decided to see if my assumption was correct by looking at how converting a 4th and 1 varies by the point spread of the particular game; where the point spread is being used as an a priori proxy for relative team strength  If you know of a better one, please let me know (I would like to make some money).


How Often Do 7 Point Underdogs Convert a 4th and 1?


For this analysis, I merged the play by play data provided at Advanced NFL Stats with point spread data from sportsdatabase.com.  I then looked at how conversion rates on 3rd and 1 and 4th and 1 varied by the point spread.  While 3rd and 1 situations aren't exactly like 4th and 1's, they are close enough that I thought it was a worthwhile view.

Here are the results (mouse over the graphs for details):







Some observations:

  • Conversion rate is correlated with point spread, but weakly.  For evenly matched teams, the expected conversion rate is about 68%.  For a seven point underdog, that only drops to ~65-66%.
  • The third and fourth down results are pretty consistent, with the fourth down results being a bit noisier due to sample size.
  • I plan on redoing this analysis at some point where I factor in the betting over/under.  By combining the point spread and the over/under, you get a more direct measure of how the market expects a team's offense to stack up against its opponent's defense.  It is possible a stronger correlation will emerge.
  • 90% of NFL games have a point spread of 10 points or less.  Within that range, there is only a +/-4% variation in expected conversion


Conclusion


Short yardage conversion rates only show a small variation due to team strength, as measured by the point spread (even a badly overmatched team can be expected to convert a 3rd and 1 or 4th and 1 the majority of the time).  So, analyses of fourth down decisions that use league averages for conversion rates are most likely still valid, even in the case of mismatched teams.

Additional Notes

  • I used all plays from the 2002 - 2011 seasons, totaling 7,913 plays for 3rd and 1, and 1,982 plays for 4th and 1.
  • I did not exclude any plays (4th and goal is included, desperation plays at the end of the game, etc.)
  • My expected conversion rates seem a bit lower than what you can find at the Advanced NFL Stats awesome fourth down calculator.  However, I was able to tie my numbers to the Pro Football Reference equally awesome Game Play Finder.  If the Advanced NFL Stats numbers exclude certain situations, that could be driving the difference.

3 comments:

  1. Brian Burke uses 3rd down conversion rates to supplement his data, and they seem to be higher than 4th down ones. (The usual disclaiimers apply of course.)
    Speaking of that, I just loked at the correlation between 3rd and 4th down conversion rates, and it's about 0.3 - much lower than I expected. Though this may be due to small samples.

    ReplyDelete
  2. But "better team overall" doesn't mean "better at every single aspect of the game". Sometimes the better team will be worse at 4th and 1, or the worse team will be better at it.

    The real question this article should have been asking was what is the 4th&1 conversate rate disparity between the best short-yardage team and the worst short-yardage team, not the disparity between the best overall team and the worst overall team.

    ReplyDelete