Data Deep Dive: Fall Season Accuracy

With better pricing and smaller movie-going crowds, the Fall Fantasy Movie League season has proven to be far more difficult than its predecessor to predict.  “Maze Runner” this past week, “The Intern” the week before that, and The Great “Everest” Whiff of 2015 during Week 4 have all shown this so I thought it was worth revisiting the accuracy of forecasts.

Why should you care?

Many players rely on professional forecasts for guidance when making selections on a weekly basis and understanding the bounds of the accuracy of those forecasts is critically important.

Where’d I get my data? historical forecast and weekend actuals for the first six weeks of the FML Fall season.  Films were included in the sample if they had a forecast and were included among the 15 films FML players had to choose from that week.

What were my findings?

For the Summer season I found that 74% of forecasts were between +10% and -20% of being correct but it has been suggested to me by multiple players that movie going audiences are very different for the present time of year so those numbers don’t necessarily hold for the Fall.  Further, it has also been suggested that new releases be separated with others.  Based on Ari’s article last week, I reran my analysis with a twist to call out not just debuting movies, but to group movies by week of release.  When you do that, you get the following graph:


This puts more precision on Ari’s assertion from last week that is quite good when you throw out the first week a movie has been released.  In particular, they do even better than the +10%/-20% range found when lumping all movies together as I did during the Summer.  There isn’t much data for movies beyond a 4th week of release, which can be attributed to the idea that only forecasts 10 movies or so each week and not many stay in the top 10 that long.  Still, for those that are this sample shows none of them performing over expectations.

Not surprisingly, there is far more variance for that crucial first week.  Flip a coin and that’s the odds that a movie will do +/- 20% of its forecast.  That percentage along with the wider spread go a long way towards explaining the outcomes we’ve seen so far this season.  More accurate pricing from the FML team, though, has also made it more difficult even for films that have already debuted when it comes to making guesses about where along that +10% to -20% spectrum a movie might land on a particular week.   Put together, an that’s the hard part of this game now.

2 thoughts on “Data Deep Dive: Fall Season Accuracy

  1. Your data confirms my suspicion that Pro.BoxOffice consistently over-projects the holdover films in it’s weekend estimate. In the second, third, and fourth weeks, the negative variance from their projection was much higher (nearly double) than the positive variance. Now it begs the question – I wonder why that is?


    1. In part, I suspect it’s because that data has been used for years by theater managers to project staffing needs and maybe they figure it’s better to be a little high. That’s speculation on my part though.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s