Data Deep Dive: Fall Season ShowBuzzDaily Accuracy

Traditionally I’ve relied on ProBoxOffice.com forecasts in my analysis but several readers and regular forum posters have pointed out the value of ShowBuzzDaily as well. Two weeks ago I looked at ProBoxOffice.com accuracy over the first six weeks of the Fall season so for an apples to apples comparison, I’ll give ShowBuzzDaily the same treatment here.

Why should you care?

Professional forecasts are helpful when compiling lineup possibilities each week of the Fantasy Movie League season. Understanding the limitations of those forecasts, as evidenced by “Jem and the Holograms” this past week, is important.

Where’d I get my data?

ShowBuzzDaily forecasts for the first six weeks of the Fall FML season and weekend actuals from BoxOfficeMojo (who will get a similar analysis of their forecasts next week).

What were my findings?

The easiest way to look at this is to simply compare the two forecasts vs actuals graphs:FallProBoxOfficeAccuracyWeek7

FallShowBuzzDailyAccuracyWeek9

The first thing that jumps out is that ShowBuzzDaily has a much tighter bounds on new films in this sample, from -1.26% to -9.73%. Their hit ratio on new films is slightly less than that of ProBoxOffice.com, but when they hit the bounds are much smaller.

For returning films, the patterns between the two sites are pretty similar, with ShowBuzzDaily not doing quite as well with films in its 3rd week of release with only 58.3% of films in a decent range, but again we see that their ranges are slightly smaller.

As I was playing around with this data, though, I made a nice discovery: ShowBuzzDaily and ProBoxOffice.com were often oppositely incorrect about individual films. In other words, ShowBuzzDaily was commonly high on its forecast for a film that ProBoxOffice.com was low on and vice versa. If you average the two, you get the following:

FallAveragedAccuracyWeek9

While the new releases don’t get much better with this approach, the other groupings do, making the bounds closer to +/- 10%. This is why, starting last week for forecasted films, I began to update the FML Comparator with ShowBuzzDaily numbers on new releases but an average between ShowBuzzDaily and ProBoxOffice.com for returning films and changed my variance simulator to run +/- 10%.  These numbers reinforce the idea that the professional forecasters are quite good and fairly predictable when it comes to returning films.  The challenge, really, is those truly outrageous new films.

Next week I’ll run a similar analysis on BoxOfficeMojo forecasts. That typically comes on on Thursdays, which doesn’t help my publishing schedule since I write my weekly picks articles on Wednesday evening so they can get published on the main FML site on Thursday morning, but may reveal something else useful for all of us.

Bonus Graph

With the large number of people who had Perfect Combinations this week, I thought it was worth revisiting this graph I produced when the scoring changes hit at the beginning of the Fall season:

PerfectLineupWeek9

This past week has the highest percentage of players ever reach Perfect Combo (5.86%) although not the most by volume.  Most weeks are still well below 2%, meaning that Perfect Combos are typically rare.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s