Data Deep Dive: The New Fall Scoring Changes

UPDATE 9/3 ~5pm Pacific, see below.

In case you missed it, during the final week of the Summer 2015 season, Fantasy Movie League posted a series of announcements on the forums, among which included some new scoring rules for the Fall 2015 season.  The two changes are:

  • Best Performer bonus is now $3M per screen, up from the previous $2M per screen
  • A new Perfect Lineup bonus grants an additional $10M

These two changes sparked immediate debate on what impact they will have on the way FML is played.  I thought it would be interesting to go back and track two top players in the Summer 2015 season and how their battle for the top spot would have played out using the Fall 2015 scoring system.

Why should you care?

By looking at how the new scoring system might have changed the final standings of the previous season, some insights are gained in how it will impact play.

Where’d I get my data?

Sleestak and Glacier were the top two players for the Summer 2015 season going into the final week so all I did was look at their choices and subsequent scores going back to the start of Week 7.  I then applied the new Fall scoring system to that eight week sample.

What were my findings?

The payoff for being correct about Best Performer is now bigger and becomes bigger still since they are likely contributors to the Perfect Combination bonus.  In fact, all the Perfect Combination for all 15 weeks of the just concluded Summer season included Best Performer every time.

So what would have happened to Sleestak and Glacier?  Here’s a table that breaks down the differences:

Week Glacier Summer Rules Sleestak Summer Rules Difference Glacier Fall Rules Sleestak Fall Rules Difference
7 $1,148,215,702 $1,171,361,274 -$23,145,572 $1,148,215,702 $1,171,361,274 -$23,145,572
8 $1,290,259,560 $1,306,547,201 -$16,287,641 $1,292,259,560 $1,306,547,201 -$14,287,641
9 $1,495,475,880 $1,511,694,626 -$16,218,746 $1,503,475,880 $1,517,694,626 -$14,218,746
10 $1,650,422,151 $1,654,049,438 -$3,627,287 $1,660,422,151 $1,660,049,438 $372,713
11 $1,787,884,173 $1,789,871,522 -$1,987,349 $1,797,884,173 $1,795,871,522 $2,012,651
12 $1,942,662,654 $1,944,650,003 -$1,987,349 $1,967,662,654 $1,965,650,003 $2,012,651
13 $2,086,870,004 $2,087,321,455 -$451,451 $2,113,870,004 $2,110,321,455 $3,548,549
14 $2,206,965,192 $2,198,383,531 $8,581,661 $2,247,965,192 $2,222,383,531 $25,581,661
15 $2,313,122,817 $2,278,340,531 $34,782,286 $2,359,122,817 $2,302,340,531 $56,782,286

The first two columns show how their scores changed week to week under the original Summer season rules while the third column shows the difference between their scores each week.  Negative numbers mean that Glacier trailed Sleestak while positive numbers mean the other way around.  Using this method, Glacier didn’t overtake Sleestak until Week 14.  The fourth and fifth columns show how their scores would have changed had the Fall scoring rules been in place with the sixth column demonstrating the differences in their scores.

Glacier had more Best Performer selections in his mix throughout the eight week sample and, using the Fall scoring method, would have overtaken Sleestak in Week 10.  Graphically, the differences in the scoring systems looks like this:

Summer vs Fall

Sleestak would have fallen behind sooner, but would have had bigger opportunities to make up ground.  Instead of going with a more popular lineup that included “War Room” during Week 15, Sleestak went contrarian and took a chance to catch up.  Unfortunately, “No Escape” didn’t out perform “War Room”  leaving Sleestak to fall to 7th overall.

Clearly, the new scoring system will have a big influence on how this game is played.  Best Performer identification was important before, but becomes even more important now.  Lead changes in leagues should become more frequent and comeback potential is large because of the big Perfect Combination bonus, meaning that even the bottom feeders in your league have a chance to make a run at the top spot.

9/3 ~5pm Update

A commenter over on the FML cross post of this article wondered out loud how the scores compared if you took out all the bonuses.  With that in mind, I created this:

Glacier Raw Score Glacier Summer Bonuses Glacier Fall Bonuses Sleestak Raw Score Sleestak Summer Bonuses Sleestak Fall Bonuses
8 $138,043,858 $4,000,000 $16,000,000 $135,185,927 $0 $0
9 $193,216,320 $12,000,000 $18,000,000 $193,147,425 $12,000,000 $18,000,000
10 $150,946,271 $4,000,000 $6,000,000 $142,354,812 $0 $0
11 $137,462,022 $0 $0 $135,822,084 $0 $0
12 $144,778,481 $10,000,000 $25,000,000 $144,778,481 $10,000,000 $25,000,000
13 $140,207,350 $4,000,000 $6,000,000 $138,671,452 $4,000,000 $6,000,000
14 $112,095,188 $8,000,000 $22,000,000 $109,062,076 $2,000,000 $3,000,000
15 $96,157,625 $10,000,000 $15,000,000 $79,957,000 $0 $0
Totals $970,907,115 $52,000,000 $108,000,000 $1,078,979,257 $28,000,000 $52,000,000

And if you take the deltas of the three different scoring methods you get this:

Diff No Bonuses $27,927,858
Diff Summer Bonuses $51,927,858
Diff Fall Bonuses $73,927,858

Bigger bonuses mean bigger swings.  As I wrote in my reply to the original comment, I like to think that cuts both ways.  The bigger bonuses mean you can get further ahead, but they also mean you can come from further behind.

6 thoughts on “Data Deep Dive: The New Fall Scoring Changes

  1. Thank you so much for this extra work! I really appreciate having someone putting numbers to my blatherings.
    I truly wish the FML team would look at this and see it for what it represents – and why these bonuses are ruining the game.
    What I am seeing …

    From No Bonuses to Summer Rules, the difference nearly doubles!
    And then there is virtually another 50% jump to the Fall Rules!!
    It is nearly 3x the monies from Base to Fall.
    This kind of infusion of false cash creates much wider gulfs from player to player.

    This is why I disagree with your optimistic belief that this means a player can leap ahead to catch up faster. The players who have figured the game out will never give up their leads, it will only increase. It is the people who miss the Perfect lineup by one film who will quickly fall behind and get frustrated that they cannot overcome the free boost other players are getting week after week.

    And it gets worse …

    I would attribute the only reason we are seeing even the $27M difference is because of the way these players were gaming the system.
    Basically this game has us all playing with only 3-4 films per week.
    You figure out which will be the Bonus film, you load up on it and fill in with one or two predicted top earners.
    No one really considers the middle tier films. Using this week as the example, who really is looking at MI5, No Escape, Uncle, Sinister, Ant Man, Gift, Minions, Hitman, AU, Vacation or WaYF? No one … these are pointless films once you figure out that it will come down to some combination of SoC, WitW, WR & Transporter.
    So the $27M represents the crazy effort to play 6 screens of one film versus another player going with 6 of another one.

    I would argue that without the bonuses, players would have to consider all of the films each week as earners – and how to fit them together against the $1,000 cap. We would see much more diversity of lineups with a closer margin of competition.

    I really appreciate your response and extra effort.
    I like this game, and I am here every week running the numbers (unfortunately I started late to be a contender).
    But I know that with a more level playing field, it would be even better.

    Sincerely,

    Sad Robot

    Like

  2. Hi Robert aka Sad Robot,

    Thanks so much for the very thorough and thoughtful reply. You are very welcome for the extra work and I’m glad it revealed more to you along the line of thinking you’ve started.

    You are right, I am optimistic about the bonus system but I have to admit your contrarian view has me thinking. I have zero influence over the game mechanics but I’ll definitely be looking at this more closely now.

    —Pete

    Like

  3. Glacier’s Raw Score during Week 8 was actually $138,043,858, and the $4 million in bonuses brought him up to the figure you have as his Raw Score. Also, he had a Perfect Cineplex that week, so his Fall Bonuses number should be $16 million, not $6 million.

    Like

      1. I just saw something else as well. Sleestak’s Raw Score for Week 12 should be the same as Glacier’s.

        Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s