Download Algorithms for Determining Tournament Payout Structures

Algorithms for Determining Tournament
Payout Structures
Christopher Musco, Maxim Sviridenko, and Justin Thaler
January 18, 2017
Massachusetts Institute of Technology, Yahoo Research, Georgetown University
1
Algorithms for game and contest management
There are a lot of interesting algorithmic challenges involved
in managing massive online games and contests.
2
Algorithms for game and contest management
There are a lot of interesting algorithmic challenges involved
in managing massive online games and contests.
• online poker tournaments
2
Algorithms for game and contest management
There are a lot of interesting algorithmic challenges involved
in managing massive online games and contests.
• online poker tournaments
• video game (eSports) tournaments
2
Algorithms for game and contest management
There are a lot of interesting algorithmic challenges involved
in managing massive online games and contests.
• online poker tournaments
• video game (eSports) tournaments
• fantasy sports contests
2
Algorithms for game and contest management
There are a lot of interesting algorithmic challenges involved
in managing massive online games and contests.
• online poker tournaments
• video game (eSports) tournaments
• fantasy sports contests
100,000s of players, complex tournament structures, real
money on the line.
2
Fantasy Sports: A Quick Review
What are fantasy sports?
3
Fantasy Sports: A Quick Review
Users “draft” a group of real-world athletes and earn points
depending on how well those players perform in games.
4
Fantasy Sports: A Quick Review
Users “draft” a group of real-world athletes and earn points
depending on how well those players perform in games.
4
The Business of Fantasy Sports
Sports covered: American football, baseball, soccer,
basketball, hockey, golf, auto racing, mixed martial arts …
5
The Business of Fantasy Sports
Sports covered: American football, baseball, soccer,
basketball, hockey, golf, auto racing, mixed martial arts …
• 57.4 million users in the US and Canada alone
+ huge international growth
5
The Business of Fantasy Sports
Sports covered: American football, baseball, soccer,
basketball, hockey, golf, auto racing, mixed martial arts …
• 57.4 million users in the US and Canada alone
+ huge international growth
• large platforms run by ESPN, NFL, Yahoo, CBS etc.
5
The Business of Fantasy Sports
Sports covered: American football, baseball, soccer,
basketball, hockey, golf, auto racing, mixed martial arts …
• 57.4 million users in the US and Canada alone
+ huge international growth
• large platforms run by ESPN, NFL, Yahoo, CBS etc.
• > 60% of participants report watching more games and
reading more about sports
5
The Business of Fantasy Sports
And now you can legally gamble on fantasy sports in the US.
6
The Business of Fantasy Sports
And now you can legally gamble on fantasy sports in the US.
Led to emergence of Daily Fantasy Sports.
6
The Business of Fantasy Sports
And now you can legally gamble on fantasy sports in the US.
Led to emergence of Daily Fantasy Sports.
(Over $1.7 billon dollars in seed funding for DraftKings
and FanDuel alone)
6
The Business of Fantasy Sports
And now you can legally gamble on fantasy sports in the US.
Led to emergence of Daily Fantasy Sports.
(Over $1.7 billon dollars in seed funding for DraftKings
and FanDuel alone)
Running contests with 10,000s - 100,000s of players.
6
Daily Fantasy Sports: Computational Challenges
7
Daily Fantasy Sports: Computational Challenges
• How to evaluate and price athletes? (Anagnostopoulos,
Cavallo, Leonardi, Sviridenko, WINE 2016)
7
Daily Fantasy Sports: Computational Challenges
• How to evaluate and price athletes? (Anagnostopoulos,
Cavallo, Leonardi, Sviridenko, WINE 2016)
• How to structure scoring to manage competition variance?
7
Daily Fantasy Sports: Computational Challenges
• How to evaluate and price athletes? (Anagnostopoulos,
Cavallo, Leonardi, Sviridenko, WINE 2016)
• How to structure scoring to manage competition variance?
• How to ensure fairness and effectively separate new
players from “sharks”? (see NY Times article)
7
Daily Fantasy Sports: Computational Challenges
• How to evaluate and price athletes? (Anagnostopoulos,
Cavallo, Leonardi, Sviridenko, WINE 2016)
• How to structure scoring to manage competition variance?
• How to ensure fairness and effectively separate new
players from “sharks”? (see NY Times article)
How to distribute prize money amongst top contestants?
7
Tournament payout structures
100,000 players → $1,000,000 in prizes → 10,000 prize winners
8
Tournament payout structures
Payouts should:
9
Tournament payout structures
Payouts should:
1. Strongly incentivize players to enter contests.
9
Tournament payout structures
Payouts should:
1. Strongly incentivize players to enter contests.
payout for i th place
10 5
10 4
10 3
0
5
10
15
20
25
30
35
40
winning position
9
Tournament payout structures
Payouts should:
1. Strongly incentivize players to enter contests.
2. Obey basic aesthetic properties.
payout for i th place
10 5
10 4
10 3
0
5
10
15
20
25
30
35
40
winning position
9
Payout structure aesthetic requirements
2. Prizes obey basic aesthetic properties.
10
Payout structure aesthetic requirements
2. Prizes obey basic aesthetic properties.
• Are “nice numbers” ($1000 is preferable to $1012.15)
10
Payout structure aesthetic requirements
2. Prizes obey basic aesthetic properties.
• Are “nice numbers” ($1000 is preferable to $1012.15)
{10, 15, …, 95,100,125, 150, …, 225, 250, 300, 350, …, 950, 1000}
10
Payout structure aesthetic requirements
2. Prizes obey basic aesthetic properties.
• Are “nice numbers” ($1000 is preferable to $1012.15)
{10, 15, …, 95,100,125, 150, …, 225, 250, 300, 350, …, 950, 1000}
• Fall into manageable number of buckets (i.e. 25 – 40)
10
Payout structure aesthetic requirements
2. Prizes obey basic aesthetic properties.
• Are “nice numbers” ($1000 is preferable to $1012.15)
{10, 15, …, 95,100,125, 150, …, 225, 250, 300, 350, …, 950, 1000}
• Fall into manageable number of buckets (i.e. 25 – 40)
• Ideally buckets increase in size for lower places
10
Payout structure aesthetic requirements
2. Prizes obey basic aesthetic properties.
• Are “nice numbers” ($1000 is preferable to $1012.15)
{10, 15, …, 95,100,125, 150, …, 225, 250, 300, 350, …, 950, 1000}
• Fall into manageable number of buckets (i.e. 25 – 40)
• Ideally buckets increase in size for lower places
10 5
$51,400
$15,200
10
10
$11,400
$10,200
$8900
$7600
$6400
$5100
$3800
$3500
$3200
$3000
4
$2500
$2200
$1900
$1600
3
0
5
10
15
20
25
30
(Bassmaster fishing tournament)
35
10
Payout structure aesthetic requirements
2. Prizes obey basic aesthetic properties.
• Are “nice numbers” ($1000 is preferable to $1012.15)
{10, 15, …, 95,100,125, 150, …, 225, 250, 300, 350, …, 950, 1000}
• Fall into manageable number of buckets (i.e. 25 – 40)
• Ideally buckets increase in size for lower places
10 5
$50,000
$20,000
$10,000
10 4
$8500
$6000
$4000
$3500
$3000
$2700
$2000
10 3
0
5
10
15
20
25
30
(Bassmaster fishing tournament)
35
10
Payout structure aesthetic requirements
2. Prizes obey basic aesthetic properties.
• Are “nice numbers” ($1000 is preferable to $1012.15)
{10, 15, …, 95,100,125, 150, …, 225, 250, 300, 350, …, 950, 1000}
• Fall into manageable number of buckets (i.e. 25 – 40)
• Ideally buckets increase in size for lower places
Prizes need to sum to the total allocated prize pool.
In Daily Fantasy Sports and other large tournaments this is
often a strict requirement.
10
Manual Solution?
How hard is it to construct payout structures by hand?
11
Manual Solution?
How hard is it to construct payout structures by hand?
Very difficult! Even for just a single contest.
11
Manual Solution?
How hard is it to construct payout structures by hand?
Very difficult! Even for just a single contest.
World Series of Poker organizers apparently struggled with the
problem for years before commissioning their own algorithm.
11
Manual Solution?
How hard is it to construct payout structures by hand?
Very difficult! Even for just a single contest.
World Series of Poker organizers apparently struggled with the
problem for years before commissioning their own algorithm.
Daily Fantasy sites run 100s of contests a week, with widely
varying entry numbers and prize pools.
11
Two Step Approach
Payouts should:
1. Strongly incentivize players to enter contests.
2. Obey basic aesthetic properties.
12
Two Step Approach
Payouts should:
1. Strongly incentivize players to enter contests.
2. Obey basic aesthetic properties.
Two step approach:
1. Choose “ideal payouts” that don’t satisfy aesthetics.
2. Round to a payout structure that does.
12
prize for i
th
place
Two Step Approach
10 4
5
10
15
20
25
30
35
40
winner rank
Ideal vs. Rounded payoff structure.
13
prize for i
th
place
Two Step Approach
10 4
5
10
15
20
25
30
35
40
winner rank
Ideal vs. Rounded payoff structure.
13
Two Step Approach
Payouts should:
1. Strongly incentivize players to enter contests.
2. Obey basic aesthetic properties.
Two step approach:
1. Choose “ideal payouts” that don’t satisfy aesthetics.
2. Round to a payout structure that does.
14
Two Step Approach
Payouts should:
1. Strongly incentivize players to enter contests.
2. Obey basic aesthetic properties.
Two step approach:
1. Choose “ideal payouts” that don’t satisfy aesthetics.
2. Round to a payout structure that does.
14
Step 1: Ideal Payouts
Fix top prize, minimum prize, and number of prize winners.
15
Step 1: Ideal Payouts
Fix top prize, minimum prize, and number of prize winners.
• Top prize = “marketing number” (i.e. $100,000 grand prize)
or around 10% - 15% of prize pool
15
Step 1: Ideal Payouts
Fix top prize, minimum prize, and number of prize winners.
• Top prize = “marketing number” (i.e. $100,000 grand prize)
or around 10% - 15% of prize pool
• Minimum prize = 2x entry fee
15
Step 1: Ideal Payouts
Fix top prize, minimum prize, and number of prize winners.
• Top prize = “marketing number” (i.e. $100,000 grand prize)
or around 10% - 15% of prize pool
• Minimum prize = 2x entry fee
• Number of winners = fixed percentage of entries (i.e. 25%)
15
Step 1: Ideal Payouts
Fix top prize, minimum prize, and number of prize winners.
• Top prize = “marketing number” (i.e. $100,000 grand prize)
or around 10% - 15% of prize pool
• Minimum prize = 2x entry fee
• Number of winners = fixed percentage of entries (i.e. 25%)
Intermediate prizes defined by simple fall-off function.
15
Power law Payoffs
We use a power law fall-off:
ith prize proportional to 1/iα , for constant α.
16
Power law Payoffs
We use a power law fall-off:
ith prize proportional to 1/iα , for constant α.
10 6
payout i
10
5
10 4
10 3
10 2
10 1
0
2000
4000
6000
8000
10000
winneri
16
Power law Payoffs
We use a power law fall-off:
ith prize proportional to 1/iα , for constant α.
10 6
payout i
10
5
10 4
10 3
10 2
10 1
0
2000
4000
6000
8000
10000
winneri
Solve for α such that:
∑total winners (
minimum prize +
i=1
total prize pool.
top prize - minimum prize
Iα
)
=
16
Why power law?
A power law richly rewards the best players, but ensures lower
winners still receive substantial prizes.
17
Why power law?
10 6
10 6
10 5
10 5
payout i
payout i
A power law richly rewards the best players, but ensures lower
winners still receive substantial prizes.
10 4
10 3
10 2
10 1
10 4
10 3
10 2
0
2000
4000
6000
8000
10000
winneri
1/iα power law fall-off.
10 1
0
2000
4000
6000
8000
10000
winneri
1/αi exponential fall-off.
17
Why power law?
18
Why power law?
Payout distributions for Daily Fantasy Sports and other large
tournaments consistent with a power law fall-off.
7
7
10
10
FanDuel NFL Kickoff Tournament
FanDuel MLB Monster Tuesday
DraftKings PGA Millionaire Maker
DraftKings NFL Kickoff Tournament
6
10
World Series of Poker 2015
Masters 2015 (golf)
Bassmaster 2015 (fishing)
5
6
i
10
payout
payouti
10
4
10
3
5
10
10
2
10
1
10 0
10
4
1
10
2
10
3
10
winneri
4
10
5
10
6
10
10 0
10
1
2
10
10
3
10
winneri
19
Two Step Approach
Payouts should:
1. Strongly incentivize players to enter contests.
2. Obey basic aesthetic properties.
Two step approach:
1. Choose “ideal payouts” that don’t satisfy aesthetics.
2. Round to a payout structure that does.
20
Two Step Approach
Payouts should:
1. Strongly incentivize players to enter contests.
2. Obey basic aesthetic properties.
Two step approach:
1. Choose “ideal payouts” that don’t satisfy aesthetics.
2. Round to a payout structure that does.
20
Two Step Approach
Payouts should:
1. Strongly incentivize players to enter contests.
2. Obey basic aesthetic properties.
Two step approach:
1. Choose “ideal payouts” that don’t satisfy aesthetics.
2. Round to a payout structure that does.
20
Rounding Payouts
Optimization Problem
Input:
Ideal payouts, {π1 , . . . , πn }.
Output:
Non-overlapping ranges of ranks, {S1 , . . . , Sk }.
Prizes {P1 , . . . , Pk }
21
Rounding Payouts
Optimization Problem
Input:
Ideal payouts, {π1 , . . . , πn }.
Output:
Non-overlapping ranges of ranks, {S1 , . . . , Sk }.
Prizes {P1 , . . . , Pk }
e.g
Input:
{4610, 4138, 3792, 3531, 3327, 3165, 3034, 2925, 2834}.
Output:
{{1}, {2}, {3}, {4 − 5}{6 − 9}}
{5000, 4500, 4000, 3500, 3000}
21
Rounding Payouts
Optimization Problem
Input:
Ideal payouts, {π1 , . . . , πn }.
Output:
Non-overlapping ranges of ranks, {S1 , . . . , Sk }.
Prizes {P1 , . . . , Pk }
Objective:
∑ ∑
minimize ki=1 j∈Si (πj − Pj )2
21
Rounding Constraints
minimize
∑k ∑
i=1
j∈Si (πj
− Pj )2
22
Rounding Constraints
minimize
∑k ∑
i=1
j∈Si (πj
− Pj )2
Such that:
• Pi is a nice number
22
Rounding Constraints
minimize
∑k ∑
i=1
j∈Si (πj
− Pj )2
Such that:
• Pi ∈ {100, 200, 300, 400, 500, 1000, . . . , 10000, 15000, . . .}
22
Rounding Constraints
minimize
∑k ∑
i=1
j∈Si (πj
− Pj )2
Such that:
• Pi ∈ {100, 200, 300, 400, 500, 1000, . . . , 10000, 15000, . . .}
• P1 > P2 > . . . > Pk ≥ minimum prize
22
Rounding Constraints
minimize
∑k ∑
i=1
j∈Si (πj
− Pj )2
Such that:
• Pi ∈ {100, 200, 300, 400, 500, 1000, . . . , 10000, 15000, . . .}
• P1 > P2 > . . . > Pk ≥ minimum prize
∑k
•
I=1 |Si | = n
22
Rounding Constraints
minimize
∑k ∑
i=1
j∈Si (πj
− Pj )2
Such that:
• Pi ∈ {100, 200, 300, 400, 500, 1000, . . . , 10000, 15000, . . .}
• P1 > P2 > . . . > Pk ≥ minimum prize
∑k
•
I=1 |Si | = n
∑k
•
I=1 |Si | · Pi = B (total prize pool)
22
Rounding Constraints
minimize
∑k ∑
i=1
j∈Si (πj
− Pj )2
Such that:
• Pi ∈ {100, 200, 300, 400, 500, 1000, . . . , 10000, 15000, . . .}
• P1 > P2 > . . . > Pk ≥ minimum prize
∑k
•
I=1 |Si | = n
∑k
•
I=1 |Si | · Pi = B (total prize pool)
• |S1 | ≤ |S2 | ≤ . . . ≤ |Sk |
22
Rounding Algorithms
How can we solve this optimization problem?
23
Rounding Algorithms
How can we solve this optimization problem?
Option 1: Multi-dimensional dynamic programming
23
Rounding Algorithms
How can we solve this optimization problem?
Option 1: Multi-dimensional dynamic programming
O(kn2 B log B) time if there are O(log B) “nice numbers” ≤ B.
23
Rounding Algorithms
How can we solve this optimization problem?
Option 1: Multi-dimensional dynamic programming
O(kn2 B log B) time if there are O(log B) “nice numbers” ≤ B.
Option 2: Integer Program
23
Rounding Algorithms
How can we solve this optimization problem?
Option 1: Multi-dimensional dynamic programming
O(kn2 B log B) time if there are O(log B) “nice numbers” ≤ B.
Option 2: Integer Program
Off-the-shelf solver (GLPK) works well for relatively small
contests
23
Rounding Algorithms
How can we solve this optimization problem?
Option 1: Multi-dimensional dynamic programming
O(kn2 B log B) time if there are O(log B) “nice numbers” ≤ B.
Option 2: Integer Program
Off-the-shelf solver (GLPK) works well for relatively small
contests
Option 3: Engineered Heuristic
23
Rounding Algorithms
How can we solve this optimization problem?
Option 1: Multi-dimensional dynamic programming
O(kn2 B log B) time if there are O(log B) “nice numbers” ≤ B.
Option 2: Integer Program
Off-the-shelf solver (GLPK) works well for relatively small
contests
Option 3: Engineered Heuristic
Matches quality of exactly optimal solutions, scales to very
large contests.
23
Heuristic Algorithm
Outline of heuristic algorithm:
24
Heuristic Algorithm
Outline of heuristic algorithm:
1. Set initial bucket sizes to increase according to a power
law, distributing n places amongst k buckets.
24
Heuristic Algorithm
Outline of heuristic algorithm:
1. Set initial bucket sizes to increase according to a power
law, distributing n places amongst k buckets.
2. Choose initial prize Pi to be the largest nice number
smaller than mean of ideal prizes in bucket Si .
24
Heuristic Algorithm
Outline of heuristic algorithm:
1. Set initial bucket sizes to increase according to a power
law, distributing n places amongst k buckets.
2. Choose initial prize Pi to be the largest nice number
smaller than mean of ideal prizes in bucket Si .
3. Merge any buckets with shared prizes and use local swaps
to keep bucket sizes monotonic.
24
Heuristic Algorithm
Outline of heuristic algorithm:
1. Set initial bucket sizes to increase according to a power
law, distributing n places amongst k buckets.
2. Choose initial prize Pi to be the largest nice number
smaller than mean of ideal prizes in bucket Si .
3. Merge any buckets with shared prizes and use local swaps
to keep bucket sizes monotonic.
4. Spend left-over budget on “singleton buckets”, by violating
nice number constraint in a bucket, and as a last resort
adding extra winners.
24
Runtime Results
Integer Program: Only scales to contests with < 100 winners.
Heuristic Algorithm: < 2 second runtimes on a laptop for
contests with millions of dollars in prizes, > 10,000 winners.
25
Runtime Results
Integer Program: Only scales to contests with < 100 winners.
Heuristic Algorithm: < 2 second runtimes on a laptop for
contests with millions of dollars in prizes, > 10,000 winners.
(Deployed in production at Yahoo.)
25
Quantitative Performance
26
Quantitative Performance
ℓ2 distance to ideal payouts within 2x-5x that of IP.
26
Qualitative Performance
5
10 4
FanDuel
Heuristic
payout
i
4
3
2
1
0
10 0
10 1
10 2
winner i
10 3
10 4
FanDuel fantasy football contest
27
Qualitative Performance
DraftKings fantasy football contest
27
Qualitative Performance
Easily patches “bad” payout structures!
10 5
$51,400
$15,200
$11,400
$10,200
$8900
$7600
$6400
$5100
$3800
$3500
$3200
$3000
10 4
10
$2500
$2200
$1900
$1600
3
0
5
10
15
20
25
30
35
(Bassmaster fishing tournament)
28
Qualitative Performance
Easily patches “bad” payout structures!
10 5
$50,000
$20,000
$10,000
10 4
$8500
$6000
$4000
$3500
$3000
$2700
$2000
10 3
0
5
10
15
20
25
30
35
(Bassmaster fishing tournament)
28
World Series of Poker Payouts
2015 WSOP
Payouts
Prize
$7,680,021
$4,469,171
$3,397,103
$2,614,558
$1,910,971
$1,426,072
$1,203,193
$1,097,009
$1,001,020
$756,897
$526,778
$411,453
$325,034
Place
1
2
3
4
5
6
7
8
9
10
11 - 12
13 - 15
16 - 18
19 - 27
$262,574
28 - 36
36 - 45
46 - 54
55 - 63
64 - 72
73 - 81
82 - 90
91 - 99
$211,821
$164,086
$137,300
$113,764
$96,445
$79,668
$68,624
$55,649
100 - 162
$46,890
163 - 225
226 - 288
289 - 351
352 - 414
415 - 477
478 - 549
550 - 648
649 - 1000
$40,433
$34,157
$29,329
$24,622
$21,786
$19,500
$17,282
$15,000
Our Alternative
Payouts
Prize
$8,000,000
$4,000,000
$2,250,000
$1,750,000
$1,250,000
$1,000,000
$950,000
$850,000
$700,000
Place
1
2
3
4
5
6
7
8
9
10 - 13
$650,000
14 - 17
$500,000
18 - 23
24 - 29
30 - 35
36 - 42
43 - 59
$300,000
$225,000
$200,000
$150,000
$125,000
$95,000
60 - 77
78 - 99
$75,000
100 - 128
128 - 164
165 - 254
$60,000
$55,000
$45,000
255 - 345
$35,000
346 - 441
$25,000
442 - 710
$22,500
711 - 1000
$20,150
29
Conclusion
30
Conclusion
• Lots of interesting algorithmic problems involved in
managing massive online tournaments.
30
Conclusion
• Lots of interesting algorithmic problems involved in
managing massive online tournaments.
• Theoretical formulation leads to provably algorithms as
well as practical heuristics.
Thanks!
30