Outline for today
Stat155
Game Theory
Lecture 6: Solving two player zero-sum games
Zero sum games
Recall: payoff matrices, mixed strategies, safety strategies, Von
Neumann’s minimax theorem
Solving two player zero-sum games
Saddle points
Dominated pure strategies
Solving 2 × 2 games
Equalizing strategies
Peter Bartlett
September 13, 2016
1 / 25
2 / 25
Recall: Two-player zero-sum games
Recall: Two-player zero-sum games
Definitions
Definitions
Player I has m actions, 1, 2, . . . , m.
A mixed strategy is a probability distribution over actions.
Player II has n actions, 1, 2, . . . , n.
A mixed strategy for Player I is a vector
 
x1
(
)
m
 x2 
X
 
xi = 1 .
x =  .  ∈ ∆m := x ∈ Rm : xi ≥ 0,
 .. 
i=1
xm
The payoff matrix A ∈
Rm×n
represents the payoff to Player I:


a11 a12 · · · a1n
 a21 a22 · · · a2n 


A= .
.. 
.
.
.
 .
.
. 
am1 am2 · · ·
amn
If Player I chooses i and Player II chooses j, the payoff to Player I is
aij and the payoff to Player II is −aij .
The sum of the payoff to Player I and the payoff to Player II is 0.
3 / 25
A mixed strategy for Player II is a vector y ∈ ∆n .
A pure strategy is a mixed strategy where one entry is 1 and the
others 0. (This is a canonical basis vector ei .)
4 / 25
Recall: Two-player zero-sum games
Recall: Two-player zero-sum games
A safety strategy for Player I is an x ∗ ∈ ∆m that satisfies
min x ∗ > Ay = max min x > Ay .
The expected payoff to Player I when Player I plays mixed strategy
x ∈ ∆m and Player II plays mixed strategy y ∈ ∆n is
EI ∼x EJ∼y aIJ =
m X
n
X
y ∈∆n
x∈∆m y ∈∆n
This mixed strategy maximizes the worst case expected gain for
Player I.
xi aij yj
i=1 j=1
Similarly, a safety strategy for Player II is a y ∗ ∈ ∆n that satisfies
>
= x Ay .
max x > Ay ∗ = min max x > Ay .
x∈∆m
y ∈∆n x∈∆m
This mixed strategy minimizes the worst case expected loss for
Player II.
5 / 25
Recall: Two-player zero-sum games
6 / 25
Outline
Von Neumann’s Minimax Theorem
For any two-person zero-sum game with payoff matrix A ∈ Rm×n ,
max min x > Ay = min max x > Ay .
x∈∆m y ∈∆n
Zero sum games
y ∈∆n x∈∆m
Recall: payoff matrices, mixed strategies, safety strategies, Von
Neumann’s minimax theorem
Solving two player zero-sum games
We call the optimal expected payoff the value of the game,
Saddle points
Dominated pure strategies
Solving 2 × 2 games
Equalizing strategies
V := max min x > Ay = min max x > Ay .
x∈∆m y ∈∆n
y ∈∆n x∈∆m
LHS: Player I plays a safety strategy.
RHS: Player II plays a safety strategy.
Safety strategies are optimal strategies.
7 / 25
8 / 25
Saddle points
Saddle points
Example
Definition
A pair (i ∗ , j ∗ ) ∈ {1, . . . , m} × {1, . . . , n} is a saddle point for a payoff
matrix A ∈ Rm×n if


−1 1 5
A =  5 3 4
6 2 1
max aij ∗ = ai ∗ j ∗ = min ai ∗ j .
i
Suppose Player I plays 2 and Player II plays 2.
j
If Player I plays i ∗ and Player II plays j ∗ , neither player has an
incentive to change.
The payoff is a22 = 3.
Should either player change their strategy?
This pair of pure strategies, (e2 , e2 ), is called a saddle point (or pure
Nash equilibrium).
Think of these as locally optimal strategies for the players.
We’ll see that they are also globally optimal strategies.
9 / 25
Saddle points: Example
10 / 25
Saddle points
1
2
3
4
5
6
7
1
-1
5
6
3
1
4
3
2
1
3
2
1
2
8
1
3
2
-4
-5
3
3
3
3
4
4
7
2
-5
5
7
-5
5
8
3
8
3
4
5
3
6
-2
1
-1
4
-6
8
4
7
7
3
2
7
7
4
7
column max
6
8
3
7
8
8
7
row min
-1
1
1
1
1
3
1
Theorem
If (i ∗ , j ∗ ) is a saddle point for a payoff matrix A ∈ Rm×n , then
ei ∗ is an optimal strategy for Player I,
ej ∗ is an optimal strategy for Player II, and
the value of the game is ai ∗ j ∗ .
Is there a saddle point?
Suppose Player I plays 6 and Player II plays 3.
Neither player should change their strategy: (e6 , e3 ), is a saddle point.
Saddle point is a row/column pair where the row min is equal to the
column max. (Need max of row mins = min of column maxes.)
11 / 25
12 / 25
Saddle points
Saddle points
Theorem
Saddle point (i ∗ , j ∗ ) (maxi aij ∗ = ai ∗ j ∗ = minj ai ∗ j ) implies ei ∗ , ej ∗ optimal.
Proof
We have seen that we should always prefer to play last but with a
saddle point, the opposite inequality is also true:
min max x > Ay ≥ max min x > Ay
y ∈∆n x∈∆m
So if we find a saddle point, the game is easy to solve.
Another way to simplify a two-player zero-sum game is by removing
dominated rows or columns.
x∈∆m y ∈∆n
≥ min ei>∗ Ay
(∗)
= ei>∗ Aej ∗ = max x > Aej ∗
(∗)
y ∈∆n
x∈∆m
>
≥ min max x Ay .
y ∈∆n x∈∆m
The inequalities are all equalities: min max x > Ay = max min x > Ay .
y
x
x
y
This proves von Neumann’s minimax theorem for this case.
And (∗) shows ei ∗ , ej ∗ optimal.
13 / 25
Outline
14 / 25
Removing dominated pure strategies
Example: Plus One (a Mendelsohn game)
Each player picks a number in {1, 2, . . . , n}.
If i = j, payoff is $0.
Zero sum games
If |i − j| = 1, the higher number wins $1.
Recall: payoff matrices, mixed strategies, safety strategies, Von
Neumann’s minimax theorem
If |i − j| ≥ 2, the higher number loses $2.
Solving two player zero-sum games
Saddle points
Dominated pure strategies
Solving 2 × 2 games
Equalizing strategies
15 / 25
16 / 25
Removing dominated pure strategies
Removing dominated pure strategies
Example: Plus One
Example: Miss-by-one (a Mendelsohn game
Players I and II choose numbers i, j ∈ {1, 2, . . . , 7}.
Player I wins 1 if |i − j| = 1, otherwise the payoff is 0.
Try it!
What is the payoff matrix?
What do you notice about row 5 and row 7?
For Player I, row 5 is always at least as good as row 7.
Player I never needs to play 7:
There is an optimal strategy x ∗ for Player I with x7∗ = 0.
Similarly for columns 5 and 7:
Player II never needs to play 5 (can choose y7∗ = 0).
Reduced payoff matrix
Similarly for row 1 and row 3, and for column 1 and column 3.
Without row 1, columns 3 and 5, we can omit row 4 and column 4.
There is an optimal strategy for each player mixing only 4 numbers.
17 / 25
18 / 25
2 × 2 games
Outline
Finding Player I’s
optimal strategy
Zero sum games
Recall: payoff matrices, mixed strategies, safety strategies, Von
Neumann’s minimax theorem
Finding Player II’s
optimal strategy
Recall: Pick-a-hand
Solving two player zero-sum games
Saddle points
Dominated pure strategies
Solving 2 × 2 games
Equalizing strategies
(Karlin and Peres, 2016)
19 / 25
(Karlin and Peres, 2016)
20 / 25
2 × 2 games
2 × 2 games
L
c
a
T
B
Consider the last two cases.
The optimal strategy for Player I is at the intersection of the lines.
This is an equalizing strategy:
Whatever Player II plays, the expected payoff is the same.
Writing x1 = Pr(T ), the equations for the lines are:
R
d
b
Finding Player I’s optimal strategy
a>b>c>d
a>b>d>c
a
a>c>b>d
a
a
L
d
T
B
T
c
R
b
T
R
cb
B
V = a + x1 (c − a).
d
d
R
db
B
L
d
L
c
R
d
c
R
V = b + x1 (d − b),
a
L
c
b
R
a>d>c>b
a
L
cb
B
a>d>b>c
a
L
b
a>c>d>b
T
B
T
Equating gives
B
T
x1 =
These represent all games (swap L ↔ R, T ↔ B, or both).
Saddle points? First four games.
What happens if c → d?
Then x1 → 1; the game approaches the fourth case.
Dominated rows? First, second, third.
Dominated columns? First, third, fourth.
a−b
.
a−b+d −c
21 / 25
2 × 2 games
22 / 25
Solving 2 × 2 games
How to solve a 2 × 2 game
1
2
Check for a saddle point.
(Is the max of row mins = min of column maxes?)
Examples
If there are no saddle points, find equalizing strategies.
Equalizing strategies satisfy (in standard notation):
x1 a11 + (1 − x1 )a21 = x1 a12 + (1 − x1 )a22 ,
y1 a11 + (1 − y1 )a12 = y1 a21 + (1 − y1 )a22 .
Solving gives
0
1
3
0
3
0
2
3
2
1
1
2
a21 − a22
,
a21 − a22 + a12 − a11
a12 − a22
y1 =
.
a12 − a22 + a21 − a11
x1 =
23 / 25
24 / 25
Outline
Zero sum games
Recall: payoff matrices, mixed strategies, safety strategies, Von
Neumann’s minimax theorem
Solving two player zero-sum games
Saddle points
Dominated pure strategies
Solving 2 × 2 games
Equalizing strategies
25 / 25