Making Decisions - A Game Theoretic approach

Dr Ganesh Neelakanta Iyer
Industry expert, Researcher, Professor
ganesh@ganeshniyer.com
http://ganeshniyer.com
Making decisions –
A Game-theoretic Approach

Notes
• This is going to be a high-level introduction to Game
Theory and some examples from Engineering/Business
• To avail e-certificates of attendance (even otherwise for
us to know who all attended this session), please fill the
feedback form towards the end of this session
• If you are not a MATH guy, enjoy the pictures I have taken
during my many world-trips and Dilbert comics 
7/18/2020 Dr Ganesh Neelakanta Iyer 2

About Me • Site Reilability Engineer working for Salesforce
• Masters & PhD from National University of Singapore (NUS)
• Several years in Industry/Academia
• Architect, Manager, Technology Evangelist, Professor
• Talks/workshops in USA, Europe, Australia, Asia
• Cloud/Edge Computing, IoT, Software Engineering, Game
Theory, Machine Learning
• Kathakali Artist, Composer, Speaker, Traveler, Photographer
GANESHNIYER http://ganeshniyer.com

4
Outline for today (Not in slide’s order)
• Overview of Game Theory
• Non – Cooperative Games
– Simultaneous Games
– Sequential Games
– Evolutionary Games
• Cooperative Games
– Bargaining Games
– Coalition Games
• Mechanism Design
– Auctions
7/18/2020 Dr Ganesh Neelakanta Iyer

Some application domains we will touch upon
Dr Ganesh Neelakanta Iyer 5

6
Overview of Game Theory
6
Raffles Place, Singapore

What is Game Theory About?
• Analysis of situations where conflict of interests are present
Goal is to prescribe how conflicts can be resolved
2
2
Game of Chicken
driver who steers away looses
What should drivers do?

Game Theory
• Study of how people interact and make decisions
• Outcome of a person’s decision depends not just on how they
choose among several options, but also on the choices made by the
people they are interacting with

TCP Back off Game
TCP Congestion Control - AIMD
 Algorithm AIMD
 Additive Increase Multiplicative Decrease
 Increment Congestion Window by one packet per RTT
 Linear increase
 Divide Congestion Window by two whenever a
timeout occurs
 Multiplicative decrease
Source Destination
…
60
20
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0
KB
T ime (seconds)
70
30
40
50
10
10.0

TCP Backoff Game
• Should you send your packets using
– Correctly-implemented TCP ( which has a “backoff” mechanism) or
– Using a defective implementation (which doesn’t)?
• This problem is an example of what we call a two-player
game:
– Both use a correct implementation: both get 1 ms delay
– One correct, one defective: 4 ms for correct, 0 ms for defective
– Both defective: both get a 3 ms delay.

Self Interested Agents
• What does it mean to say that an agent is self-interested?
– Not that they want to harm others or only care about themselves
• Only that the agent has its own
description of states of the world that
it likes, and acts based on this
description

Self Interested Agents
• Each such agent has a
utility function
– quantifies degree of
preference across alternatives
– explains the impact of
uncertainty
– Decision-theoretic rationality:
act to maximize expected
utility

What is a game?
Players: who are the decision makers?
• People? Governments? Companies?
• Somebody employed by a Company?...
Actions: What can the players do?
• Enter a bid in an auction?
• Decide whether to end a strike?
• Decide when to sell a stock?
Strategies: Which action did I choose
• actions which a player chooses to follow
• I will sell the stock today, I will vote for XYZ Party
Payoffs: what motivates the players?
• Do they care about some profit?
• Do they care about other players?...
Outcome: What is the result?
• Determined by mutual choice of strategies

Different types of
Games

Types of Games
Games
Simultaneous
Games
Sequential
Games

Types of Games
Games
Symmetric
Games
Asymmetric
games

Types of Games
Games
Perfect
Information
Imperfect
Information

Types of Games
Games
Cooperative
Non-
cooperative

Types of Games
Games
Zero sum Non zero sum

Types of Games
Games
Single Shot
game
Repeated
Game
mutually assured nuclear destruction

Different Types – by Concept
Cooperative
• Bargaining
• Coalitions
• Stable matching
Non-Cooperative
• Zero-sum
• Non-Zero sum
• Bayesian
• Evolutionary
• Congestion/Potential
• Matrix
• Stackelberg
• Jamming
• Markovian
• Combinatorial
• Differential
Mechanism Design
• Auctions
• VCG Mechanisms
• Social Choice
• Optimal Mechanisms
And a lot
more…
7/18/2020

Game Theory:
Applications
• Economics: Oligopoly markets, Mergers and
acquisitions pricing, auctions
• Political Science: fair division, public choice,
political economy
• Biology: modeling competition between
tumor and normal cells, Foraging bees
• Sports coaching staffs: run vs pass or pitch
fast balls vs sliders
• Engineering: Wireless Networks, Distributed
systems, Computer Networks, AI
http://customergauge.com/wordpress/wp-content/uploads/2008/10/power_retailers_oligopoly.jpg
http://cricketradius.com/wp-content/uploads/2011/11/fast-bowling.jpg

Game Theory:
Engineering Applications
• Electronics and Communication
– Power Allocation
– Cognitive Radio Networks
– Wireless Networks
• Computer Science
– Distributed systems
– Computer Networks
– Artificial Intelligence
– Multi-agent systems
• Electrical Engineering
– Smart Grid
– Voltage Regulation
• Civil
– Construction Engineering
http://www.cee.ntu.edu.sg/Programmes/graduate/MSc_CE/Pages/Overview.aspx
https://www.edie.net/news/6/Britain-gears-up-for-smart-energy-transition-with-new-consultation/
http://ncel.ie.cuhk.edu.hk/content/qos-support-cognitive-radio-networks

Types of non-cooperative Games
Non
Cooperative
Games
Static
(Simultaneous)
Dynamic
(Sequential)

Defining Games:
Two standard representations
• Normal Form (a.k.a. Matrix Form, Strategic Form) List what
payoffs get as a function of their actions
– It is as if players moved simultaneously
– But strategies encode many things...
• Extensive Form Includes timing of moves (later in course)
– Players move sequentially, represented as a tree
• Chess: white player moves, then black player can see white’s move and react…
– Keeps track of what each player knows when he or she makes each
decision
• Poker: bet sequentially – what can a given player see when they bet?

Defining Games:
The Normal Form
• Finite, n-person normal form game: ⟨N, A, u⟩:
– Players: N = {1, … , n} is a finite set of n, indexed by i
– Action set for player i, Ai :
• a = (a1, … ,an) ∈ A = A1 X … X An is an action profile
– Utility function or Payoff function for player i: ui : A→ R
• u = (u11, …, un) , is a profile of utility functions

Example:
Coin matching game
• Roger and Colleen play a game; Each one has a coin
• They will both show a side of their coin simultaneously
• If both show heads, no money will be exchanged
• If Roger shows heads and Colleen shows tails then
Colleen will give Roger $1
• If Roger shows tails and Colleen shows heads, then
Roger will pay Colleen $1
• If both show tails, then Colleen will give Roger $2

Example:
Coin matching game
• This is a Two person game, the players are Roger and
Colleen
• It is also a zero-sum game
• This means that Roger’s gain is Colleen’s loss
• We can use a 2 × 2 array or matrix to show all four
situations and the results

Example:
Coin matching game
• If Roger shows heads and Colleen shows tails then Colleen will give Roger $1
• If Roger shows tails and Colleen shows heads, then Roger will pay Colleen $1
Heads Tails
Heads
Tails
Roger
Colleen

Example:
Coin matching game
Heads Tails
Heads
Roger pays $0,
Colleen pays $0
Tails
Roger
Colleen

Example:
Coin matching game
• If Roger shows heads and Colleen shows tails then
Colleen will give Roger $1
Heads Tails
Heads
Roger pays $0,
Colleen pays $0
Roger gets $1,
Colleen pays $1
Tails
Roger
Colleen

Example:
Coin matching game
• If Roger shows tails and Colleen shows heads, then
Roger will pay Colleen $1
Heads Tails
Heads
Roger pays $0,
Colleen pays $0
Roger gets $1,
Colleen pays $1
Tails
Roger pays $1,
Colleen gets $1
Roger
Colleen

Example:
Coin matching game
Heads Tails
Heads
Roger pays $0,
Colleen pays $0
Roger gets $1,
Colleen pays $1
Tails
Roger pays $1,
Colleen gets $1
Roger gets $2,
Colleen pays $2
Roger
Colleen

Example:
Coin matching game
• The amount won by either
player in any given situation
is called the pay-off for that
player
• A negative pay-off denotes
a loss of that amount for the
player
Heads Tails
Heads 0,0 1,-1
Tails -1,1 2,-2
Roger
Colleen
• This is called a two-person, zero-sum game because the
amount won by each player is equal to the negative of the
amount won by the opponent for any given situation

Example:
Coin matching game
• The pay-off matrix for a game shows
only the pay-off for the row player
for each scenario
• A player’s plan of action against the
opponent is called a strategy
• In the above example, each player
has two possible strategies; H and T
Heads Tails
Heads 0 1
Tails -1 2
Roger
Colleen
• Since it is a zero-sum game, we can deduce the pay-off of one
player from that of the other, thus we can deduce all of the
above information from the pay-off matrix shown here

TCP Backoff Game in matrix form
Correct Defective
Correct
Defective
Player1
Player 2
-1,-1
-3,-30,-4
-4,0
• Should you send your packets using
– Correctly-implemented TCP ( which has
a “backoff” mechanism) or using a
defective implementation (which
doesn’t)?
• This problem is an example of what
we call a two-player game:
– Both use a correct implementation: both
get 1 ms delay
– One correct, one defective: 4 ms for
correct, 0 ms for defective
– Both defective: both get a 3 ms delay.

A Large Collective Action Game
• Players: N = {1, . . . , 10,000,000}
• Action set for player i Ai = {Revolt, Not}
• Utility function for player i:
– ui(a) = 1 if #{j : aj = Revolt} ≥ 2,000,000
– ui(a) = −1 if #{j : aj = Revolt} < 2,000,000 and ai = Revolt
– ui(a) = 0 if #{j : aj = Revolt} < 2,000,000 and ai = Not
• . Game

https://bemorepanda.com/en/posts/158479596
1-compilation-of-funniest-coronavirus-jokes-to-
lift-up-your-mood-and-stay-positive

Corona Time – Should I quarantine?
• Two people came to a village from
abroad
– Persons decide whether to quarantine or
not
– If both quarantine, both stay indoor for
14 days
– If both do not quarantine, then both will
be sentenced to 56 days in hospital (may
be infected) / jail (violation of rules)
– If one quarantine and the other does not,
then the quarantined person gets freed
after 14 days and the non-quarantined
guy sentenced to 28 days of
jail/hospitalization
– What should each person do?
7/18/2020 Dr Ganesh Neelakanta Iyer 42https://bemorepanda.com/en/posts/1584795961-compilation-of-
funniest-coronavirus-jokes-to-lift-up-your-mood-and-stay-positive

• Two people came to a village from abroad
– Persons decide whether to quarantine or not
– If both quarantine, both stay indoor for 14 days
– If both do not quarantine, then both will be
sentenced to 56 days in hospital (may be
infected) / jail (violation of rules)
– If one quarantine and the other does not, then
the quarantined person gets freed after 14
days and the non-quarantined guy sentenced
to 28 days of jail/hospitalization
– What should each person do?
Quarantine
No
Quarantine
Quarantine
No
Quarantine
Person1
Person 2
-14,-14
-56,-56-28,-14
-14,-28

• Each player’s predicted strategy is the
best response to the predicted
strategies of other players
• No incentive to deviate unilaterally
• Strategically stable or self-enforcing
Quarantine
No
Quarantine
Quarantine
No
Quarantine
Person1
Person 2
-14,-14
-56,-56-28,-14
-14,-28
So obey the instructions by Government; Lets
overcome this tough time soon

Prisoner’s Dilemma
• Two suspects arrested for a crime
– Prisoners decide whether to confess
or not to confess
– If both confess, both sentenced to 3
months of jail
– If both do not confess, then both will
be sentenced to 1 month of jail
– If one confesses and the other does
not, then the confessor gets freed (0
months of jail) and the non-confessor
sentenced to 9 months of jail
– What should each prisoner do?

Prisoner’s Dilemma: Revisited
• Two suspects arrested for a crime
• Prisoners decide whether to confess or not to
confess
• If both confess, both sentenced to 3 months of jail
• If both do not confess, then both will be sentenced
to 1 month of jail
• If one confesses and the other does not, then the
confessor gets freed (0 months of jail) and the
non-confessor sentenced to 9 months of jail
• What should each prisoner do?
Confess
Not
Confess
Confess
Not
Confess
Prisoner2
Prisoner 1
-3,-3
-1,-1-9,0
0,-9

Prisoner’s Dilemma: Nash Equilibrium
• Each player’s predicted strategy is the
best response to the predicted
strategies of other players
Confess Not Confess
Confess
Not Confess
Prisoner2
Prisoner 1
-3,-3
-1,-1-9,0
0,-9

48
Prisoner’s Dilemma: Nash Equilibrium
• Each player’s predicted strategy is the best response to the predicted strategies of other players
Confess Not Confess
Confess
Not Confess
Prisoner1
Prisoner 2
-3,-3
-1,-1-9,0
0,-9
http://www.environmentalgraffiti.com/people/news-are-humans-selfish-concept-homo-economicus

PD in general form
• Prisoner’s dilemma is any game
with c > a > d > b
C D
C
D
Player1 Player 2
a,a b, c
c, b d, d

Let’s play a game

Rock-paper-scissors game
• A probability distribution over the pure strategies of the game
• Rock-paper-scissors game
– Each player simultaneously forms his or her hand into the shape of either a
rock, a piece of paper, or a pair of scissors
– Rule: rock beats (breaks) scissors, scissors beats (cuts) paper, and paper
beats (covers) rock
• No pure strategy Nash equilibrium
• One mixed strategy Nash equilibrium – each player plays rock, paper and
scissors each with 1/3 probability

Rock-paper-scissors game
Rock Paper Scissor
Rock 0,0 -1,1 1,-1
Paper 1,-1 0,0 -1,1
Scissor -1,1 1,-1 0,0

Nash Equilibrium
NASH EQUILIBRIUM occurs when each player is pursuing their best possible
strategy in the full knowledge of the other players’ strategies. A Nash equilibrium is
reached when nobody has any incentive to change their strategy. It is named after
John Nash, a mathematician and Nobel prize-winning economist
John F. Nash, 1928 - 2015 Russell Crow portrays John Nash in A Beautiful Mind

Nash Equilibrium
• “A strategy profile is a Nash Equilibrium if and
only if each player’s prescribed strategy is a best
response to the strategies of others”
– Equilibrium that is reached even if it is not the best joint
outcome
4 , 6 0 , 4 4 , 4
5 , 3 0 , 0 1 , 7
1 , 1 3 , 5 2 , 3
Player 2
L C R
Player 1
U
M
D

Nash Equilibrium – Player 1 analysis
4 , 6 0 , 4 4 , 4
5 , 3 0 , 0 1 , 7
1 , 1 3 , 5 2 , 3
Player 2
L C R
Player 1
U
M
D
• If Player 2 chooses L, What is my best response?  M

4 , 6 0 , 4 4 , 4
5 , 3 0 , 0 1 , 7
1 , 1 3 , 5 2 , 3
Player 2
L C R
Player 1
U
M
D
• If Player 2 chooses C, What is my best response?  D

4 , 6 0 , 4 4 , 4
5 , 3 0 , 0 1 , 7
1 , 1 3 , 5 2 , 3
Player 2
L C R
Player 1
U
M
D
• If Player 2 chooses R, What is my best response?  U

4 , 6 0 , 4 4 , 4
5 , 3 0 , 0 1 , 7
1 , 1 3 , 5 2 , 3
Player 2
L C R
Player 1
U
M
D
• If Player 2 chooses L, What is my best response?  M
• If Player 2 chooses C, What is my best response?  D
• If Player 2 chooses R, What is my best response?  U

Do this analysis for Player 2
4 , 6 0 , 4 4 , 4
5 , 3 0 , 0 1 , 7
1 , 1 3 , 5 2 , 3
Player 2
L C R
Player 1
U
M
D

Solution for Player 2
• If Player 1 chooses U, What is my best response?  L
• If Player 2 chooses M, What is my best response?  R
• If Player 2 chooses D, What is my best response?  C
4 , 6 0 , 4 4 , 4
5 , 3 0 , 0 1 , 7
1 , 1 3 , 5 2 , 3
Player 2
L C R
Player 1
U
M
D

Nash Equilibrium
• “A strategy profile is a Nash Equilibrium if and only if
each player’s prescribed strategy is a best response to
the strategies of others”
– Equilibrium that is reached even if it is not the best joint outcome
4 , 6 0 , 4 4 , 4
5 , 3 0 , 0 1 , 7
1 , 1 3 , 5 2 , 3
Player 2
L C R
Player 1
U
M
D
Strategy Profile: {D,C} is
the Nash Equilibrium
**There is no incentive
for either player to
deviate from this
strategy profile

Problem 2
• Construct the payoff matrix and find NE
• Rules:
– Total market share equals 10
– Cost of advertising is 4 for high, 2 for low
– If firms both choose the same advertising level they split the
market, if one firm chooses high and the other low, than the firm
that chose high advertising gets the entire market.

Solution 2
• If column player chooses High, best response for Row player is to choose High (1>-2)
• If column player chooses Low, best response for Row player is to choose High (6>3)
• Similar analysis for Column player for each choices of row player
• (High, High) is the NE
Column Player
Row
Player
High Low
High (1,1) (6,-2)
Low (-2,6) (3,3)

64
Some Notations
niii
niii
iiniii
SXSXSXXSS
sssss
ssssssss
......
)...,,,...,(
),()...,,,,...,(
111
111
111






S-i simply means everything except Si

Strictly dominated strategies
• Lets take our grade game example from earlier discussion
• Play alpha! – Indeed, no matter what the pair does, by
playing alpha you would obtain a higher payoff
Definition: We say that my strategy alpha
strictly dominates my strategy beta, if my
payoff from alpha is strictly greater than that
from beta, regardless of what others do.
Do not play a strictly dominated strategy!

Another example: Two people want to choose an elective
and they put their mutual choices and possible grades
Biometrics Game Theory Cryptography
Biometrics A, B+ O, A A, B
Game Theory B, A A, B+ C, D
Cryptography O, A B, A B+, C
If I am ROW player can I eliminate one of the choice from my action list? If
yes, which one? Why?
For all possible actions by COLUMN player, Game Theory will give me a
bad grade compared to Biometrics. So eliminate Game Theory from my
action list.

67
Definition: Strictly Dominated Strategy
In a normal-form game G = {S1,…, Sn; u1,…, un}, let si’ and si’’ ϵ Si. Strategy si’ is
strictly dominated by strategy si” (or strategy si” strictly dominates strategy si’) if
for each feasible combination of other player’s strategies, player i’s payoffs from
playing si’ is strictly less than the payoff from playing si”. i.e.,
Rational players do not play strictly dominated strategies since they are always
not optimal no matter what strategies others would choose.
iiiiiiii Ssssussu   ),,(),( "'
QUESTION: What is the strictly dominated strategy and strictly dominant strategy for the game
“Prisoner’s Dilemma?

How do we find Nash equilibrium?
• In some cases, we can eliminate dominated strategies
• These are strategies that are inferior for every opponent
action
• Dominated strategy = A strategy dominated by some other
strategy
• Dominant strategy = A strategy better that some other
strategy
• Master/Slave  Who is dominated Master or Slave?
• Slave is dominated by Master; Master is dominant

Examples of dominance:
2, 3 5, 0
1, 0 4, 3
U
2
1
L
D
R
For player 1, U strictly dominates D.

Example #3
• A 3x3 example:
Left Middle
Down 0,3
1.5,2
0,1
1,0
Row
Column
Up
Right
0,1
2,0
70http://ganeshniyer.com

Example #3
• A 3x3 example:
Left Middle
Down 0,3
1.5,2
0,1
1,0
Row
Column
Up
Right
0,1
2,0

Example #3
• A 3x3 example:
Left Middle
Down 0,3
1.5,2
0,1
1,0
Row
Column
Up
Right
0,1
2,0
So the solution (Nash
Equilibrium) is (Row, Middle)
with payoff (1.5,2)

Game with no pure NE
Left Right
Left 1/0 0/1
Right 0/1 1/0
Penalty Taker
Goalie
Penalty taking in football (soccer)
https://www.youtube.com/watch?v=RqGb1Gx0t9U#t=41

Games with multiple NE
Compact Disc battle
• Battle for competing technical standards
• Sony and Philips competing for a standard for CD in late
1970s
• Each wanted their own system
Std A Std B
Std A 5/4 1/1
Std B 0/0 4/5
Philips
Sony
In the end, the result was a
mix of both

76
Battle of Sexes
• At the separate workplaces, Ram and Sita must choose to attend either
cricket or a movie in the evening.
• Both Ram and Sita know the following:
Both would like to spend the evening together.
But Ram prefers the cricket
Sita prefers the movie
2 , 1 0 , 0
0 , 0 1 , 2
Ram
Sita
Movie
Cricket
Movie
Cricket

77
Mixed Strategy
• A mixed strategy of a player is a probability distribution over
player’s (pure) strategies.
A mixed strategy for Ram is a probability distribution (p, 1-p), where p
is the probability of playing cricket, and 1-p is that probability of
watching movie.
If p=1 then Ram actually plays cricket. If p=0 then Ram actually
watches movie.
Battle of sexes Sita
Cricket Movie
Ram
Cricket (p) 2 , 1 0 , 0
Movie (1-p) 0 , 0 1 , 2

Wireless
Networks
Helsingør, Denmark

Example 1: The Forwarder’s Dilemma
?
?
Blue Green

80/37
From a problem to a game
• Users controlling the devices are rational = try to maximize their benefit
• Game formulation: G = (P,S,U)
– P: set of players
– S: set of strategy functions
– U: set of payoff functions
• Strategic-form representation
• Reward for packet reaching
the destination: 1
• Cost of packet forwarding:
c (0 < c << 1)
(1-c, 1-c) (-c, 1)
(1, -c) (0, 0)
Blue
Green
Forward
Drop
Forward Drop

81/37
Solving the Forwarder’s Dilemma (1/2)
' '
( , ) ( , ), ,i i i i i i i i i iu s s u s s s S s S       
iu U
i is S 
Strict dominance: strictly best strategy, for any strategy of the other player(s)
where: payoff function of player i
strategies of all players except player i
In Example 1, strategy Drop strictly dominates strategy Forward
(1-c, 1-c) (-c, 1)
(1, -c) (0, 0)
Blue
Green
Forward
Drop
Forward Drop
Strategy strictly dominates ifis

82/37
Solving the Forwarder’s Dilemma (2/2)
Solution by iterative strict dominance:
(1-c, 1-c) (-c, 1)
(1, -c) (0, 0)
Blue
Green
Forward
Drop
Forward Drop
Result: Tragedy of the commons ! (Hardin, 1968)
Drop strictly dominates Forward
Dilemma
Forward would result in a better outcome
BUT }

83/37
Example 2: The Multiple Access game
Reward for successful
transmission: 1
Cost of transmission: c
(0 < c << 1)
There is no strictly dominating strategy
(0, 0) (0, 1-c)
(1-c, 0) (-c, -c)
Blue
Green
Quiet
Transmit
Quiet Transmit
There are two Nash equilibria
Time-division channel

• Sequential moves are strategies where there
is a strict order of play.
• Perfect information implies that players know
everything that has happened prior to
making a decision.
• Complex sequential move games are most
easily represented in extensive form, using a
game tree.
• Chess is a sequential-move game with
perfect information.
Summary of Sequential Games

http://ganeshniyer.com 85
Dynamic Games of Complete
Information
Game Tree
Guess this place…

The E.T. “chocolate wars”
In the movie E.T. a trail of Reese's Pieces, one of
Hershey's chocolate brands, is used to lure the little
alien into the house. As a result of the publicity created
by this scene, sales of Reese's Pieces tripled, allowing
Hershey to catch up with rival Mars.
Page 87

Chocolate wars…the details
– Universal Studio's original plan was to use a trail of Mars’ M&Ms
and charge Mars $1mm for the product placement.
– However, Mars turned down the offer, presumably because it
thought $1mm was high.
– The producers of E.T. then turned to Hershey, who accepted the
deal, which turned out to be very favorable to them (and
unfavorable to Mars).
Page 88

Formal analysis of the chocolate wars
• Suppose:
– Publicity from M&M product placement increases Mars’ profits
by $800 k, decreases Hershey’s by $100 k
– Publicity from Reases Pieces product placement increases
Hershey’s profits by $1.2 m, decreases Mars’ by $500 k
– No product placement:
“business as usual”
Page 89

Extensive Form Games
• Also known as tree-form games
• Best to describe games with sequential actions
• Decision nodes indicate what player is to move (rules)
• Branches denote possible choices
• End nodes indicate each player’s payoff (by order of
appearance)
• Games solved by backward induction (more on this later)

Chocolate wars
Page 91
– Publicity from M&M product placement increases Mars’ profits by $800 k,
decreases Hershey’s by $100 k
– Publicity from Reases Pieces product placement increases Hershey’s profits by
$1.2 m, decreases Mars’ by $500 k
– No product placement: “business as usual”
[-500, 200]
[0, 0]
[-200, -100]
buy
not buy
not buy
buy
M
H
H

Chocolate wars [-500, 200]
[0, 0]
[-200, -100]
buy
not buy
not buy
buy
M
H
H
Page 92
Equilibrium strategies
– H chooses “buy”
– Anticipating H’s move, M chooses “buy”

Terrorists
Terrorists
President
(1, -.5)
(-.5, -1) (-1, 1)
(0, 1)
In the Movie Air Force One,
Terrorists hijack Air Force One and
take the president hostage. Can we
write this as a game?
In the third stage, the best response is to kill
the hostages
Given the terrorist response, it is optimal for
the president to negotiate in stage 2
Given Stage two, it is optimal for the
terrorists to take hostages

Terrorists
Terrorists
President
(1, -.5)
(-.5, -1) (-1, 1)
(0, 1)
The equilibrium is always (Take
Hostages/Negotiate). How could we change this
outcome?
Suppose that a constitutional amendment is
passed ruling out hostage negotiation (a
commitment device)
Without the possibility of negotiation, the new
equilibrium becomes (No Hostages)

95
Solving sequential games
• To solve a sequential game we look for the ‘subgame perfect
Nash equilibrium’
• For our purposes, this means we solve the game using ‘rollback’
– To use rollback, start at the end of each branch and work backwards,
eliminating all but the optimal choice for the relevant player

97
Subgame
• Its game tree is a branch of the original game tree
• The information sets in the branch coincide with the
information sets of the original game and cannot include
nodes that are outside the branch.
• The payoff vectors are the same as in the original game.

98
Subgame perfect equilibrium & credible threats
• Proper subgame = subtree (of the game tree) whose root
is alone in its information set
• Subgame perfect equilibrium
– Strategy profile that is in Nash equilibrium in every proper
subgame (including the root), whether or not that subgame is
reached along the equilibrium path of play

99
Backwards induction
• Start from the smallest subgames containing the terminal nodes of the game
tree
• Determine the action that a rational player would choose at that action node
– At action nodes immediately adjacent to terminal nodes, the player should maximize the
utility, This is because she no longer cares about strategic interactions. Regardless of
how she moves, nobody else can affect the payoff of the game.
Replace the subgame with the payoffs corresponding to the terminal node that
would be reached if that action were played
• Repeat until there are no action nodes left

Summary
Games
Non-
cooperative
Games
Static games of
complete
information
Sequential
games of
complete
information

Reputation
Reputation is intimately bound up with repetition.
For example:
1. Firms, both small and large, develop reputations for product quality
and after sales service through dealings with successive
customers.
2. Retail and Service chains and franchises develop reputations for
consistency in their product offerings across different outlets.
3. Individuals also cultivate their reputations through their personal
interactions within a community.

Definition of a repeated game
These examples motivate why we study reputation by analyzing the
solutions of repeated games.
When a game is played more than once by the same players in the
same roles, it is called a repeated game.
We refer to the original game (that is repeated) as the kernel game.
The number of rounds count the repetitions of the kernel game.
A repeated game might last for a fixed number of rounds, or be
repeated indefinitely (perhaps ending with a random event).

Games repeated a finite number of times
We begin the discussion by focusing on games that have a
finite number of rounds.
There are two cases to consider. The kernel game has:
1. a unique solution
2. multiple solutions.
For finitely repeated games this distinction turns out to be
the key to discussing what we mean by a reputation.

Two-Stage Repeated Game
• Two-stage prisoners’ dilemma
Two players play the following simultaneous move game twice
The outcome of the first play is observed before the second play
begins
The payoff for the entire game is simply the sum of the payoffs from
the two stages. That is, the discount factor is 1.
Player 2
L2 R2
Player 1
L1 1 , 1 5 , 0
R1 0 , 5 4 , 4
For ease of analysis, I
represent the values here
as positive and numbers
are representative

Game Tree of the Two-stage
Prisoners’ Dilemma
1
L1
R1
2
L2 R2
2
L2 R2
L1 R1
2
L2 R2
2
L2 R2
L1 R1
2
L2 R2
2
L2 R2
L1 R1
2
L2 R2
2
L2 R2
L1 R1
2
L2 R2
2
L2 R2
1+1
1+1
1+5
1+0
1+0
1+5
1+4
1+4
1 1 1 1
5+1
0+1
5+5
0+0
5+0
0+5
5+4
0+4
0+1
5+1
0+5
5+0
0+0
5+5
0+4
5+4
4+1
4+1
4+5
4+0
4+0
4+5
4+4
4+4

Informal Game Tree of the Two-
Stage Prisoners’ Dilemma
1
L1
R1
2
L2 R2
2
L2 R2
L1 R1
2
L2 R2
2
L2 R2
L1 R1
2
L2 R2
2
L2 R2
L1 R1
2
L2 R2
2
L2 R2
L1 R1
2
L2 R2
2
L2 R2
1
1
5
0
0
5
4
4
1 1 1 1(1, 1) (5, 0) (0, 5) (4, 4)
1
1
5
0
0
5
4
4
1
1
5
0
0
5
4
4
1
1
5
0
0
5
4
4

Infinitely repeated Prisoner’s Dilemma
• A game repeated infinitely
• Suppose the players play (C,C), (D,D), (C,C), (D,D),….
Forever
• We know the stage game payoffs 3,1,3,1,….
• Overall payoffs in a game with “x” repetitions can be
represented as
3, 3 0, 9
9, 0 1, 1
cooperate defect
cooperate
defect

t
x
x
iuE
1
)(

• In games of infinite repetitions there are two ways:
Limit average reward: lim inft→∞(1/t)Σx=1..tE[ui
x] e.g. if payoffs are 3, 1,
3, 1, …, payoff is 2
Future-discounted reward:
• E.g. if stage payoffs are 3, 1, 3, 1, … and discount factor δ =.9, then
payoff is 3 + 1*.9 + 3*.92+ 1*.93+ ...
Delta takes into the account that “present” is more important than “future”.
Definition of Nash Equilibrium though remains unchanged.
3, 3 0, 9
9, 0 1, 1
cooperate defect
cooperate
defect





1
1
3
2
21 )...
x
i
t
uuuu 

• Tit-for-tat strategy:
– Cooperate the first round,
– In every later round, do the same thing as the other player did in the previous
round
• Trigger strategy:
– Cooperate as long as everyone cooperates
– Once a player defects, defect forever
• What about one player playing tit-for-tat and the other playing trigger?
4, 4 0, 5
5, 0 1, 1
cooperate defect
cooperate
defect

Mechanism Design Principles
A shop in Belgium

Markets

Stock Markets

Internet

• Revolution in definition of markets

• New markets defined by
– Google
– Amazon
– Yahoo!
– Ebay

• Massive computational power available
for running these markets in a
centralized or distributed manner

• Massive computational power available
for running these markets in a
centralized or distributed manner
• Important to find good models and
algorithms for these markets

Theory of Algorithms
• Powerful tools and techniques
developed over last 4 decades.

Theory of Algorithms
• Powerful tools and techniques
developed over last 4 decades.
• Recent study of markets has contributed
handsomely to this theory as well!

Adwords Market
• Created by search engine companies
– Google
– Yahoo!
– MSN
• Multi-billion dollar market
• Totally revolutionized advertising, especially
by small companies.

The Adwords Problem:
N advertisers;
– Daily Budgets B1, B2, …, BN
– Each advertiser provides bids for keywords he is interested in.
Search Engine

N advertisers;
Search Enginequeries
(online)

N advertisers;
Search Engine
Select one Ad
Advertiser
pays his bid
queries
(online)
Maximize total revenue

Example:
$1 $0.99
$1 $0
Book
CD
Bidder1 Bidder 2
B1 = B2 = $100
Queries: 100 Books then 100 CDs
Bidder 1 Bidder 2
Algorithm Greedy
LOST
Revenue
100$

Example:
$1 $0.99
$1 $0
Book
CD
Bidder1 Bidder 2
B1 = B2 = $100
Queries: 100 Books then 100 CDs
Bidder 1 Bidder 2
Optimal Allocation
Revenue
199$

What is an Auction?
auc•tion
1. A public sale in which property or
merchandise are sold to the highest bidder.
2. A market institution with explicit rules
determining resource allocation and prices
on the basis of bids from participants.
3. Games: The bidding in bridge
[Latin: auctiō, auctiōn – from auctus,
past participle of augēre, to increase]
130

Examples of Auctions
• Definition:
– A market institution with rules governing resource allocation on the
basis of bids from participants
• Over 30% of US GDP moves through auctions:
131
 Wine
 Art
 Flowers
 Fish
 Electric power
 Treasury bills
 IPOs
 Emissions permits
 Radio Spectrum
 Import quotas
 Mineral rights
 Procurement

Overview of Auctions
• Auctions are a tricky business
• Different auction mechanisms
– sealed vs. open auctions
– first vs. second price
– optimal bidding & care in design
• Different sources of uncertainty
– private vs. common value auctions
– the winner’s curse
132

Private Value Auction
• Dinner
133

Common Value Auction
• Unproven oil fields
134

Sources of Uncertainty
• Private Value Auction
– Each bidder knows his or her value for the object
– Bidders differ in their values for the object
– e.g., memorabilia, consumption items
• Common Value Auction
– The item has a single though unknown value
– Bidders differ in their estimates of the true value of the object
– e.g., FCC spectrum, drilling, corporate takeovers
135

Basic Auction Types
• Open Auctions (sequential)
• English Auctions
• Dutch Auctions
• Japanese Auctions
• Sealed Auctions (simultaneous)
• First Price Sealed Bid
• Second Price Sealed Bid
136

English Auctions (Ascending Bid)
• Bidders call out prices (outcry)
• Auctioneer calls out prices (silent)
• Bidders hold down button (Japanese)
• Highest bidder gets the object
• Pays a bit over the next highest bid
137

Dutch (Tulip) Auction Descending Bid
• “Price Clock” ticks down the price
• First bidder to “buzz in” and stop the clock is the
winner
• Pays price on clock
138

Sealed-Bid First Price Auctions
• All buyers submit bids
• Buyer submitting the highest bid wins and pays
the price he or she bid
139
$700
$400
$500
$300
WINNER! Pays
$700

Sealed-Bid Second Price Auctions
• All buyers submit bids
• Buyer submitting the highest bid wins and pays
the second highest bid
140
$700
$400
$500
$300
WINNER! Pays
$500

Why Second Price?
• Bidding strategy is easy
– Bidding one’s true valuation
is a dominant strategy
• Intuition:
– The amount a bidder pays is not dependent on her bid
141

Optimal Bidding Strategy
in Second Price Auctions
142
You LoseYou Win
higher
Your bid
Others’ bids
Your value

Bidding Higher Than My Valuation
143
Case 1 Case 2 Case 3
No difference No difference Lose moneyhttp://ganeshniyer.com

Bidding Lower Than My Valuation
144
Case 1 Case 2 Case 3
No difference No difference Lose moneyhttp://ganeshniyer.com

Second Price Auction
• In a second price auction, always bid your true
valuation
• Winning bidder’s surplus
• Difference between the winner’s valuation and the second highest
valuation
• Surplus decreases with more bidders
145

More Bidders
• More bidders lead to higher prices
• Example
– Second price auction
– Each bidder has a valuation of either $20 or $40, each with
equal probability
– What is the expected revenue?
146

Number of Bidders
• Two bidders
– Each has a value of 20 or 40
– There are four value combinations:
Pr{20,20}=Pr{20,40}=Pr{40,20}=Pr{40,40}= ¼
Expected price = ¾ (20)+ ¼ (40) = 25
147

Number of Bidders
• Three bidders
– Each has a value of 20 or 40
– There are eight value combinations:
Pr{20,20,20}=Pr{20,20,40}=Pr{20,40,20}
= Pr{20,40,40}=Pr{40,20,20}=Pr{40,20,40}
= Pr{40,40,20}=Pr{40,40,40}= 1/8
Expected price = ½ (20)+ ½ (40) = 30
148

Number of Bidders
• Assume more generally that valuations are drawn uniformly from [20,40]:
149
20
25
30
35
40
1 10 100 1000
Number of Bidders
ExpectedPrice

Online Auctions
• Different types of auctions
– Increase-price auction (English auction)
– Decrease-price auction (Dutch auction)
– Second-price sealed-bid auction (Vickrey auction)
• English auction has become the most popular one in online
auction houses (e.g., eBay).
• However, it is time-consuming for a human user to search and
place bids on an auctioned item.
• There is a pressing need to introduce agent technology into online
auction systems.

Double Auction
• Prices are represented as a bid/ask spread
• This is the highest unmet bid to buy, and the lowest unmet bid to sell.
• Example:
– buy: 34, 36, 40, 47, 48
– sell: 50,52, 55, 60
– Bid/ask spread = 48-50
• Any “buy” greater than 50, or any sell less than 48 will close immediately.
• In theory, the market will converge to an equilibrium

Combinatorial auctions
• In all the problems we’ve seen so far, a single good is
being sold.
• Often, a seller would like to sell multiple interrelated
goods.
– FCC spectrum is the classic example.
– Bidders would like to bid on combinations of items.
• “I want item A, but only if I also win the auction for item B.”

• If we sell each good in a separate auction, agents
have a hard bidding problem.
– I don’t want to win only A, so I need to estimate my chances
of winning B.
• We might also let people place bids on combinations
of goods.
– Problem: determining the winner is NP-hard.
– Determining what to bid is at least that hard.
• Compromise: allow restricted combinations of bids.
(e.g. only XOR)
Combinatorial auctions

154
Auction Theory
• In economic theory, an auction may refer to any mechanism or set of trading rules for exchange.
• English Auction:open ascending price auction.
• Dutch Auction:open descending price auction.
• Vickery Auction: Sealed-bid second price auction
• First Price auction: Highest bidder pays the price they submitted
• Call Market: Mediator determines market clearing price based on number of bid and ask orders.
• CDA: Continuous Double Auctions

References
• H. Xu and Y-T Cheng Model Checking Bidding Behaviors in Internet Concurrent
Auctions. International Journal of Computer Systems Science & Engineering
(IJCSSE), July 2007, Vol. 22, No. 4, pp. 179-191.
• R. Patel, H. Xu, and A. Goel Real-Time Trust Management in Agent Based Online
Auction Systems. Proceedings of the19th International Conf. on Software
Engineering and Knowledge Engineering (SEKE'07), Boston, USA, July 2007, pp.
244-250.
• Y-T Cheng and H. Xu A Formal Approach to Detecting Shilling Behaviors in
Concurrent Online Auctions. Proceedings of the 8th International Conf. on Enterprise
Information Systems (ICEIS 2006), May 2006, Paphos, Cyprus, pp. 375-381.

Algorithmic
Mechanism Design
156
Bali, Indonesiahttp://ganeshniyer.com

Mechanism Design
Find correct rules/incentives

Mechanism Design
• Mechanism design can be viewed as the reverse engineering
of games or equivalently as the art of designing the rules of a
game to achieve a specific desired outcome.
• The main focus of mechanism design is to create institutions
or protocols that satisfy certain desired objectives, assuming
that the individual agents, interacting through the institution,
will act strategically and may hold private information that is
relevant to the decision at hand.

MD and network protocols
• Large networks (e.g. Internet) are built and controlled by
diverse and competitive entities
• Entities own different components of the network and hold
private information
• Entities are selfish and have different preferences
• MD is a useful tool to design protocols working in such an
environment

An example: auctions
t1=10
t2=12
t3=7
r1=11
r2=10
r3=7
the winner should be the
guy with highest value
the mechanism decides
the winner and the
corresponding payment
ti: type of player i
value player i is willing to pay
if player i wins and has to pay p
his utility is ui=ti-p 160http://ganeshniyer.com

A simple mechanism: no payment
t1=10
t2=12
t3=7
r1=+
r2=+
r3=+
…it doesn’t work…
?!?
The highest bid wins
and the price of the item
is 0

Another simple mechanism: pay your bid
t1=10
t2=12
t3=7
r1=9
r2=8
r3=6
Is it the right
choice?
and the winner will
pay his bid
The winner is
player 1 and he’ll
pay 9
Each player i will bid ri< ti
…player 2 could bid 9+ (=r1+), but he knows only his type…
…it doesn’t work… 162http://ganeshniyer.com

An elegant solution: Vickrey’s second price auction
t1=10
t2=12
t3=7
r1=10
r2=12
r3=7
every player has convenience
to declare the truth!
I know they are not
lying
and the winner will
pay the second
highest bid
The winner is
player 2 and he’ll
pay 10

Vickrey auction
(minimization version)
t1=10
t2=12
t3=7
r1=10
r2=12
r3=7
I want to allocate the
job to the true
cheapest machine
The cheapest bid wins
and the winner will
get the second
cheapest bid
The winner is
machine 3 and it
will receive 10
job to be
allocated to
machines
ti: cost incurred by i if i does the job
if machine i is selected and receives
a payment of p its utility is p-ti 164http://ganeshniyer.com

Some examples

Multiunit auction
t1
ti
tN
f(t): the set XF with
the highest total value
the set of k winners and the
corresponding payments
Each of N players wants an object
ti: value player i is willing to pay
if player i gets an object at price p
his utility is ui=ti-p
F={ X{1,…,N} : |X|=k }
...
k identical objects
(k < N)

Public project
t1
ti
tN
whether to build and the
payments from citizens
ti: value of the bridge
for citizen i
if the bridge is built and
citizen i has to pay pi
his utility is ui=ti-pi
F={build, not-build}
C: cost of
the bridge
to build or
not to build?
f(t):
build only if
iti > C

Bilateral trade
tb
decides whether
to trade and payments
ts: value of the object
if trade
seller’s utility:
ps-ts
F={trade, no-trade}
f(t):
trade only if
tb > ts
tb: value of the object
Mechanism
rs rb
if trade
buyer’s utility:
tb-pb
ps
pb
seller
ts
buyer

Buying a path in a network
decides the path
and the payments
te: cost of edge e
if edge e is selected
and receives a payment of pe
e’s utility:
pe-te
F: set of all paths
between s and t
f(t):
a shortest path w.r.t. the
true edge costs
Mechanism
t5
t3
t6
t2
t4
t1
s
t

Multiple Cloud
Orchestration
using Auctions
170
NYSE, New York

Cloud Orchestration
• Relates to the connectivity of IT and business process levels between
Cloud environments.
• As cloud emerges as a competitive sourcing strategy, a demand is
clearly arising for the integration of Cloud environments to create an end-
to-end managed landscape of cloud-based functions.
• Benefits include
– Helps users to choose the best service they are looking for (for example the
cheapest or the best email provider)
– Helps providers to offer better services and adapt to market conditions quickly
– Ability to create a best of breed service-based environment in which a change of
provider does not break the business process
Reference: Ganesh Neelakanta Iyer, Bharadwaj Veeravalli and Ramkumar Chandrasekaran, "Auction-based vs. Incentive-based Multiple-Cloud
Orchestration Mechanisms", IEEE International Conference on Communication, Networks and Satellite (COMNETSAT 2012), JULY 2012

Typical Cloud Broker
ecosystem showing the
players involved
The Broker helps to connect the
providers and users
http://www.optimis-project.eu/
Cloud Brokers

Auction Theory: Continuous Double
Auction
• A mechanism to match buyers and sellers of a particular good, and
to determine the prices at which trades are executed.
• At any point in time, traders can place limit orders in the form of bids
(buy orders) and asks (sell orders).
• Buyers and sellers can not modify their bids
Reference: Ganesh Neelakanta Iyer, Bharadwaj Veeravalli and Ramkumar Chandrasekaran, "Auction-based vs. Incentive-based Multiple-Cloud
Orchestration Mechanisms", IEEE International Conference on Communication, Networks and Satellite (COMNETSAT 2012), JULY 2012

Sealed-bid Continuous Double
Auctions
Comparison of revenue
• Hit Ratio is the ratio of the number
of successful auctions to the total
number of auctions.
• Fair revenue for all users
• Lowers user expenditure at the
expense of response-time for
choosing appropriate CSP.
Reference: Ganesh Neelakanta Iyer, Bharadwaj Veeravalli and Ramkumar Chandrasekaran, “Broker-agent based Cloud Service Arbitrage
Mechanisms using Sealed-bid Double Auctions and Incentives”, Journal of Network and Computer Applications (JNCA), Elsevier 2012

Cooperative Games
Kronborg, Denmark

Cooperative Games
Bargaining
Approaches
Coalition
Games

Bargaining Games
177

Cooperative Game Theory
• Players have mutual benefit to cooperate
– Startup company: everybody wants IPO, while competing for more stock shares.
– Coalition in Parliament
• Bargaining Games and Coalitional game
[178]http://ganeshniyer.com

Introduction to Bargaining
• Bargaining situation
– A number of individuals have a common interest to cooperate but a conflicting
interest on how to cooperate
• Key tradeoff
– Players wish to reach an agreement rather than disagree but…
– Each player is self interested
• What is bargaining?
– Process through which the players on their own attempt to reach an agreement
– Can be tedious, involving offers and counter-offers, negotiations, etc.
• Bargaining theory studies these situations, their
outcome, and the bargaining process

Introduction
• Key issues in bargaining
1. The players must inspect efficiency and the effect of delay and
disagreement on it
 They seek a jointly efficient mutual agreement
2. Distribution of the gains from the agreement
 Which element from the efficient set must the players elect?
3. What are the joint strategies that the players must choose to get the
desired outcome?
4. How to finally enforce the agreement?
• Link to game theory
– Issues 1 and 2 are tackled traditionally by cooperative game theory
– Issues 3 and 4 are strongly linked to non-cooperative game theory

Motivating Example
Rich Man
Can be deemed unsatistifactory
Given each Man’s wealth!!!
Bargaining theory
and the Nash
bargaining solution!
I can give you
100$ if and only if
you agree on how to
share it

The Nash Bargaining Solution
• John Nash’s approach
–When presented with a bargaining problem such as
the rich man – poor man case, how can we pick a
reasonable outcome?
–Interested in the outcome rather than the process
• The Nash Bargaining Solution was proposed in
1950 using an axiomatic approach and is considered
as one of the key foundations of bargaining problems

• Given a bargaining problem between two players
• Consider a utility region S that is compact and convex
– A utility is a function that assigns a value to every player, given the
strategy choices of both players
• Define the disagreement or threat point d in S which
corresponds to the minimum utilities that the players want to
achieve
• A Nash bargaining problem is defined by the pair (S,d)

• Can we find a bargaining solution, i.e., a function f that
specifies a unique outcome f(S,d) ϵ S ?
• Axiomatic approach proposed by Nash
– Axiom 1: Feasibility
– Axiom 2: Pareto efficiency
– Axiom 3: Symmetry
– Axiom 4: Invariance to linear transformation
– Axiom 5: Independence of irrelevant alternatives

• Axiom 1: Feasibility
– Can be sometimes put as part of the definition of the space
S
• Feasibility implies that
– The outcome of the bargaining process, denoted (u*,v*)
cannot be worse than the disagreement point d = (d1,d2),
i.e., (u*,v*) ≥ (d1,d2)
– Strict inequality is sometimes defined
• Trivial requirement but important: the disagreement
point is a benchmark and its selection is very
important in a problem!

• Axiom 2: Pareto efficiency
– Players need to do as well as they can without hurting one
another
• At the bargaining outcome, no player can improve
without decreasing the other player’s utility
– Pareto boundary of the utility region
• Formally, no point (u,v) ϵ S exists such that u > u* and
v ≥ v* or u ≥ u* and v > v*

• Axiom 3: Symmetry
– The principle of symmetry says that symmetric utility
functions should ensure symmetric payoffs
– Payoff should not discriminate between the identities of the
players.
– Formally, if d1 = d2 and S is symmetric around u = v, then
u*= v*
• Axiom 4: Invariance to linear transformation
– Simple axiom stating that the bargaining outcome varies
linearly if the utilities are scaled using an affine
transformation [187]http://ganeshniyer.com

• Axiom 5: Independence of irrelevant alternatives
– If the solution of the bargaining problem lies in a subset U
of S, then the outcome does not vary if bargaining is
performed on U instead of S

• Nash showed that there exists a unique solution f
satisfying the axioms, and it takes the following form:
Known as the Nash
product
When d1 = d2 = 0, this is equivalent to the
famous solution of telecommunication
networks: Proportional fairness

Rich man – poor man problem revisited
• Considering logarithmic utilities and considering that what the
men’s current wealth is as the disagreement point
– The Nash solution dictates that the rich man receives a larger share
of the 100$
• Is it fair?
– Fairness is subjective here, the rich man has more bargaining power
so he can threaten more to stop the deal
• The poor man also values little money big as he is already poor!
– Variant of the problem considers the 100$ as a debt, and, in that case,
the NBS becomes fair, the richer you are the more you pay!

• The NBS is easily extended to the N-person case
– The utility space becomes N-dimensional and the
disagreement point as well
– Computational complexity definitely increases and
coordination on a larger scale is required
• Solution to the following maximization problem

• If we drop the Symmetry axiom we define the
Generalize Nash Bargaining Solution
• Solution to the following maximization problem
Value between 0 and 1 representing the bargaining power of player i
If equal bargaining powers are used, this is equivalent to the NBS

Nash Bargaining Solution – Summary
• The NBS/GNBS are a very interesting concept for allocating
utilities in a bargaining problem
– Provide Pareto optimality
– Account for the bargaining power of the players but..
– Can be unfair, e.g., the rich man – poor man problem
– Require convexity of the utility region
– Independence of irrelevant alternatives axiom
– Provide only a static solution for the problem, i.e., no discussion of
the bargaining process
• Alternatives?
– The Kalai – Smorodinsky solution
– Dynamic bargaining and the Rubinstein process

Resource Allocation in Cloud Computing Environments
Ulu Watu, Bali, Indonesia

Resource Allocation in Cloud
Suitable for both independent tasks, Bag-of-Tasks (BoT) and tasks from workflow schemes
Assumption: Tasks are known apriori, but it can handle real-time arrival of tasks
Reference: Ganesh Neelakanta Iyer and Bharadwaj Veeravalli, “On the Resource Allocation and Pricing Strategies in Compute Clouds Using
Bargaining Approaches”, IEEE International Conference on Networks (ICON 2011), Singapore, December 2011.

Axiomatic Bargaining Approaches
• Good to derive fair and Pareto-optimal solution
• Pareto optimal: It is impossible to increase the allocation of a
connection without strictly decreasing another one.
• It assumes some desirable and fair properties, defined using axioms,
about the outcome of the resource bargaining process.
• Two approaches:
– Nash Bargaining Solution (NBS)
– Raiffa-Kalai-Smorodinsky Bargaining Solution (RBS)

Nash Bargaining Solution (NBS)
Solving, we obtain

Raiffa-Kalai-Smorodinsky Bargaining Solution (RBS)
Solving, we obtain

Resource Allocation in Cloud
Performance evaluation:
Deadline based Real-time task arrival

Coalition Games
Phang Nga Bay, Thailand

12-201 McGraw-Hill/Irwin
©2006 The McGraw-Hill Companies,
Inc., All Rights Reserved
Situations with More than
Two Parties
Variations on a three-party negotiation:
1. One buyer is representing the other and two negotiations
are occurring
2. The seller is conducting a sequenced series of one-on-
one transactions
3. The seller is about to unwittingly compromised by the
buyers (this happens when the parties form coalitions or
subgroups in order to strengthen their bargaining position
through collection action).

A Seller and Two Buyers

What Is a Coalition?
• Interacting groups of individuals
• Deliberately constructed and issue oriented
• Exist independent of formal structure
• Lack formal structure
• Focus goal external to the coalition
• Require collective action to achieve goals
• Members are trying to achieve outcomes that satisfy
the interests of the coalition

Cooperative/coalitional game theory
• There is a set of agents N
• Each subset (or coalition) S of agents can work together in various ways,
leading to various utilities for the agents
• Cooperative/coalitional game theory studies which outcome will/should
materialize
• Key criteria:
– Stability: No coalition of agents should want to deviate from the solution and go
their own way
– Fairness: Agents should be rewarded for what they contribute to the group
• (“Cooperative game theory” is the standard name (distinguishing it from noncooperative game theory,
which is what we have studied so far). However this is somewhat of a misnomer because agents still
pursue their own interests. Hence some people prefer “coalitional game theory.”)

Types of Coalitions
• Potential coalition: an emergent interest group that
has the potential to become a coalition by taking
collective action but has not yet done so.
– Two forms:
• Latent coalitions
– Emergent interest group that has not yet formed
• Dormant coalitions
– Interest group that previously formed, but is currently inactive

Types of Coalitions
• Operating coalition: one that is currently operating,
active, and in place.
– Two forms:
• Established coalition
– Relatively stable, active, and ongoing across an indefinite time
and space
– Members represent a broad range of interests
• Temporary coalition
– Operates for a short time
– Focused on a single issue or problem

Types of Coalitions
• Recurring coalitions: may have started as
temporary, but then determined that the issue or
problem does not remain resolved
– Members need to remobilize themselves every time the
presenting issue requires collective attention

Coalitions
• Players can form coalitions
• Coalition is a collective decision-maker
• Worth of each coalition is the total amount that the players
from the coalition can jointly guarantee themselves, it is
measured in abstract units of utility
http://staff.utia.cas.cz/kroupa/?id=3#TKH

Mathematical Model of Coalition Game

Problem 1: Market with two sellers and one
buyer
• There are three players in this game: two sellers and a
buyer. Each seller has one DVD, for which he paid $100,
and oﬀers to sell it. The buyer sets a worth of $200 on the
DVD. He is interested in paying the lowest possible price
for the DVD, and, of course, he is unwilling to pay more
than $200.
• Calculate the coalition function

• N = {1, 2, 3}; players 1 and 2 are the sellers and player 3
is the buyer.
– Seller 1 has a DVD worth $100 to him : v(1) = 100
– Seller 2 has a DVD worth $100 to him : v(2) = 100
– Buyer 3 does not have a DVD : v(3) = 0
• v({1,3}) = 200
• v({2,3}) = 200
• The worth of coalition {1, 2} is v(1, 2) = 200. This is a
coalition of two sellers in which each seller has a DVD
valued at $100 and so the coalition worth is $200

• v({1,2,3})
• Seller 1 says to seller 2: “Quit the game and don’t compete with me. In
return, I’ll pay you p dollars upon closing the deal.” Then player 1 sells
the DVD to buyer 3 for q dollars (of course, q > p). After the sale, the
players are left with the following amounts:
– Seller 1: the amount of (q − p) dollars.
– Seller 2: the amount of p dollars and a DVD worth $100 to him, for a
total of (100 + p).
– Buyer 3: the amount of (200−q) dollars: a DVD worth $200 to him
minus the amount q that he paid for the DVD. Therefore,
• v(1, 2, 3) = (q − p) + (100 + p) + (200 − q) = 300

Solution
• v(ϕ) = 0;
• v(1) = 100, v(2) = 100, v(3) = 0;
• v(1, 2) = 200, v(1, 3) = 200, v(2, 3) = 200;
• v(1, 2, 3) = 300

Shapely Value
• A simple system of four conditions or axioms which offers the
unbiased judge an opportunity to decide how to divide v(N) fairly
among the players in any given game. We shall see that these
axioms determine a unique way to divide v(N) in every game.
• The axioms were first formulated in 1953 by Lloyd Shapley, who
showed that indeed they dictate to the judge how to decide in every
case. The division of payoffs according to this decision is called the
Shapley value.

Shapely Value: Definition

Remarks

Problem
• Identify symmetric players and null player(s) in the following
game:
• N = {1, 2, 3} v(1) = v(2) = v(3) = 0 v(ϕ) = 0
• v(1, 2) = 30 v(1, 3) = 0 v(2, 3) = 0
• v(1, 2, 3) = 30

Solution
• In this game, players 1 and 2 are symmetric players and
player 3 is a null player
• Then what should be the division of payoffs?
• Since player 3 contributes nothing, it is reasonable that he should
get nothing.
• Hence the division of payoffs in this game is (15, 15, 0).

2
1
9
Mobile Cloud Environments
• Mobile cloud computing combines wireless access service and cloud
computing to improve the performance of mobile applications.
• Mobile applications can offload some computing modules (such as
online gaming) to be executed on a powerful server in a cloud.
• A scenario where multiple CSPs cooperatively offer mobile services
to users.
• Coalition games
Reference: Dusit Niyato, Ping Wang, Ekram Hossain, Walid Saad, and Zhu Han, “Game Theoretic Modeling of Cooperation amongService Providers
in Mobile Cloud Computing Environments”, IEEE Wireless Communications and Networking Conference, 2012

Shapely Value
• This expression can be viewed as capturing the “average
marginal contribution” of agent i, where we average over
all the different sequences according to which the grand
coalition could be built up from the empty coalition
• Given a coalitional game (N, v), the Shapley value of player i is
given by

• Imagine that the coalition is assembled by starting with the empty set
and adding one agent at a time, with the agent to be added chosen
uniformly at random.
• Within any such sequence of additions, look at agent i′s marginal
contribution at the time he is added. If he is added to the set S, his
contribution is [v(S ∪ {i}) − v(S)]
• Now multiply this quantity by the |S|! different ways the set S could
have been formed prior to agent i’s addition and
• by the (|N| − |S| − 1)! different ways the remaining agents could be
added afterward.
• Finally, sum over all possible sets S and obtain an average by
dividing by |N|!, the number of possible orderings of all the agents.

Some applications of
Shapely Value in Networks
Domain
Ship crossing between Sweden & Denmark

1. Content Delivery Networks
a. Live streaming, video-on-demand. Use peer-to-peer architecture to
reduce their operating cost
b. Users opt-in to dedicate part of the resources they own to help the
content delivery, in exchange for receiving the same service at a
reduced price
c. Shapely Value: This ensures that each player, be it the provider or a
peer, receives an amount proportional to its contribution and
bargaining power when entering the game
http://paloalto.thlab.net/uploads/papers/fp200-misra.pdf
http://conferences.sigcomm.org/co-next/2012/eproceedings/conext/p133.pdf

2. Profit Sharing in Wireless Networks:
a. Several service providers offer wireless access service to their
respective subscribed customers through potentially multi-hop routes
b. If providers cooperate, i.e., pool their resources, such as spectrum
and base stations, and agree to serve each others’ customers, their
aggregate payoffs, and individual shares, can potentially substantially
increase through efficient utilization of resources
http://repository.upenn.edu/cgi/viewcontent.cgi?article=1558&context=ese_papers

Multiuser Wireless Multimedia Transmission
Painting lighthouse,
Martha’s vineyards, USA

• Static multimedia resource allocation does not exploit network resources efficiently
• Channel conditions, video characteristics, number of users, users desired utilities etc
varies with time
• It does not provide adequate QoS support when network is congested
• Users can untruthfully declare their resource requirements to obtain a longer
transmission time
• Solution: Non-collaborative resource management game
Reference: Mihaela Van Der Schmar and Philip A Chou, “Multimedia over IP and Wireless Networks”, Academic Press 2007, Chapter 12, Section 12.9.4

2
2
8
Multiuser Wireless Multimedia
Transmission
•To play dynamic resource management game:
•Users deploy three different types of strategies at different stages of the
game
•Optimal expected cross-layer strategies and revealing strategies (before
transmission)
•Optimal real-time cross-layer strategy (during actual transmission)
•Thus users play competitive dynamic resource management game

Evolutionary Games
Yes…. The North Pole… View from Emirates

Non-cooperative vs Evolutionary Games
• Regular game theory
– Individual players make decisions
– Payoffs depend on decisions made by all
– The reasoning about what other players might do happens
simultaneously
• Evolutionary game theory
– Game theory continues to apply even if no individual is overtly
reasoning or making explicit decisions
– Decisions may thus not be conscious
– What behavior will persist in a population?

Overview of Evolutional Game
• Evolutionary game theory has been developed as a mathematical framework
to study the interaction among rational biological agents in a population
• Agent adapts (i.e., evolves) the chosen strategy based on its fitness (i.e.,
payoff)
• Example, hawk (be aggressive) and dove (be mild)

Evolutionary Stable Strategies (ESS)
• ESS is the key concept in the evolutionary process in which a group
of agents choosing one strategy will not be replaced by other
agents choosing a different strategy when the mutation mechanism
is applied
• Initial group of agents in a population chooses incumbent strategy s
• Small group of agents whose population share is ε choosing a
different mutant strategy s’
• Strategy s is called evolutionary stable if
where u(s, s’) denote the payoff of strategy s given that the opponent
chooses strategy s’

COVID-19
• One recent example
– Corona Virus A
• Infects human
– Corona Virus B (No mask, sanitizer, social distancing)
• Mutated version of A
• Can replicate inside human, but less efficiently
• Benefits from presence of A
– Is B evolutionarily stable?
http://static.businessworld.in/article/article_extra_large_image/1585130230_dgYMdV_Untitled_design_14_.jpg

Virus game
• Look at interactions between two viruses
– Viruses in a pure A population do better than viruses in pure B
population
– But regardless of what other viruses do, higher payoff to be B
• Thus B is evolutionarily stable
A B
A 1.00, 1.00 0.65, 1.99
B 1.99, 0.65 0.83, 0.83

Applications of Evolutionary Game
Congestion control – TCP AIMD
• The competition among two types of behaviors (i.e., aggressive and
peaceful) in wireless nodes to access the channel using a certain
protocol can be modeled as an evolutionary game
• Congestion control is (transport layer) to avoid performance
degradation by the ongoing users by limiting transmission rate
• The transmission rate (i.e., of TCP) can be adjusted by changing the
congestion window size (i.e., the maximum number of packets to be
transmitted)
• The speed-of-transmission rate to be increased and decreased
defines the aggressiveness of the protocol

Congestion control – Static game
• Analysis of the TCP protocol in a wireless environment is performed in which
the evolutionary game model (similar to the Hawk and Dove game)
• There are two populations (i.e., groups) of flows with TCP
• The flow from population i is characterized by parameters αi and βi, which are
the increase and decrease rates, respectively
• Strategy s of flow is to be aggressive (i.e., hawk or H) to be peaceful (i.e.,
dove or D)
• The parameters associated with these strategies are given as

• The packet loss occurs when the total transmission rate of all flows reaches
the capacity C- i.e., x1r1 +x2r2 = C, where xi is the proportion of population
choosing aggressive behavior
• The payoff of flow in population i is defined as follows:
where τi is the average throughput, L is the loss rate, and ω is the weight for
the loss
• Throughput of flow from population i can be obtained from

• The average throughput and loss rate can be defined as functions of
strategies of two populations i.e., τi(si, sj) and L(si, sj)
• It is shown that τi(H, H) = τi(D, D)
• When the loss rate is considered, it increases as the flow becomes more
aggressive, i.e., larger values of αi and βi
• Therefore, it can be shown that ui(H, H) < ui(D, D) and ui(D, H) < ui(D, D)
• Game becomes a Hawk and Dove model whose solution is ESS
• Briefly, it is found that the application that is loss-sensitive will tend to use a
less aggressive strategy at ESS

Evolutionary Games in Cloud/Edge/Fog Computing
Cape Cod, USA
Reference: Dr. Ganesh Neelakanta Iyer, “Evolutionary Games for Cloud, Edge and Fog Computing – A Comprehensive
Study”, in 5th International Conference on Computational Intelligence in Data Mining (ICCIDM-2018), Odisha, India , 2018.

Cloud/Edge/Fog Computing

Cloud Selection
• In [5], they study price competition in a heterogeneous
market cloud formed by CSPs, brokers and users
• Initially the competition among CSPs in selling the service
opportunities has been modelled using Non-cooperative
games where CSPs tries to maximize their revenues
Evolutionary game has been used to study dynamic
behaviour of cloud users to select a CSP based on
different factors such as price and delay
[5] C. T. Do et.al., Toward service selection game in a heterogeneous market cloud computing. IFIP/IEEE
International Symposium on Integrated Network Management (IM), 2015

Deployment in cloud environments
• In [7] for VM deployment under objectives such as energy
efficiency, budget and deadline, evolutionary games are used
• The works in [8], [9], [10] uses evolutionary game theory to
deploy a set of appli-cations in a set of hosts based on certain
performance objectives such as CPU and bandwidth
availability, response time, power consumption etc
243
7. K. Han, X. Cai and H. Rong. Hangzhou An Evolutionary Game Theoretic Approach for Efficient Virtual Machine Deployment in Green
Cloud.: IEEE, 2015. 2015 International Conference on Computer Science and Mechanical Automation (CSMA). pp. 1-4.
8. Y. Ren, J. Suzuki, A. Vasilakos, S. Omura and K. Oba. Cielo: An Evolutionary Game Theoretic Framework for Virtual Machine Placement in
Clouds. IEEE, 2014. 2014 Inter-national Conference on Future Internet of Things and Cloud, Barcelona, pp. 1-8. doi: 10.1109/FiCloud.2014.11
9. Yi Ren, Junichi S, Chonho Lee, Athanasios V. V, Shingo O, and Katsuya Oba Balancing performance, resource efficiency and energy
efficiency for virtual machine deployment in DVFS-enabled clouds: an evolutionary game theoretic approach. In Proceedings of Con-ference on
Genetic and Evolutionary Computation (GECCO Comp '14). pp. 1205-1212
10 Cheng, Yi Ren. Evolutionary Game theoretic multi-objective optimizaion algorithms and their applications. Computer Science, University of
Massachusetts Boston, 2017

Security issues in Fog computing
• When computation is performed at the fog nodes, they are
more susceptible to security vulnerabilities due to their
diverse and distributed nature
• In [12], the authors analyse the security issues in such an
environment using evolutionary games
• Replicator dynamics are used to understand the behavioural
strategy selection
• They show that when normal nodes show cooperative
strategy, the malicious nodes are forced to show “non-attack”
strategy
244
12. Yan Sun, Fuhong Lin, Nan Zhang,A security mechanism based on evolutionary game in fog computing, 2,
Saudi Arabia : ScienceDirect, 2018, Saudi Journal of Biological Sciences, , Vol. 25, pp. 237-241. ISSN 1319-562X

Optimal sensor configuration in Edge
computing
• The work in [13] uses game theory for configuring Body
Sensor Networks (BSNs) to be used with Cloud based on
operational conditions which depends on different constraints
such as resource consumption and data yield
• Their concept is based on a layered architecture where cloud
provider has virtual sensors and physical sensors are
operated through their cloud-based virtual counterparts
• They primarily use evolutionary games to study fine-tuning
sensing intervals and selection rates for sensors
245
Y. C. Ren et.al. An Evolutionary Game Theoretic Approach for Configuring Cloud-Integrated Body Sensor Networks
Cambridge, MA 2014 IEEE 13th International Symposium on Network Computing and Applications. pp. 277-281

Evolutionary games for Cloud/Edge/Fog
ComputingWork Type of
issue Basic Concept System Objectives Limitations
[5] Cloud
selection
Price selection in heterogeneous market cloud Cloud Price, delay Study is mostly on a
duopoly setup
[6] Cloud
selection
Price selection in the presence of multiple cloud
providers
Cloud Price, delay Multiple service delivery
models, SLA
agreements and
operations costs need to
be considered
[7] VM
deployme
nt
Optimal VM deployment based on several performance
objectives
Cloud Energy efficiency, budget,
deadline
[8],
[9],
[10]
Applicatio
n
deployme
nt
Deploy a set of applications in a set of hosts based on
certain performance objectives. They study adaptability
and stability
Cloud CPU and bandwidth
availability, response time,
power consumption
[11] Applicatio
n
deployme
nt
Help applications to choose their locations and in
allocating resources based on different characteristics
Cloud Response time Several other objectives
are important such as
energy efficiency and
price
[12] Security Security issues in fog computing environments. Fog Consumption cost, profit
from attacks
Lack real performance
studies
[13] Sensor
configurati
on
Configuring BSNs to be used with Cloud based on
operational conditions with respect to different
constraints
Edge, Cloud Resource consumption,
data yield
[14] Computati
onal off-
loading
Computational offloading for mobile edge computing Edge, Cloud Time, energy
consumption, monetary
cost
[15],
[16]
Resource
allocation
QoS constrained cloud resource allocation Cloud Budget, computation time
[17] Resource Allocating resources within the cloud mobile social Cloud Price, Processing rate
246

Summary...
Water-puppetry, Vietnam

To Summarize...
• Various Concept in Game Theory can be used almost everywhere to handle
conflicting situations and for cooperation enforcement
• Topics not covered (much more than what is discussed)
– Congestion and Potential Games
– Bayesian games
– Combinatorial auctions
– Differential games
– Signaling Games
– Markovian Games
– Stackleberg Games
– …

My lunch at Martha’s vineyards, USA

Dr Ganesh Neelakanta Iyer
ganesh@ganeshniyer.com
ganesh.vigneswara@gmail.com
GANESHNIYER
https://amrita.edu/faculty/ni-ganesh

Making Decisions - A Game Theoretic approach

Recommended

Recommended

More Related Content

Similar to Making Decisions - A Game Theoretic approach

Similar to Making Decisions - A Game Theoretic approach (20)

More from Dr Ganesh Iyer

More from Dr Ganesh Iyer (20)

Recently uploaded

Recently uploaded (20)

Making Decisions - A Game Theoretic approach