Game Theory and Engineering Applications

Game Theory and
Engineering Applications
INDICON 2018 Tutorials
Dr Ganesh Neelakanta Iyer
Amrita Vishwa Vidyapeetham, Coimbatore
Associate Professor, Dept of Computer Science and Engg
https://amrita.edu/faculty/ni-ganesh
http://ganeshniyer.com

About Me • Associate Professor, Amrita Vishwa Vidyapeetham
• Masters & PhD from National University of Singapore (NUS)
• Several years in Industry/Academia
• Sasken Communications, NXP Semiconductors, Progress
Software, IIIT-HYD, NUS (Singapore)
• Architect, Manager, Technology Evangelist, Visiting Faculty
• Talks/workshops in USA, Europe, Australia, Asia
• Cloud/Edge Computing, IoT, Game Theory, Software QA
• Kathakali Artist, Composer, Speaker, Traveler, Photographer
GANESHNIYER http://ganeshniyer.com

3
Outline for today (Not in slide’s order)
• Overview of Game Theory
• Non – Cooperative Games
– Two person Games
– Repeated Games
– Congestion Games and Potential Games
– Evolutionary Games
• Cooperative Games
– Bargaining Games
– Coalition Games
• Mechanism Design
– Auctions
12/16/2018 Dr Ganesh Neelakanta Iyer

Some application domains we will touch upon
Dr Ganesh Neelakanta Iyer 4

5
Overview of Game Theory
5
Raffles Place, Singapore

What is Game Theory About?
• Analysis of situations where conflict of interests are present
Goal is to prescribe how conflicts can be resolved
2
2
Game of Chicken
driver who steers away looses
What should drivers do?
12/16/2018 Dr Ganesh Neelakanta Iyer 6

Game Theory
• Study of how people interact and make decisions
• Outcome of a person’s decision depends not just on how they
choose among several options, but also on the choices made by the
people they are interacting with

TCP Back off Game
TCP Congestion Control - AIMD
◼ Algorithm AIMD
 Additive Increase Multiplicative Decrease
 Increment Congestion Window by one packet per RTT
◼ Linear increase
 Divide Congestion Window by two whenever a
timeout occurs
◼ Multiplicative decrease
Source Destination
…
60
20
1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0
KB
T ime (seconds)
70
30
40
50
10
10.0

TCP Backoff Game
• Should you send your packets using
– Correctly-implemented TCP ( which has a “backoff” mechanism) or
– Using a defective implementation (which doesn’t)?
• This problem is an example of what we call a two-player
game:
– Both use a correct implementation: both get 1 ms delay
– One correct, one defective: 4 ms for correct, 0 ms for defective
– Both defective: both get a 3 ms delay.

Self Interested Agents
• What does it mean to say that an agent is self-interested?
– Not that they want to harm others or only care about themselves
• Only that the agent has its own
description of states of the world that
it likes, and acts based on this
description

Self Interested Agents
• Each such agent has a
utility function
– quantifies degree of
preference across alternatives
– explains the impact of
uncertainty
– Decision-theoretic rationality:
act to maximize expected
utility

What is a game?
Players: who are the decision makers?
• People? Governments? Companies?
• Somebody employed by a Company?...
Actions: What can the players do?
• Enter a bid in an auction?
• Decide whether to end a strike?
• Decide when to sell a stock?
Strategies: Which action did I choose
• actions which a player chooses to follow
• I will sell the stock today, I will vote for XYZ Party
Payoffs: what motivates the players?
• Do they care about some profit?
• Do they care about other players?...
Outcome: What is the result?
• Determined by mutual choice of strategies

Different types of
Games

Types of Games
Games
Simultaneous
Games
Sequential
Games

Types of Games
Games
Symmetric
Games
Asymmetric
games

Types of Games
Games
Perfect
Information
Imperfect
Information

Types of Games
Games
Cooperative
Non-
cooperative

Types of Games
Games
Zero sum Non zero sum

Types of Games
Games
Single Shot
game
Repeated
Game
mutually assured nuclear destruction

Different Types – by Concept
Cooperative
• Bargaining
• Coalitions
• Stable matching
Non-Cooperative
• Zero-sum
• Non-Zero sum
• Bayesian
• Evolutionary
• Congestion/Potential
• Matrix
• Stackelberg
• Jamming
• Markovian
• Combinatorial
• Differential
Mechanism Design
• Auctions
• VCG Mechanisms
• Social Choice
• Optimal Mechanisms
And a lot
more…
12/16/2018

Game Theory:
Applications
• Economics: Oligopoly markets, Mergers and
acquisitions pricing, auctions
• Political Science: fair division, public choice,
political economy
• Biology: modeling competition between
tumor and normal cells, Foraging bees
• Sports coaching staffs: run vs pass or pitch
fast balls vs sliders
• Engineering: Wireless Networks, Distributed
systems, Computer Networks, AI
http://customergauge.com/wordpress/wp-content/uploads/2008/10/power_retailers_oligopoly.jpg
http://cricketradius.com/wp-content/uploads/2011/11/fast-bowling.jpg

Game Theory:
Engineering Applications
• Electronics and Communication
– Power Allocation
– Cognitive Radio Networks
– Wireless Networks
• Computer Science
– Distributed systems
– Computer Networks
– Artificial Intelligence
– Multi-agent systems
• Electrical Engineering
– Smart Grid
– Voltage Regulation
• Civil
– Construction Engineering
http://www.cee.ntu.edu.sg/Programmes/graduate/MSc_CE/Pages/Overview.aspx
https://www.edie.net/news/6/Britain-gears-up-for-smart-energy-transition-with-new-consultation/
http://ncel.ie.cuhk.edu.hk/content/qos-support-cognitive-radio-networks

Non-Cooperative Games

Defining Games:
Two standard representations
• Normal Form (a.k.a. Matrix Form, Strategic Form) List what
payoffs get as a function of their actions
– It is as if players moved simultaneously
– But strategies encode many things...
• Extensive Form Includes timing of moves (later in course)
– Players move sequentially, represented as a tree
• Chess: white player moves, then black player can see white’s move and react…
– Keeps track of what each player knows when he or she makes each
decision
• Poker: bet sequentially – what can a given player see when they bet?

Defining Games:
The Normal Form
• Finite, n-person normal form game: ⟨N, A, u⟩:
– Players: N = {1, … , n} is a finite set of n, indexed by i
– Action set for player i, Ai :
• a = (a1, … ,an) ∈ A = A1 X … X An is an action profile
– Utility function or Payoff function for player i: ui : A→ R
• u = (u11, …, un) , is a profile of utility functions

Normal Form Games
The Standard Matrix Representation
• Writing a 2-player game as a matrix:
– “row” player is player 1, “column” player is player 2
– rows correspond to actions a1 ∈ A1, columns correspond to actions
a2 ∈ A2
– cells listing utility or payoff values for each player: the row player first,
then the column
• .
QUESTION: Write TCP Backoff Game in matrix form

TCP Backoff Game in matrix form
Correct Defective
Correct
Defective
Player1
Player 2
-1,-1
-3,-30,-4
-4,0
• Should you send your packets using
– Correctly-implemented TCP ( which has
a “backoff” mechanism) or using a
defective implementation (which
doesn’t)?
• This problem is an example of what
we call a two-player game:
– Both use a correct implementation: both
get 1 ms delay
– One correct, one defective: 4 ms for
correct, 0 ms for defective
– Both defective: both get a 3 ms delay.

A Large Collective Action Game
• Players: N = {1, . . . , 10,000,000}
• Action set for player i Ai = {Revolt, Not}
• Utility function for player i:
– ui(a) = 1 if #{j : aj = Revolt} ≥ 2,000,000
– ui(a) = −1 if #{j : aj = Revolt} < 2,000,000 and ai = Revolt
– ui(a) = 0 if #{j : aj = Revolt} < 2,000,000 and ai = Not
• . Game

Prisoner’s Dilemma
• Two suspects arrested for a crime
– Prisoners decide whether to confess
or not to confess
– If both confess, both sentenced to 3
months of jail
– If both do not confess, then both will
be sentenced to 1 month of jail
– If one confesses and the other does
not, then the confessor gets freed (0
months of jail) and the non-confessor
sentenced to 9 months of jail
– What should each prisoner do?

Prisoner’s Dilemma: Revisited
• Two suspects arrested for a crime
• Prisoners decide whether to confess or not to
confess
• If both confess, both sentenced to 3 months of jail
• If both do not confess, then both will be sentenced
to 1 month of jail
• If one confesses and the other does not, then the
confessor gets freed (0 months of jail) and the
non-confessor sentenced to 9 months of jail
• What should each prisoner do?
Confess
Not
Confess
Confess
Not
Confess
Prisoner2
Prisoner 1
-3,-3
-1,-1-9,0
0,-9

Prisoner’s Dilemma: Nash Equilibrium
• Each player’s predicted strategy is the
best response to the predicted
strategies of other players
• No incentive to deviate unilaterally
• Strategically stable or self-enforcing
Confess Not Confess
Confess
Not Confess
Prisoner2
Prisoner 1
-3,-3
-1,-1-9,0
0,-9

Let’s play a game

Rock-paper-scissors game
• A probability distribution over the pure strategies of the game
• Rock-paper-scissors game
– Each player simultaneously forms his or her hand into the shape of either a
rock, a piece of paper, or a pair of scissors
– Rule: rock beats (breaks) scissors, scissors beats (cuts) paper, and paper
beats (covers) rock
• No pure strategy Nash equilibrium
• One mixed strategy Nash equilibrium – each player plays rock, paper and
scissors each with 1/3 probability

Rock-paper-scissors game
Rock Paper Scissor
Rock 0,0 -1,1 1,-1
Paper 1,-1 0,0 -1,1
Scissor -1,1 1,-1 0,0

Definition: Normal form of a Game
• The normal-form (also called strategic-form) representation of an n-player
game specifies the players' strategy spaces S1, …, Sn and their payoff
functions u1…un. We denote this game by
G = {S1,…, Sn; u1,…, un}
• Let (s1,…,sn) be a combination of strategies, one for each player. Then ui(s1,…,sn) is the payoff
to player i if for each j = 1,…,n, player j chooses strategy sj.
• The payoff a player depends not only on his own action but also on the actions of others! This
inter-dependence is the essence of games!

Question: Normal form representation of
Prisoner’s dilemma
G = {S1,S2; u1,u2}
S1 = {Confess, Not Confess} = S2
u2(C,NC)= -9, u1(C,NC)= 0, …
Confess Not Confess
Confess
Not Confess
Prisoner1
Prisoner 2
-3,-3
-1,-1-9,0
0,-9

Nash Equilibrium
NASH EQUILIBRIUM occurs when each player is pursuing their best possible
strategy in the full knowledge of the other players’ strategies. A Nash equilibrium is
reached when nobody has any incentive to change their strategy. It is named after
John Nash, a mathematician and Nobel prize-winning economist
John F. Nash, 1928 - 2015 Russell Crow portrays John Nash in A Beautiful Mind

Nash Equilibrium
• “A strategy profile is a Nash Equilibrium if and only if
each player’s prescribed strategy is a best response to
the strategies of others”
– Equilibrium that is reached even if it is not the best joint outcome
4 , 6 0 , 4 4 , 4
5 , 3 0 , 0 1 , 7
1 , 1 3 , 5 2 , 3
Player 2
L C R
Player 1
U
M
D
Strategy Profile: {D,C} is
the Nash Equilibrium
**There is no incentive
for either player to
deviate from this
strategy profile

Wireless
Networks
Helsingør, Denmark

Example 1: The Forwarder’s Dilemma
?
?
Blue Green

42/37
From a problem to a game
• Users controlling the devices are rational = try to maximize their benefit
• Game formulation: G = (P,S,U)
– P: set of players
– S: set of strategy functions
– U: set of payoff functions
• Strategic-form representation
• Reward for packet reaching
the destination: 1
• Cost of packet forwarding:
c (0 < c << 1)
(1-c, 1-c) (-c, 1)
(1, -c) (0, 0)
Blue
Green
Forward
Drop
Forward Drop

43/37
Solving the Forwarder’s Dilemma (1/2)
' '
( , ) ( , ), ,i i i i i i i i i iu s s u s s s S s S− − − −    
iu U
i is S− −
Strict dominance: strictly best strategy, for any strategy of the other player(s)
where: payoff function of player i
strategies of all players except player i
In Example 1, strategy Drop strictly dominates strategy Forward
(1-c, 1-c) (-c, 1)
(1, -c) (0, 0)
Blue
Green
Forward
Drop
Forward Drop
Strategy strictly dominates ifis

44/37
Solving the Forwarder’s Dilemma (2/2)
Solution by iterative strict dominance:
(1-c, 1-c) (-c, 1)
(1, -c) (0, 0)
Blue
Green
Forward
Drop
Forward Drop
Result: Tragedy of the commons ! (Hardin, 1968)
Drop strictly dominates Forward
Dilemma
Forward would result in a better outcome
BUT }

45/37
Example 2: The Joint Packet Forwarding Game
?
Blue GreenSource Dest
?
No strictly dominated strategies !
• Reward for packet
reaching the destination: 1
• Cost of packet forwarding:
c (0 < c << 1)
(1-c, 1-c) (-c, 0)
(0, 0) (0, 0)
Blue
Green
Forward
Drop
Forward Drop

46/37
Weak dominance
?
Blue GreenSource Dest
?
'
( , ) ( , ),i i i i i i i iu s s u s s s S− − − −  
Weak dominance: strictly better strategy for at least one opponent strategy
with strict inequality for at least one s-i
Iterative weak dominance
(1-c, 1-c) (-c, 0)
(0, 0) (0, 0)
Blue
Green
Forward
Drop
Forward Drop
BUT
The result of the iterative weak
dominance is not unique in general !
Strategy s’i is weakly dominated by strategy si if

47/37
Nash equilibrium (1/2)
Nash Equilibrium: no player can increase its payoff by deviating unilaterally
(1-c, 1-c) (-c, 1)
(1, -c) (0, 0)
Blue
Green
Forward
Drop
Forward Drop
E1: The Forwarder’s
Dilemma
E2: The Joint Packet
Forwarding game (1-c, 1-c) (-c, 0)
(0, 0) (0, 0)
Blue
Green
Forward
Drop
Forward Drop

48/37
Example 3: The Multiple Access game
Reward for successful
transmission: 1
Cost of transmission: c
(0 < c << 1)
There is no strictly dominating strategy
(0, 0) (0, 1-c)
(1-c, 0) (-c, -c)
Blue
Green
Quiet
Transmit
Quiet Transmit
There are two Nash equilibria
Time-division channel

49/37
Mixed strategy Nash equilibrium
objectives
– Blue: choose p to maximize ublue
– Green: choose q to maximize ugreen
(1 )(1 ) (1 )blueu p q c pqc p c q= − − − = − −
(1 )greenu q c p= − −
1 , 1p c q c= − = −
p: probability of transmit for Blue
q: probability of transmit for Green
is a Nash equilibrium

50/37
Example 4: The Jamming game
transmitter:
• reward for successful
transmission: 1
• loss for jammed
transmission: -1
jammer:
• reward for successful
jamming: 1
• loss for missed
jamming: -1
There is no pure-strategy
Nash equilibrium
two channels:
C1 and C2
(-1, 1) (1, -1)
(1, -1) (-1, 1)
Blue
Green
C1
C2
C1 C2
transmitter
jammer
1 1
,
2 2
p q= = is a Nash equilibrium
p: probability of transmit
on C1 for Blue
q: probability of transmit
on C1 for Green

http://ganeshniyer.com 51
Dynamic Games of Complete
Information
Game Tree
Guess this place…

The E.T. “chocolate wars”
In the movie E.T. a trail of Reese's Pieces, one of
Hershey's chocolate brands, is used to lure the little
alien into the house. As a result of the publicity created
by this scene, sales of Reese's Pieces tripled, allowing
Hershey to catch up with rival Mars.
Page 53

Chocolate wars…the details
– Universal Studio's original plan was to use a trail of Mars’ M&Ms
and charge Mars $1mm for the product placement.
– However, Mars turned down the offer, presumably because it
thought $1mm was high.
– The producers of E.T. then turned to Hershey, who accepted the
deal, which turned out to be very favorable to them (and
unfavorable to Mars).
Page 54

Formal analysis of the chocolate wars
• Suppose:
– Publicity from M&M product placement increases Mars’ profits
by $800 k, decreases Hershey’s by $100 k
– Publicity from Reases Pieces product placement increases
Hershey’s profits by $1.2 m, decreases Mars’ by $500 k
– No product placement:
“business as usual”
Page 55

Extensive Form Games
• Also known as tree-form games
• Best to describe games with sequential actions
• Decision nodes indicate what player is to move (rules)
• Branches denote possible choices
• End nodes indicate each player’s payoff (by order of
appearance)
• Games solved by backward induction (more on this later)
http://ganeshniyer.com 56

Chocolate wars
Page 57
– Publicity from M&M product placement increases Mars’ profits by $800 k,
decreases Hershey’s by $100 k
– Publicity from Reases Pieces product placement increases Hershey’s profits by
$1.2 m, decreases Mars’ by $500 k
– No product placement: “business as usual”
[-500, 200]
[0, 0]
[-200, -100]
buy
not buy
not buy
buy
M
H
H

Chocolate wars [-500, 200]
[0, 0]
[-200, -100]
buy
not buy
not buy
buy
M
H
H
Page 58
Equilibrium strategies
– H chooses “buy”
– Anticipating H’s move, M chooses “buy”

59/37
Extensive-form games
• Example 3 modified: the Sequential Multiple Access game:
• Blue plays first, then Green plays.
Green
Blue
T Q
T Q T Q
(-c,-c) (1-c,0) (0,1-c) (0,0)
Reward for successful
transmission: 1
Cost of transmission: c
(0 < c << 1)
Green
Time-division channel

60/37
Strategies in dynamic games
• The strategy defines the moves for a player for every node in the game,
even for those nodes that are not reached if the strategy is played.
Green
Blue
T Q
T Q T Q
(-c,-c) (1-c,0) (0,1-c) (0,0)
Green
strategies for Blue:
T, Q
strategies for Green:
TT, TQ, QT and QQ
TQ means that player p2 transmits if p1 transmits and remains quiet if p1 remains quiet.

61/37
Backward induction
• Solve the game by reducing from the final stage
• Eliminates Nash equilibria that are incredible threats
Green
Blue
T Q
T Q T Q
(-c,-c) (1-c,0) (0,1-c) (0,0)
GreenIncredible threat: (Q, TT)
Backward induction solution: h={T, Q}

62/37
Subgame perfection
Extends the notion of Nash equilibrium
Green
Blue
T Q
T Q T Q
(-c,-c) (1-c,0) (0,1-c) (0,0)
Green
Subgame perfect equilibria: (T, QT) and (T, QQ)
One-deviation property: A strategy si conforms to the one-deviation property if there does not exist
any node of the tree, in which a player i can gain by deviating from si and apply it otherwise.
Subgame perfect equilibrium: A strategy profile s constitutes a subgame perfect equilibrium if the
one-deviation property holds for every strategy si in s.
Finding subgame perfect equilibria using backward
induction
Stackelberg games have one leader and one or several followers

Terrorists
Terrorists
President
(1, -.5)
(-.5, -1) (-1, 1)
(0, 1)
In the Movie Air Force One,
Terrorists hijack Air Force One and
take the president hostage. Can we
write this as a game?
In the third stage, the best response is to kill
the hostages
Given the terrorist response, it is optimal for
the president to negotiate in stage 2
Given Stage two, it is optimal for the
terrorists to take hostages

Terrorists
Terrorists
President
(1, -.5)
(-.5, -1) (-1, 1)
(0, 1)
The equilibrium is always (Take
Hostages/Negotiate). How could we change this
outcome?
Suppose that a constitutional amendment is
passed ruling out hostage negotiation (a
commitment device)
Without the possibility of negotiation, the new
equilibrium becomes (No Hostages)

66/37
Repeated games
• Repeated interaction between the players (in stages)
• move: decision in one interaction
• strategy: defines how to choose the next move, given the
previous moves
• history: the ordered set of moves in previous stages
– most prominent games are history-1 games (players consider only
the previous stage)
• initial move: the first move with no history
• finite-horizon vs. infinite-horizon games
• stages denoted by t (or k)

67/37
Strategies in the repeated game
Example strategies in the Forwarder’s Dilemma:
Blue (t) initial
move
F D strategy name
Green (t+1) F F F AllC
F F D Tit-For-Tat (TFT)
D D D AllD
F D F Anti-TFT

68/37
The Repeated Forwarder’s Dilemma
(1-c, 1-c) (-c, 1)
(1, -c) (0, 0)
Blue
Green
Forward
Drop
Forward Drop
?
?
Blue Green
stage payoff

69/37
Analysis of the Repeated Forwarder’s
Dilemma (1/3)
Blue strategy Green strategy
AllD AllD
AllD TFT
AllD AllC
AllC AllC
AllC TFT
TFT TFT
infinite game with discounting: ( )
0
t
i i
t
u u t 

=
= 
Blue payoff Green payoff
0 0
1 -c
1/(1-ω) -c/(1-ω)
(1-c)/(1-ω) (1-c)/(1-ω)
(1-c)/(1-ω) (1-c)/(1-ω)
(1-c)/(1-ω) (1-c)/(1-ω)

70/37
Dilemma (2/3)
Blue strategy Green strategy
AllD AllD
AllD TFT
AllD AllC
AllC AllC
AllC TFT
TFT TFT
Blue payoff Green payoff
0 0
1 -c
1/(1-ω) -c/(1-ω)
(1-c)/(1-ω) (1-c)/(1-ω)
(1-c)/(1-ω) (1-c)/(1-ω)
(1-c)/(1-ω) (1-c)/(1-ω)
• AllC receives a high payoff with itself and TFT, but
• AllD exploits AllC
• AllD performs poor with itself
• TFT performs well with AllC and itself, and
• TFT retaliates the defection of AllD
TFT is the best strategy if ω is high !

71/37
Dilemma (3/3)
Theorem: In the Repeated Forwarder’s Dilemma, if both players play AllD, it is a
Nash equilibrium.
Theorem: In the Repeated Forwarder’s Dilemma, both players playing TFT is a Nash
equilibrium as well.
Blue strategy Green strategy Blue payoff Green payoff
AllD AllD 0 0
TFT TFT (1-c)/(1-ω) (1-c)/(1-ω)
The Nash equilibrium sBlue = TFT and sGreen = TFT is
Pareto-optimal (but sBlue = AllD and sGreen = AllD is not) !

72/37
Conclusions
• Non-cooperative games help us model rational and selfish
players and the interaction among them
• Game theory can help modeling greedy behavior in
wireless networks and in many other areas

Non cooperative games for Security

74/38
Who is malicious? Who is selfish?
➔ Both security and game theory backgrounds are useful in many cases !!
Harm everyone: viruses,…
Selective harm: DoS,… Spammer
Cyber-gangster:
phishing attacks,
trojan horses,…
Big brother
Greedy operator
Selfish mobile station

Why Game Theory for Security?
Reference and Courtesy: Assane Gueye, NIST/ITL/CCTG, Gaithersberg, “Game theoretic modeling, analysis, and mitigation of security risks.”, 2011

Why Game Theory for Security?

Intruder Game
Encryption is not always practical ….
Formulation: Game between Intruder and User
What if it is possible that...

Intruder Game
Strategies (mixed i.e. randomized)
Trudy: (p0,p1), Bob: (q0,q1)
Payoffs:
Strategies can be formulated and Nash Equilibrium can be found

Mechanism Design Principles
A shop in Belgium

Ad-word Market – Google/Facebook
Melaka, Malaysia

Adwords Market
• Created by search engine companies
– Google
– Yahoo!
– MSN
• Multi-billion dollar market
• Totally revolutionized advertising, especially
by small companies.

Dr Ganesh Neelakanta Iyer 8312/16/2018

The Adwords Problem:
N advertisers;
– Daily Budgets B1, B2, …, BN
– Each advertiser provides bids for keywords he is interested in.
Search Engine

N advertisers;
Search Enginequeries
(online)

N advertisers;
Search Engine
Select one Ad
Advertiser
pays his bid
queries
(online)
Maximize total revenue

Example:
$1 $0.99
$1 $0
Book
CD
Bidder1 Bidder 2
B1 = B2 = $100
Queries: 100 Books then 100 CDs
Bidder 1 Bidder 2
Algorithm Greedy
LOST
Revenue
100$

Example:
$1 $0.99
$1 $0
Book
CD
Bidder1 Bidder 2
B1 = B2 = $100
Queries: 100 Books then 100 CDs
Bidder 1 Bidder 2
Optimal Allocation
Revenue
199$

Multiple Cloud
Orchestration
using Auctions
90
NYSE, New York

Cloud Orchestration
• Relates to the connectivity of IT and business process levels between
Cloud environments.
• As cloud emerges as a competitive sourcing strategy, a demand is
clearly arising for the integration of Cloud environments to create an end-
to-end managed landscape of cloud-based functions.
• Benefits include
– Helps users to choose the best service they are looking for (for example the
cheapest or the best email provider)
– Helps providers to offer better services and adapt to market conditions quickly
– Ability to create a best of breed service-based environment in which a change of
provider does not break the business process
Reference: Ganesh Neelakanta Iyer, Bharadwaj Veeravalli and Ramkumar Chandrasekaran, "Auction-based vs. Incentive-based Multiple-Cloud
Orchestration Mechanisms", IEEE International Conference on Communication, Networks and Satellite (COMNETSAT 2012), JULY 2012

Cloud Brokers
• Cloud Broker plays an intermediary role to help customers locate the
best and the most cost-effective CSP for the customer needs
• One stop solution for Multiple Cloud Orchestration (aggregating,
integrating, customizing and governing Cloud services for SMEs and
large enterprises)
• Advantages are cost savings, information availability and market
adaptation
• Some ways to implement :- Auctions, Incentives

Typical Cloud Broker
ecosystem showing the
players involved
The Broker helps to connect the
providers and users
http://www.optimis-project.eu/
Cloud Brokers

Auction Theory
• In economic theory, an auction may refer to any
mechanism or set of trading rules for exchange

Auction Theory: English Auction
• Open ascending price auction
• Protocol: Each bidder is free to raise his bid. When no bidder is willing to
raise, the auction ends, and the highest bidder wins the item at the price of his
bid
• Strategy: Series of bids as a function of agent’s private value, his prior
estimates of others’ valuations, and past bids
• Best strategy: In private value auctions, bidder’s dominant strategy is to
always bid a small amount more than current highest bid, and stop when his
private value price is reached

Auction Theory: Dutch Auction
• Open descending price auction
• Protocol: Auctioneer continuously lowers the price until a bidder takes the item at the current
price
• Strategy: Bid as a function of agent’s private value and his prior estimates of others’ valuations
• Best strategy: No dominant strategy in general
• Example: Suppose a business is auctioning off a used company car: the bidding may start at
$15,000. The bidders will wait as the price is successively reduced to $14,000, $13,000,
$12,000, $11,000 and $10,000. When the price reaches $10,000, Bidder A decides to accept that
price and, because he is the first bidder to do so, wins the auction and has to pay $10,000 for the
car.

Auction Theory: First Price
Auction
• Single round of bidding
• Protocol: Each bidder submits one bid without knowing
others’ bids. The highest bidder wins the item at the price
of his bid
• Strategy: Bid as a function of agent’s private value and
his prior estimates of others’ valuations
• Best strategy: No dominant strategy in general

Auction Theory: Vickery Auction
• Second price sealed bid
• Protocol: Each bidder submits one bid without knowing (!) others’ bids. Highest bidder wins
item at 2nd highest price
• Strategy: Bid as a function of agent’s private value and his prior estimates of others’ valuations
• Best strategy: In a private value auction with risk neutral bidders, Vickrey is strategically
equivalent to English. In such settings, dominant strategy is to bid one’s true valuation
• Example: Google AdWords. Let's say there is an auction of 3 people: Joe bids $10, George bids
$20, and Bill bids $30. In this case, Bill wins the bid but pays only $20.
• Benefit: Reduces the likelihood that a bidder will overpay for an item. It also increases the
likelihood that the seller will get the most he can get for his item.

Auction Theory: Call Market Auction
• Protocol: Buyers enter competitive bidders and sellers enter
competitive offers simultaneously. Match is made when the buyer and
seller call out the same price
• Features: Usually features large number of buyers and sellers
• Example: trade negotiations such as NYSE, BSE, AMEX
http://www.thehindu.com/business/markets/article447353.ece http://blogs.reuters.com/moneyonthemarkets/tag/bse/page/2/

Auction Theory: Continuous Double
Auction
• A mechanism to match buyers and sellers of a particular good, and
to determine the prices at which trades are executed.
• At any point in time, traders can place limit orders in the form of bids
(buy orders) and asks (sell orders).
• Buyers and sellers can not modify their bids

Sealed-bid Continuous Double
Auctions
Comparison of revenue
• Hit Ratio is the ratio of the number
of successful auctions to the total
number of auctions.
• Fair revenue for all users
• Lowers user expenditure at the
expense of response-time for
choosing appropriate CSP.
Reference: Ganesh Neelakanta Iyer, Bharadwaj Veeravalli and Ramkumar Chandrasekaran, “Broker-agent based Cloud Service Arbitrage
Mechanisms using Sealed-bid Double Auctions and Incentives”, Journal of Network and Computer Applications (JNCA), Elsevier 2012

Cooperative Games
Kronborg, Denmark

Cooperative Games
Bargaining
Approaches
Coalition
Games

Resource Allocation in Cloud Computing Environments
Ulu Watu, Bali, Indonesia

Resource Allocation in Cloud
• Problem under consideration is “Resource Allocation and
Pricing Strategies for tasks in Compute Cloud Environments”.
• We employ “Axiomatic Bargaining Approaches to derive the
optimal solution for allocating resources in a Compute Cloud”.
• Nash Bargaining Solution (NBS) and Raiffa Bargaining Solution (RBS)
• Handling various parameters such as deadline, budget constraints etc
• Introduction of asymmetric pricing scheme for CSPs
• Handling auto-elasticity, fairness
Reference: Ganesh Neelakanta Iyer and Bharadwaj Veeravalli, “On the Resource Allocation and Pricing Strategies in Compute Clouds Using
Bargaining Approaches”, IEEE International Conference on Networks (ICON 2011), Singapore, December 2011.

Suitable for both independent tasks, Bag-of-Tasks (BoT) and tasks from workflow schemes
Assumption: Tasks are known apriori, but it can handle real-time arrival of tasks

Axiomatic Bargaining Approaches
• Good to derive fair and Pareto-optimal solution
• Pareto optimal: It is impossible to increase the allocation of a
connection without strictly decreasing another one.
• It assumes some desirable and fair properties, defined using axioms,
about the outcome of the resource bargaining process.
• Two approaches:
– Nash Bargaining Solution (NBS)
– Raiffa-Kalai-Smorodinsky Bargaining Solution (RBS)

Nash Bargaining Solution (NBS)
Solving, we obtain

Raiffa-Kalai-Smorodinsky Bargaining Solution (RBS)
Solving, we obtain

Performance evaluation:
Deadline based Real-time task arrival

1
1
1
Pricing Analysis
Symmetric:
price/resource = $0.75
Asymmetric:
A value in [0.5,1.0]
Tasks specify maximum budget
Current CSPs follow symmetric pricing schemes (EC2, Azure)
Introducing asymmetric pricing approach, which would give adequate flexibility in
managing the resources as well as generating more revenue.

1
1
2
Observations:
• Allocation in NBS and RBS depends on bargaining power and is within the Pareto
boundary
• When NBS maximizes the product of the gain of all players, RBS in addition
considers how much other players gave up
NBS efficiently utilizes maximum number of resourcesRBS indirectly maps to an
energy efficient solution by meeting the deadline with less number of
resources.RBS effectively handles auto-elasticity and task dynamicsNBS is
shown to be suitable for shorter deadline tasks whereas RBS is for handling tasks
of longer deadline tasks. Asymmetric pricing scheme

1
1
3
Multimedia Resource Management
Singapore Zoological Gardens, Singapore
Photo courtesy: GIRISH N, My brother

1
1
4
• Another application for Nash Bargaining Solution
• Need for deploying resource demanding applications
such as multimedia streaming, video surveillance,
video gaming etc over bandwidth-constrained
infrastructure
• Players: N video users
• Each player i seeks to obtain a share of the
bandwidth xi
• Utility: ui(xi), S is the N-dimensional bargaining region
• Disagreement point d is the minimum utility
• Bargaining problem (s,d)
Reference: Zhu Han, Dusit Niyato, Walid Saad, Tamer Baar, Are Hjørungnes, “Game Theory in Wireless and Communication Networks, Theory,
Models, and Applications”, Cambridge University Press, 2012, Chapter 7

• x0 is the bandwidth share at disagreement point
• D0 and µi are rate distortion parameters which depends on video
sequence characteristics, resolutions and delay.
• The bargaining problem is:
Using an adequate distortion rate model, utility function is
• 𝛂i is the bargaining power.
•Outcome of this problem would determine the resource allocation that would treat users fairly and
provide Pareto-optimal utilities for the video users
•Choice of bargaining power allows to prioritize users based on video/communication channel
characteristics

Revenue Maximization
in Mobile Clouds
From my home, Thodupuzha, Kerala

Simple examples

Mathematical Model of Coalition Game

An Example
• Players = {1,2,3}
• All nonempty subset (named as coalition) {1}, {2}, {3}, {1,2}, {1,3},
{2,3}, {1,2,3}
• A cost function c related to all coalitions. c({1}) = v1, c({2}) = v2, ...,
c({1,2,3}) = v7
• c(S) is the amount that the players in the coalition S have to pay
collectively in order to have access to a service.

Properties

Horse Market Example

Mobile Cloud Environments
• Mobile cloud computing combines wireless access service and
cloud computing to improve the performance of mobile
applications.
• Mobile applications can offload some computing modules (such as
online gaming) to be executed on a powerful server in a cloud.
• A scenario where multiple CSPs cooperatively offer mobile
services to users.
Reference: Dusit Niyato, Ping Wang, Ekram Hossain, Walid Saad, and Zhu Han, “Game Theoretic Modeling of Cooperation amongService Providers in Mobile Cloud Computing
Environments”, IEEE Wireless Communications and Networking Conference, 2012

Mobile Clouds and Coalition Game
•Mobile applications are supported by the mobile CSPs in which the radio (bandwidth) and computing
(servers) resources are reserved for the users.
•To improve resource utilization and revenue, mobile CSPs cooperate to form a coalition and create a
resource pool for users running mobile applications.
•Revenue sharing among the CSPs is based on a coalitional game.
•With a coalition, providers can optimize the capacity expansion, which determines the reserved bandwidth
and servers for a resource pool.
•The objective of provider is to maximize the profit from supporting mobile applications through a resource
pool.
Reference: Dusit Niyato, Ping Wang, Ekram Hossain, Walid Saad, and Zhu Han, “Game Theoretic Modeling of Cooperation amongService Providers in Mobile Cloud
Computing Environments”, IEEE Wireless Communications and Networking Conference, 2012

1
2
5
Mobile Cloud Environments
• Mobile cloud computing combines wireless access service and cloud
computing to improve the performance of mobile applications.
• Mobile applications can offload some computing modules (such as
online gaming) to be executed on a powerful server in a cloud.
• A scenario where multiple CSPs cooperatively offer mobile services
to users.
• Coalition games
Reference: Dusit Niyato, Ping Wang, Ekram Hossain, Walid Saad, and Zhu Han, “Game Theoretic Modeling of Cooperation amongService Providers
in Mobile Cloud Computing Environments”, IEEE Wireless Communications and Networking Conference, 2012

Coalition Game: An example
• Players = {1,2,3}
• All nonempty subset (named as coalition) {1}, {2}, {3}, {1,2}, {1,3}, {2,3},
{1,2,3}
• A cost function c related to all coalitions. c({1}) = v1, c({2}) = v2, ...,
c({1,2,3}) = v7
• c(S) is the amount that the players in the coalition S have to pay
collectively in order to have access to a service.

Coalition Game: Core
•The problem is to find the core of this coalition game.
•Core is a cost distribution of the grand coalition such that no other coalition can obtain
an outcome better for all its members than the current assignment.
•There may not exist any core.
•Emptiness of the core.
•There may exist many cores.
•Some players would unhappy with the cost allocation.

Coalition Game: Example
• We want to find the cost allocation {x1, x2, x3} such that
• x1+x2+x3 = c({1,2,3})
• x1 ≦ c({1})
• x2 ≦ c({2})
• x3 ≦ c({3})
• x1+x2 ≦ c({1, 2})
• x1+x3 ≦ c({1, 3})
• x2+x3 ≦ c({2, 3})
• Given a solution in the core, there is no incentive for a player to leave the grand coalition.

Shapely Value
• A simple system of four conditions or axioms which offers the unbiased
judge an opportunity to decide how to divide v(N) fairly among the players in
any given game. We shall see that these axioms determine a unique way to
divide v(N) in every game.
• The axioms were first formulated in 1953 by Lloyd Shapley, who showed
that indeed they dictate to the judge how to decide in every case. The
division of payoffs according to this decision is called the Shapley value.

Shapely Value: Definition

Remarks

Some applications of
Shapely Value in Computer
Science Domain
Ship crossing between Sweden & Denmark

1. Content Delivery Networks
a. Live streaming, video-on-demand. Use peer-to-peer architecture to
reduce their operating cost
b. Users opt-in to dedicate part of the resources they own to help the
content delivery, in exchange for receiving the same service at a
reduced price
c. Shapely Value: This ensures that each player, be it the provider or a
peer, receives an amount proportional to its contribution and
bargaining power when entering the game
http://paloalto.thlab.net/uploads/papers/fp200-misra.pdf
http://conferences.sigcomm.org/co-next/2012/eproceedings/conext/p133.pdf

2. Profit Sharing in Wireless Networks:
a. Several service providers offer wireless access service to their
respective subscribed customers through potentially multi-hop routes
b. If providers cooperate, i.e., pool their resources, such as spectrum
and base stations, and agree to serve each others’ customers, their
aggregate payoffs, and individual shares, can potentially substantially
increase through efficient utilization of resources
http://repository.upenn.edu/cgi/viewcontent.cgi?article=1558&context=ese_papers

3. Web Services Cooperation
a. WEB services (WSs) are a kind of software built upon a set of widely
accepted speciﬁcations.
b. Existing WSs can cooperate with each other to provide more valuable
WSs.
c. On one hand, WSs with different functionality can work togother to
generate WSs with new functionality.
d. For example, a WS translating Chinese into English can cooperate
with another WS translating English into German to create a new WS
that can translate Chinese into German
http://staff.ustc.edu.cn/~anliu/papers/TSC-12-Liu.pdf

More Applications…
• Distributed caching via rewarding in P2P-VoD systems
• Multi-cast routing
• Distributed Databases
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6490320&isnumber=4359390
http://www.cs.ubc.ca/~kevinlb/teaching/cs532l%20-%202007-8/projects/report532l_jonatan.pdf
http://link.springer.com/chapter/10.1007%2F978-81-322-0740-5_4

Smart Grid Systems
Kek Lok Si Temple, Penang, Malaysia

Traditional Grid vs Smart Grid
https://alumni.iit.edu/document.doc?id=83

The role of information sciences

Non-cooperative Games

Cooperative Games

Congestion Games,
Potential Games
Phang Nga Bay, Thailand

Games with many agents
• How do we represent a (say, simultaneous-move) game with n agents?
• Even with only 2 actions (pure strategies) per player, there are 2n possible outcomes
– Impractical to list them all
• Real-world games often have structure that allows us to describe them concisely
• E.g., complete symmetry among players
– How would we represent such a game?
– How many numbers (utilities) do we need to specify?
• What other structure can we make use of?

• In the Multicast Routing problem, we are given a graph G=(V,E).
• Each agent i must buy edges connecting si∊V to ti∊V.
• The cost of an edge e∊E is distributed equally between the agents that
bought it ⇒ The more agents buy e, the cheaper it is for each agent.
• The cost of path P:si→ti is ∑e∊P(Ce / Ne) where Ce is the cost of e and Ne is the
number of agents that bought e.
• The goal is to minimize the cost.
Example 1: Multicast Routing

s2
1
t2
s1
t1
2
6
4
13
7
agent 1 cost:
agent 2 cost:
7
6

s2
1
t2
s1
t1
2
6
4
13
7
agent 1 cost:
agent 2 cost:
7
2+4+1=7

s2
1
t2
s1
t1
2
6
4
13
7
agent 1 cost:
agent 2 cost:
1+4/2+3=6
2+4/2+1=5

• In the Traffic problem, we are given a graph G=(V,E).
• Each agent i must choose a path connecting si∊V to ti∊V.
• The delay on an edge e∊E is proportional to the number of agents using this
edge ⇒ The more agents use e, the higher the delay for each agent.
• The cost for using path P:si→ti is ∑e∊P(Ce * Ne) where Ce is the cost of e and Ne
is the number of agents that use e.
• The goal is to minimize the cost (delay).
Example 2: Traffic

s2
1
t2
s1
t1
2
62
13
7
agent 1 cost
agent 2 cost
7
6
Example 2: Traffic

s2
1
t2
s1
t1
2
62
13
7
agent 1 cost
agent 2 cost
7
2+2+1=5
Example 2: Traffic

s2
1
t2
s1
t1
2
62
13
7
agent 1 cost
agent 2 cost
1+2*2+3=8
2+2*2+1=7
Example 2: Traffic

(General) Congestion Games
• resources
– roads 1,2,3,4
• players
– driver A, driver B
• strategies: which roads I use for
reach my destination?
– A wants to go in Salerno
– e.g. SA={{1,2},{3,4}}
– B wants to go in Napoli
– e.g. SB={{1,4},{2,3}}
• what about the payoffs?
Roma
Salerno
Milano
Napoli
road 1
road 2
road 3
road 4
A B

Payoffs in (G)CG: an example
• A choose path 1,2
• B choose path 1,4
• uA = - (c1(2) + c2(1)) = - 4
• uB = - (c1(2) + c4(1)) = - 5
Roma
Salerno
Milano
Napoli
road 1
road 2
road 3
road 4
A B
c1(1)=2 c1(2)= 3
c2(1)=1 c2(2)= 4
c3(1)=4 c3(2)= 6
c4(1)=2 c4(2)= 5
Costs for the roads
SIRF (Small Index Road First)
(-4,-5) (-6,-8)
(-9,-7) (-8,-7)
{1,2}
{3,4}
{1,4} {2,3}
B
A

Other Applications
• Multi-agent Systems
– Resource sharing among un-coordinated selfish agents
• Distributed Systems
– Load balancing among unreliable resources
• CDMA systems
– Interference Avoidance
• Cognitive Radio networks
– Power allocation for secondary users

Cyber-Physical Systems Robustness
Painting lighthouse,
Martha’s vineyards, USA

Attack and defense in cyber-physical
systems
• Cyber physical systems :- Systems which need cyber and physical components to function.
• Examples: Cloud Computing systems, Sensor network systems, Communication networks
• Players: Defenders aim to keep the system functioning and the attacker aims to disrupt.
• Actions represent the resources deployed by the defender and disrupted by the attacker, respectively.
• Costs and benefits: Each player has a payoff function U consisting of two parts: benefit B and/or cost C.
The attacker incurs a cost in launching an attack, and the defender incurs a cost in deploying the
resources. In a game, either player will aim to maximize its payoff given the other player's best strategy.
• Existence and solutions of pure and mixed-strategy Nash Equilibria can be found
Reference: Chris Y. T. Ma, Nageswara S. V. Rao and David K. Y. Yau, “A Game Theoretic Study of Attack and Defense in Cyber-Physical Systems”, The
First IEEE International Workshop on Cyber-Physical Networking Systems, 2011

Multiuser Wireless
Multimedia Transmission
Salad, Bali, Indonesia

Multiuser Wireless Multimedia Transmission
• Static multimedia resource allocation does not exploit network resources efficiently
• Channel conditions, video characteristics, number of users, users desired utilities etc
varies with time
• It does not provide adequate QoS support when network is congested
• Users can untruthfully declare their resource requirements to obtain a longer
transmission time
• Solution: Non-collaborative resource management game
Reference: Mihaela Van Der Schmar and Philip A Chou, “Multimedia over IP and Wireless Networks”, Academic Press 2007, Chapter 12, Section 12.9.4

Multiuser Wireless Multimedia Transmission

Multiuser Wireless Multimedia
Transmission
• Modulator (CSM):
• Social decision: After receiving the
requests from all users, resource
allocation is made in order to maximize
the system utility
• Transfer Computation: Computes the transfers associated with the allocation to
enforce users to reveal their real type truthfully
• Polling users: Polls users for packet transmission according to the allocated time

Transmission
• Users
• Private Information Estimation: Each user
estimates expected video source
characteristics and channel conditions
• Selection of optimal joint strategy: It is determined by selecting the optimal cross layer
strategy which maximizes the expected received video quality without considering the
impact on other users. Based on this optimal revealing strategy is determined. It is
revealed to the CSM
• Transmit video packets: When polled by CSM, optimal real-time cross-layer strategy is
determined.

1
6
2
Transmission
To summarize:
•To play dynamic resource management game:
•Users deploy three different types of strategies at different stages
of the game
•Optimal expected cross-layer strategies and revealing strategies
(before transmission)
•Optimal real-time cross-layer strategy (during actual transmission)
•Thus users play competitive dynamic resource management game

Evolutionary Games
Yes…. The North Pole… View from airlines

Non-cooperative vs Evolutionary Games
• Regular game theory
– Individual players make decisions
– Payoffs depend on decisions made by all
– The reasoning about what other players might do happens
simultaneously
• Evolutionary game theory
– Game theory continues to apply even if no individual is overtly
reasoning or making explicit decisions
– Decisions may thus not be conscious
– What behavior will persist in a population?

Background
• Evolutionary biology
– The idea that an organism‘s genes largely determine its
observable characteristics (fitness) in a given environment
• More fit organisms will produce more offspring
• This causes genes that provide greater fitness to increase their
representation in the population
– Natural selection

Overview of Evolutional Game
• Evolutionary game theory has been developed as a mathematical framework
to study the interaction among rational biological agents in a population
• Agent adapts (i.e., evolves) the chosen strategy based on its fitness (i.e.,
payoff)
• Example, hawk (be aggressive) and dove (be mild)

Overview of Evolutional Game
• Evolutionary game theory has the following advantages over the
traditional noncooperative game theory
– The solution of the evolutionary game (i.e., evolutionary stable strategies
(ESS) or evolutionary equilibrium) can serve as a refinement to the Nash
equilibrium (e.g., Nash equilibrium is not necessarily efficient, there could
be multiple Nash equilibria in a game, or the Nash equilibrium may not
exist)
– The strong rationality assumption is not required in evolutionary game
as evolutionary game theory has been developed to model the behavior of
biological agents
– Evolutionary game is based on an evolutionary process, which is
dynamic in nature which can model and capture the adaptation of agents
to change their strategies and reach equilibrium over time

Evolution Process
• In an evolutionary game, the game is played repeatedly by agents who are
selected from a large population
• Two major mechanisms of the evolutionary process and the evolutionary
game are mutation and selection
– Mutation is a mechanism of modifying the characteristics of an agent (e.g., genes
of the individual or strategy of player), and agents with new characteristics are
introduced into the population
– The selection mechanism is then applied to retain the agents with high fitness
while eliminating agents with low fitness
• In evolutionary game, mutation is described by the evolutionary stable
strategies (ESS) from static system perspective
• Selection mechanism is described by the replicator dynamics from dynamic
system perspective

Evolutionary Stable Strategies (ESS)
• ESS is the key concept in the evolutionary process in which a group
of agents choosing one strategy will not be replaced by other
agents choosing a different strategy when the mutation mechanism
is applied
• Initial group of agents in a population chooses incumbent strategy s
• Small group of agents whose population share is ε choosing a
different mutant strategy s’
• Strategy s is called evolutionary stable if
where u(s, s’) denote the payoff of strategy s given that the opponent
chooses strategy s’

Evolutionary game theory
• Key insight
– Many behaviors involve the interaction of multiple organisms in a
population
– The success of an organism depends on how its behavior interacts
with that of others
• Can‘t measure fitness of an individual organism
– So fitness must be evaluated in the context of the full population in
which it lives
• Analogous to non-cooperative game theory!
– Organisms‘s genetically determined characteristics and behavior =
Strategy
– Fitness = Payoff
– Payoff depends on strategies of organisms with which it interacts =
Game matrix

Motivating example
• Let‘s look at a species of a beetle
– Each beetle‘s fitness depends on finding and processing
food effectively
– Mutation introduced
• Beetles with mutation have larger body size
• Large beetles need more food
• What would we expect to happen?
– Large beetles need more food
– This makes them less fit for the environment
– The mutation will thus die out over time
• But there is more to the story...

Motivating example
• Beetles compete with each other for food
– Large beetles more effective at claiming above-average share
of the food
• Assume food competition is among pairs
– Same sized beetles get equal shares of food
– A large beetle gets the majority of food from a smaller beetle
– Large beetles always experience less fitness benefit from given
quantity of food
• Need to maintain their expensive metabolism

Motivating example
• The body-size game between two beetles
• Something funny about this
– No beetle is asking itself: “Do I want to be small or
large?“
• Need to think about strategy changes that operate
over longer time scales
– Taking place as shifts in population under evolutionary
forces
Small Large
Small 5, 5 1, 8
Large 8, 1 3, 3

Evolutionary stable strategies
• The concept of a Nash equilibrium doesn‘t work in this
setting
– Nobody is changing their personal strategy
• Instead, we want an evolutionary stable strategy
– A genetically determined strategy that tends to persist once it is
prevalent in a population
• Need to make this precise...

Evolutionarily stable strategies
• Suppose each beetle is repeatedly paired off with other beetles at
random
– Population large enough so that there are no repeated interactions
between two beetles
• A beetle‘s fitness = average fitness from food interactions =
reproductive success
– More food thus means more offspring to carry genes (strategy) to the next
generation
• Def:
– A strategy is evolutionarily stable if everyone uses it, and any small group
of invaders with a different strategy will die off over multiple generations

Evolutionarily stable strategies
• Def: More formally
– Fitness of an organism in a population = expected payoff from
interaction with another member of population
– Strategy T invades a strategy S at level x (for small x) if:
• x fraction of population uses T
• 1-x fractionof population uses S
– Strategy S is evolutionarily stable if there is some number y such
that:
• When any other strategy T invades S at any level x < y, the fitness of an
organism playing S is strictly greater than the fitness of an organism playing T

• Is Small an evolutionarily stable strategy?
• Let‘s use the definition
– Suppose for some small number x, a 1-x fraction of
population use Small and x use Large
• What is the expected payoff to a Small beetle in a
random interaction?
– With prob. 1-x, meet another Small beetle for a payoff of 5
– With prob. x, meet Large beetle for a payoff of 1
– Expected payoff: 5(1-x) + 1x = 5-4x
Motivating example

• Is Small an evolutionarily stable strategy?
• Let‘s use the definition
– Suppose for some small number x, a 1-x fraction of
population use Small and x use Large
• What is the expected payoff to a Large beetle in a
random interaction?
– With prob. 1-x, meet a Small beetle for payoff of 8
– With prob. x, meet another Large beetle for a payoff of 3
– Expected payoff: 8(1-x) + 3x = 8-5x
Motivating example

Motivating example
• Expected fitness of Large beetles is 8-5x
• Expected fitness of Small beetles is 5-4x
– For small enough x (and even big x), the fitness of Large
beetles exceeds the fitness for Small
– Thus Small is not evolutionarily stable
• What about the Large strategy?
– Assume x fraction are Small, rest Large.
– Expected payoff to Large: 3(1-x) + 8x = 3+5x
– Expected payoff to Small: 1(1-x) + 5x = 1+4x
– Large is evolutionarily stable

Motivating example
• Summary
– A few large beetles introduced into a population consisting of
small beetles
– Large beetles will do really well:
• They rarely meet each other
• They get most of the food in most competitions
– Population of small beetles cannot drive out the large ones
• So Small is not evolutionarily stable

Motivating example
• Summary
– Conversely, a few small beetles will do very badly
• They will lose almost every competition for food
– A population of large beetles resists the invasion of small
beetles
– Large is thus evolutionarily stable

Relationship with Nash Equilibrium
• Nash equilibrium
– Rational players choosing mutual best responses to each
other‘s strategy
– Great demands on the ability to choose optimally and
coordinate on strategies that are best responses to each
other
• Evolutionarily stable strategies
– No intelligence or coordination
– Strategies hard-wired into players (genes)
– Successful strategies produce more offspring
• Yet somehow they are almost the same!

Replicator Dynamics
• Population can be divided into multiple groups, and each
group adopts a different pure strategy
• Replicator dynamics can model the evolution of the group
size over time (unlike ESS, in replicator dynamics agents will
play only pure strategies)
• The proportion or fraction of agents using pure strategy s (i.e.,
population share) is denoted by xs(t) whose vector is x(t)
• Let payoff of an agent using strategy s given the population
state x be denoted by u(s, x)
• Average payoff of the population, which is the payoff of an
agent selected randomly from a population, is given by
12/16/2018 183

Replicator Dynamics
• The reproduction rate of each agent (i.e., the rate at which the
agent switches from one strategy to another) depends on the
payoff (agents will switch to strategy that leads to higher payoff)
• Group size of agents ensuring higher payoff will grow over time
because the agents having low payoff will switch their
strategies
• Dynamics (time derivative) of the population share can be
expressed as follows:
• Evolutionary equilibrium can be determined at
where actions of the population choosing different strategies
cease to change

Replicator Dynamics
• It is important to analyze the stability of the replicator dynamics
to determine the evolutionary equilibrium
• Evolutionary equilibrium can be stable (i.e., equilibrium is
robust to the local perturbation) in the following two cases:
– 1) Given the initial point of replicator dynamics sufficiently close to the
evolutionary equilibrium, the solution path of replicator dynamics will
remain arbitrarily close to the equilibrium (Lyapunov stability)
– 2) Given the initial point of replicator dynamics close to the
evolutionary equilibrium, the solution path of replicator dynamics
converges to the equilibrium (asymptotic stability)
• Two main approaches to prove the stability of evolutionary
equilibrium are based on the Lyapunov function and the
eigenvalue of the corresponding matrix

Example: Prisoner's Dilemma
• Two agents choose a strategy of cooperate or defect
where T > R > P > S
• xC and xD denote the proportions of the population adopting cooperate and
defect strategies, respectively
• Average fitness of agents adopting these two strategies are denoted by uC
and uD, respectively
• Average fitness of the entire population is obtained from
Change in fitness

• The future proportion of the population adopting the strategies
depends on the current proportion
• Consider small time interval, the differential equations (replicator
dynamics) are
Cooperate Defect

• For the prisoner's dilemma case, we have uC = u0 + xCR + xDS and uD = u0 +
xCT + xDP
• Since T > R and P > S, it is clear that uD > uC, and
• Therefore, as time increases, the proportion of the population adopting the
cooperate strategy will approach zero (i.e., becomes extinct)
• From replicator dynamics, defect strategy constitutes the evolutionary
equilibrium
• Also, it can be proven that defect strategy is the ESS of the prisoner's
dilemma game

Applications of Evolutionary Game
Congestion control
• The competition among two types of behaviors (i.e., aggressive and
peaceful) in wireless nodes to access the channel using a certain
protocol can be modeled as an evolutionary game
• Congestion control is (transport layer) to avoid performance
degradation by the ongoing users by limiting transmission rate
• The transmission rate (i.e., of TCP) can be adjusted by changing the
congestion window size (i.e., the maximum number of packets to be
transmitted)
• The speed-of-transmission rate to be increased and decreased
defines the aggressiveness of the protocol

Congestion control
• TCP protocol with the additive increase multiplicative
decrease (AIMD) mechanism can control this
aggressiveness through the parameters determining the
increase and decrease
• If the transmitted packet is successful, the window size
will linearly increase by α packets for every round trip time
• Otherwise, the window size will decrease by β
proportional to the current size

Congestion control
• Multiple flows share the same link, competitive situation
arises
• It is found that the aggressive strategy of all flows (i.e.,
large values of α and β) becomes the Nash equilibrium,
and the performance will degrade significantly due to the
congestion
Senders Receivers
Shared link

Congestion control – Static game
• Analysis of the TCP protocol in a wireless environment is performed in which
the evolutionary game model (similar to the Hawk and Dove game)
• There are two populations (i.e., groups) of flows with TCP
• The flow from population i is characterized by parameters αi and βi, which are
the increase and decrease rates, respectively
• Strategy s of flow is to be aggressive (i.e., hawk or H) to be peaceful (i.e.,
dove or D)
• The parameters associated with these strategies are given as

• The packet loss occurs when the total transmission rate of all flows reaches
the capacity C- i.e., x1r1 +x2r2 = C, where xi is the proportion of population
choosing aggressive behavior
• The payoff of flow in population i is defined as follows:
where τi is the average throughput, L is the loss rate, and ω is the weight for
the loss
• Throughput of flow from population i can be obtained from

• The average throughput and loss rate can be defined as functions of
strategies of two populations i.e., τi(si, sj) and L(si, sj)
• It is shown that τi(H, H) = τi(D, D)
• When the loss rate is considered, it increases as the flow becomes more
aggressive, i.e., larger values of αi and βi
• Therefore, it can be shown that ui(H, H) < ui(D, D) and ui(D, H) < ui(D, D)
• Game becomes a Hawk and Dove model whose solution is ESS
• Briefly, it is found that the application that is loss-sensitive will tend to use a
less aggressive strategy at ESS

Congestion control – Dynamic game
• Dynamics of strategy selection by the flows in two populations can also be
analyzed using the replicator dynamics
xs is the proportion of the population choosing strategy s and xs(t) is a vector
of xs at time t; u(s, x(t)) is the payoff of using strategy s, and K is a speed
constant (positive)

Evolutionary Games in Cloud/Edge/Fog Computing
Cape Cod, USA
Reference: Dr. Ganesh Neelakanta Iyer, “Evolutionary Games for Cloud, Edge and Fog Computing – A Comprehensive
Study”, in 5th International Conference on Computational Intelligence in Data Mining (ICCIDM-2018), Odisha, India , 2018.

Cloud/Edge/Fog Computing

Cloud Selection
• In [5], they study price competition in a heterogeneous
market cloud formed by CSPs, brokers and users
• Initially the competition among CSPs in selling the service
opportunities has been modelled using Non-cooperative
games where CSPs tries to maximize their revenues
Evolutionary game has been used to study dynamic
behaviour of cloud users to select a CSP based on
different factors such as price and delay
[5] C. T. Do et.al., Toward service selection game in a heterogeneous market cloud computing. IFIP/IEEE
International Symposium on Integrated Network Management (IM), 2015

Cloud Selection
• In [6], the same authors extend the same work to consider
the presence of multiple cloud providers at the same time
• They have used Wardrop equilibrium [25] concepts and
replicator dynamics in order to calculate the equilibrium
and for choosing the cloud service, they characterized its
convergence properties.
[6] Cuong T. Do, Nguyen H. Tran, Eui-Nam Huh, Choong Seon Hong, Dusit Niyato, Zhu Han, Dynamics
of service selection and provider pricing game in heterogeneous cloud market, Elsvier, 2016, Journal of
Network and Computer Applications, , Vol. 69, pp.

Deployment in cloud environments
• In [7] for VM deployment under objectives such as energy
efficiency, budget and deadline, evolutionary games are used
• The works in [8], [9], [10] uses evolutionary game theory to
deploy a set of appli-cations in a set of hosts based on certain
performance objectives such as CPU and bandwidth
availability, response time, power consumption etc
201
7. K. Han, X. Cai and H. Rong. Hangzhou An Evolutionary Game Theoretic Approach for Efficient Virtual Machine Deployment in Green
Cloud.: IEEE, 2015. 2015 International Conference on Computer Science and Mechanical Automation (CSMA). pp. 1-4.
8. Y. Ren, J. Suzuki, A. Vasilakos, S. Omura and K. Oba. Cielo: An Evolutionary Game Theoretic Framework for Virtual Machine Placement in
Clouds. IEEE, 2014. 2014 Inter-national Conference on Future Internet of Things and Cloud, Barcelona, pp. 1-8. doi: 10.1109/FiCloud.2014.11
9. Yi Ren, Junichi S, Chonho Lee, Athanasios V. V, Shingo O, and Katsuya Oba Balancing performance, resource efficiency and energy
efficiency for virtual machine deployment in DVFS-enabled clouds: an evolutionary game theoretic approach. In Proceedings of Con-ference on
Genetic and Evolutionary Computation (GECCO Comp '14). pp. 1205-1212
10 Cheng, Yi Ren. Evolutionary Game theoretic multi-objective optimizaion algorithms and their applications. Computer Science, University of
Massachusetts Boston, 2017

Security issues in Fog computing
• When computation is performed at the fog nodes, they are
more susceptible to security vulnerabilities due to their
diverse and distributed nature
• In [12], the authors analyse the security issues in such an
environment using evolutionary games
• Replicator dynamics are used to understand the behavioural
strategy selection
• They show that when normal nodes show cooperative
strategy, the malicious nodes are forced to show “non-attack”
strategy
202
12. Yan Sun, Fuhong Lin, Nan Zhang,A security mechanism based on evolutionary game in fog computing, 2,
Saudi Arabia : ScienceDirect, 2018, Saudi Journal of Biological Sciences, , Vol. 25, pp. 237-241. ISSN 1319-562X

Optimal sensor configuration in Edge
computing
• The work in [13] uses game theory for configuring Body
Sensor Networks (BSNs) to be used with Cloud based on
operational conditions which depends on different constraints
such as resource consumption and data yield
• Their concept is based on a layered architecture where cloud
provider has virtual sensors and physical sensors are
operated through their cloud-based virtual counterparts
• They primarily use evolutionary games to study fine-tuning
sensing intervals and selection rates for sensors
203
Y. C. Ren et.al. An Evolutionary Game Theoretic Approach for Configuring Cloud-Integrated Body Sensor Networks
Cambridge, MA 2014 IEEE 13th International Symposium on Network Computing and Applications. pp. 277-281

Evolutionary games for Cloud/Edge/Fog
ComputingWork Type of
issue Basic Concept System Objectives Limitations
[5] Cloud
selection
Price selection in heterogeneous market cloud Cloud Price, delay Study is mostly on a
duopoly setup
[6] Cloud
selection
Price selection in the presence of multiple cloud
providers
Cloud Price, delay Multiple service delivery
models, SLA
agreements and
operations costs need to
be considered
[7] VM
deployme
nt
Optimal VM deployment based on several performance
objectives
Cloud Energy efficiency, budget,
deadline
[8],
[9],
[10]
Applicatio
n
deployme
nt
Deploy a set of applications in a set of hosts based on
certain performance objectives. They study adaptability
and stability
Cloud CPU and bandwidth
availability, response time,
power consumption
[11] Applicatio
n
deployme
nt
Help applications to choose their locations and in
allocating resources based on different characteristics
Cloud Response time Several other objectives
are important such as
energy efficiency and
price
[12] Security Security issues in fog computing environments. Fog Consumption cost, profit
from attacks
Lack real performance
studies
[13] Sensor
configurati
on
Configuring BSNs to be used with Cloud based on
operational conditions with respect to different
constraints
Edge, Cloud Resource consumption,
data yield
[14] Computati
onal off-
loading
Computational offloading for mobile edge computing Edge, Cloud Time, energy
consumption, monetary
cost
[15],
[16]
Resource
allocation
QoS constrained cloud resource allocation Cloud Budget, computation time
[17] Resource Allocating resources within the cloud mobile social Cloud Price, Processing rate
204

Other Applications
1. Information security problem in the mobile electronic
commerce chain
2. Trust and reputation in e-commerce systems
3. Cognitive Radio Networks
1. W. Sun, X. Kong, D. He, and X. You. Information security problem research based on game theory.
International Symposium on Publication Electronic Commerce and Security, 2008
2. Lik Mui, Mojdeh Mohtashemi, Ari Halberstadt, A Computational Model of Trust and Reputation,
System Sciences, 2002. HICSS. Proceedings of the 35th Annual Hawaii International Conference on
3. Beibei Wang et. al., Game theory for cognitive radio networks: An overview, Computer Networks
12/16/2018

Summary...
Water-puppetry, Vietnam

To Summarize...
• Various Concept in Game Theory can be sued almost everywhere to handle
conflicting situations and for cooperation enforcement
• Topics not covered (much more than what is discussed)
– Bayesian games
– Combinatorial auctions
– Differential games
– Signaling Games
– Markovian Games
– …

WS01: Containerization For Next Generation
Application Development And Deployment
INDICON 2018 Workshop at ABII, Lab 3 from 2 PM to 5 PM
Amrita Vishwa Vidyapeetham, Coimbatore
Associate Professor, Dept of Computer Science and Engg

My lunch at Martha’s vineyards, USA

ni_amrita@cb.amrita.edu
ganesh.vigneswara@gmail.com
GANESHNIYER

Game Theory and Engineering Applications

Recommended

Recommended

More Related Content

Similar to Game Theory and Engineering Applications

Similar to Game Theory and Engineering Applications (20)

More from Dr Ganesh Iyer

More from Dr Ganesh Iyer (20)

Recently uploaded

Recently uploaded (20)

Game Theory and Engineering Applications