Games We Play
Defenses and Disincentives
Allison Miller
Overview
Overview of econ & game theory concepts

Game theory games

Infosec issues as games

Designing games to win

Walk-through a defense built on
disincentives

Wrap-up
Economics applied to security
Utility theory

Externalities 

Information Asymmetries

Signaling 

Marginal cost
Game theory
Branch of applied mathematics

Studies decisions made by players
interacting (or competing)

- Scenarios have rules and pay-offs

- Costs & benefits dependent on decisions of other
players

Used as a framework in economics, comp
sci, biology, & philosophy

- Also business, negotiation, and military strategy
Discussing Games
Mechanics of a payoff matrix
Player 2
A B
Player 1
A A1, A2 A1, B2
B B1, A2 B1, B2
Discussing Games
Mechanics of decision trees
UP
DOWN
CIRCLE
RED
BLUE
MARIO
LUIGI
KIRBY
GIZMO
10, 3
2, 10
2, 5
-3, 3
A
B
B
A
A
A
Typical game theory "games"
Chicken / Brinkmanship

- Push it to the edge

Volunteer’s Dilemma

- For the greater good

Tragedy of the Commons

- Share and share alike (cumulative effect of
cheating)

Prisoner’s Dilemma
Discussing Games
Prisoner’s Dilemma
Player 2
Keep quiet Confess
Player 1
Keep
quiet
-1, -1

Mutual cooperation
0, -10

Individual defection
Confess -10, 0

Individual defection
-3, -3

Mutual punishment
Predicting outcomes
Cooperation

Defection

Dominant
strategies

Equilibrium
Nash Equilibrium
Equilibrium is reached when:

- Players in a game have selected a strategy

- Neither side can change it’s strategy
independently & improve position

Optimal solution in games with limited
outcomes
Discussing Games
Prisoner’s Dilemma
Player 2
Keep quiet Confess
Player 1
Keep
quiet
-1, -1

Mutual cooperation
0, -10

Individual defection
Confess -10, 0

Individual defection
-3, -3

Mutual punishment
Setting up risk problems as games
Identify players in the game

Clarify the “rules”

Show me your moves

Describe payoffs

Single move or repeated game
Discussing Games
Tragedy of the Commons: Spam, Bandwidth usage
Everyone else’s choices
> n choose wise
usage
Less than n choose
wise usage
Individual
choice
Use
resource
wisely
Cost, but
social benefit

Mutual cooperation
Cost

(Subsidize social
use)
Overuse
resource
Social benefit

(Benefit w/o cost)
0

Resources depleted
Discussing Games
Chicken/Brinkmanship: Vulnerability Disclosure
Vulnerability Researcher
Report Exploit
Asset
Owner
Reward /
Respond
0, 0

Responsible
disclosure
-2, +2

Early disclosure
Ignore /
Deny
+2, -2

Defer vulnerability
-10, -10

0-day go boom
Discussing Games
Volunteer’s Dilemma: Data breach cost info sharing
All other victims
At least one
shares
All keep quiet
Victim
Share 0 0

Cost, limited benefit
Keep
quiet
1

Benefit w/o cost
-10

Everyone’s in the
dark
How games are won
Clarify dominant strategies

Find equilibrium

Pursue equilibrium or change the
payoffs
Moves
Current game-play

- Controls are layered or chained until we're satisfied that for some set of attackers,
the cost of the attack is higher than the utility associated with their payoff 

Reputation requirements for participation

Role requirements for participation (access control) 

Incremental authentication

Content/context based filtering

Blacklisting / whitelisting

Rate limiting 

Bot limiters (Captcha)

Obfuscation/Encryption
Counter-moves
For every move there is
a counter-move
Putting the pieces on the board
The amount of friction inserted into the
system depends on:

- Value of asset to the owner

- Value of the asset to potential attackers

- Number of attackers expected

- Portion of attacks that must be averted

- Disincentive value of each layer of friction for an
attacker

Now it’s time to play our game
Does this sound familiar?
Managing Decisions
Game Theory is a framework for studying decisions

- Since payoffs depends on the choices of other players, moves
are risky

- Players play based on their risk appetite

- Risk management = decision management

Defenders design control systems that make decisions

- Where risks manifest in observable behavior

- That make moves/counter-moves depending on the context
and understanding of an actor’s identity or intent

- Where system or individual costs/payoffs depend on the
outcome of an actor’s actions
SHALL WE PLAY A GAME?
(SINCE WE CAN’T PLAY “CLUE” FOR EVERY LOGIN
TRANSACTION
NEW USER
MESSAGE
FRIEND REQUEST
ATTACHMENT
PACKET
WINK
POKE
CLICK
WE BUILD RISK MODELS)
Applying Decisions
Risk management is
decision management
ACTOR
ATTEMPTS
ACTION
SUBMIT
WHAT IS THE
REQUEST
HOW TO
HONOR THE
REQUEST
SHOULD WE
HONOR?
RESULT
ACTION
OCCURS
Not all risk decisions have a
competitive element, but all
competition / games have risks
Create account using fake identity
Script completion of verifications
Outsource captcha
Create accounts across virtual
devices
Distribute creation of accts using
botnet
Scrape identities from public sites
Age accounts, then reactivate
Use stolen credentials
Defraud verification process
...
Require email verification
Test for human behind keyboard
Rate limit by device ID
Rate limit by IP/location
Look for similarities across
accounts
Require reputation level to
proceed
Filter for content / context, add
auth challenge
Require manual verification
Manual review of account/event
...
Except one small thing...
...what kind of game is this?
Multi-player Mode
Offense
Attempt Success
Defense
Deflect
4, 4
 0, 10

Ignore
10, 0
 1, 1
 Offense
Attempt Success
Defense
Deflect
4, 4
 0, 10

Ignore
10, 0
 1, 1
Offense
Attempt Success
Defense
Deflect
4, 4
 0, 10

Ignore
10, 0
 1, 1

Offense
Attempt Success
Defense
Deflect
4, 4
 0, 10

Ignore
10, 0
 1, 1

Attackers are not the only players in
the game

Legitimate users that are also
affected by added friction
Team Dynamics
So this adds another factor into the
appropriate level of friction question,
which is:

- Disincentive value of each layer of friction for
an innocent

- Likelihood the disincentive will be incorrectly
applied to an innocent

- Likelihood the disincentive value > payoff
value for the innocent (go find a new game)
Decisions, Decisions
Authorize Block
Good
false
positive
Bad
false
negative
RESPONSE
POPULATION
Incorrect decisions have a cost 

Correct decisions are free (usually)
Good Action
Gets
Blocked
Bad Action
Gets
Through
Downstream
Impacts
GAME OVER
1-UP?
Why are we still playing?
Economic/mathematical models
depend on rational participants

Free will doesn’t imply rationality

Economics studies what should
happen, behavioral economics
studies what does happen
Example of rational irrationality
Ultimatum Game

- Player A given $1000

Player A needs to split the $ with Player B

Player A gets to choose the split

- Player B receives offer

If B accepts, both get $

If B rejects, both get 0
Take it or leave it
Outcomes

- Player A’s usually offer ~50%

- Player B’s often reject if offered <30%

- This behavior occurs across cultures, levels of wealth

Emotions matter

- Heightened brain activity in 

Bilateral antierior insula (disgust) w/low offers

Dorsolateral prefrontal cortext (cognitive decision making)
w/high offers

- Fairness, Fear, Punishing the mean
Therefore: Winning strategies
depend on understanding behavior
Both attackers and defenders may exhibit bias when
making decisions - about the game and other players

Retrofit conceptual models to actual experiences

Fill in the blanks on player costs/payoffs

Risk controls still either need to

- Change friction (cost), or

- Change expected value of pay-off

Continue to analyze game dynamics over time 

- Low-risk, high frequency interactions (data) 

- High-risk, low frequency interactions (negotiation)
Prediction is very difficult, especially
about the future
Niels Bohr
Allison Miller
@selenakyle
Some references
Axelrod, Robert. The Evolution of Cooperation.

Dixit, Avinash and Nalebuff, Barry. The Art of Strategy: A
Game Theorist’s Guide to Success in Business and in Life.

Fisher, Len. Rock, Paper, Scissors: Game Theory in
Everyday Life.

Gibbons, Robert. Game Theory for Applied Economists.

Meadows, Donella. Thinking in Systems: A Primer.

Wikipedia’s sections on Game Theory, Economics, &
Probability.

2012.12 Games We Play: Defenses & Disincentives

  • 1.
    Games We Play Defensesand Disincentives Allison Miller
  • 2.
    Overview Overview of econ& game theory concepts Game theory games Infosec issues as games Designing games to win Walk-through a defense built on disincentives Wrap-up
  • 3.
    Economics applied tosecurity Utility theory Externalities Information Asymmetries Signaling Marginal cost
  • 4.
    Game theory Branch ofapplied mathematics Studies decisions made by players interacting (or competing) - Scenarios have rules and pay-offs - Costs & benefits dependent on decisions of other players Used as a framework in economics, comp sci, biology, & philosophy - Also business, negotiation, and military strategy
  • 5.
    Discussing Games Mechanics ofa payoff matrix Player 2 A B Player 1 A A1, A2 A1, B2 B B1, A2 B1, B2
  • 6.
    Discussing Games Mechanics ofdecision trees UP DOWN CIRCLE RED BLUE MARIO LUIGI KIRBY GIZMO 10, 3 2, 10 2, 5 -3, 3 A B B A A A
  • 7.
    Typical game theory"games" Chicken / Brinkmanship - Push it to the edge Volunteer’s Dilemma - For the greater good Tragedy of the Commons - Share and share alike (cumulative effect of cheating) Prisoner’s Dilemma
  • 8.
    Discussing Games Prisoner’s Dilemma Player2 Keep quiet Confess Player 1 Keep quiet -1, -1 Mutual cooperation 0, -10 Individual defection Confess -10, 0 Individual defection -3, -3 Mutual punishment
  • 9.
  • 10.
    Nash Equilibrium Equilibrium isreached when: - Players in a game have selected a strategy - Neither side can change it’s strategy independently & improve position Optimal solution in games with limited outcomes
  • 11.
    Discussing Games Prisoner’s Dilemma Player2 Keep quiet Confess Player 1 Keep quiet -1, -1 Mutual cooperation 0, -10 Individual defection Confess -10, 0 Individual defection -3, -3 Mutual punishment
  • 12.
    Setting up riskproblems as games Identify players in the game Clarify the “rules” Show me your moves Describe payoffs Single move or repeated game
  • 13.
    Discussing Games Tragedy ofthe Commons: Spam, Bandwidth usage Everyone else’s choices > n choose wise usage Less than n choose wise usage Individual choice Use resource wisely Cost, but social benefit Mutual cooperation Cost (Subsidize social use) Overuse resource Social benefit (Benefit w/o cost) 0 Resources depleted
  • 14.
    Discussing Games Chicken/Brinkmanship: VulnerabilityDisclosure Vulnerability Researcher Report Exploit Asset Owner Reward / Respond 0, 0 Responsible disclosure -2, +2 Early disclosure Ignore / Deny +2, -2 Defer vulnerability -10, -10 0-day go boom
  • 15.
    Discussing Games Volunteer’s Dilemma:Data breach cost info sharing All other victims At least one shares All keep quiet Victim Share 0 0 Cost, limited benefit Keep quiet 1 Benefit w/o cost -10 Everyone’s in the dark
  • 16.
    How games arewon Clarify dominant strategies Find equilibrium Pursue equilibrium or change the payoffs
  • 17.
    Moves Current game-play - Controlsare layered or chained until we're satisfied that for some set of attackers, the cost of the attack is higher than the utility associated with their payoff Reputation requirements for participation Role requirements for participation (access control) Incremental authentication Content/context based filtering Blacklisting / whitelisting Rate limiting Bot limiters (Captcha) Obfuscation/Encryption
  • 18.
    Counter-moves For every movethere is a counter-move
  • 19.
    Putting the pieceson the board The amount of friction inserted into the system depends on: - Value of asset to the owner - Value of the asset to potential attackers - Number of attackers expected - Portion of attacks that must be averted - Disincentive value of each layer of friction for an attacker Now it’s time to play our game
  • 20.
    Does this soundfamiliar?
  • 21.
    Managing Decisions Game Theoryis a framework for studying decisions - Since payoffs depends on the choices of other players, moves are risky - Players play based on their risk appetite - Risk management = decision management Defenders design control systems that make decisions - Where risks manifest in observable behavior - That make moves/counter-moves depending on the context and understanding of an actor’s identity or intent - Where system or individual costs/payoffs depend on the outcome of an actor’s actions
  • 22.
    SHALL WE PLAYA GAME? (SINCE WE CAN’T PLAY “CLUE” FOR EVERY LOGIN TRANSACTION NEW USER MESSAGE FRIEND REQUEST ATTACHMENT PACKET WINK POKE CLICK WE BUILD RISK MODELS)
  • 23.
    Applying Decisions Risk managementis decision management ACTOR ATTEMPTS ACTION SUBMIT WHAT IS THE REQUEST HOW TO HONOR THE REQUEST SHOULD WE HONOR? RESULT ACTION OCCURS
  • 24.
    Not all riskdecisions have a competitive element, but all competition / games have risks
  • 25.
    Create account usingfake identity Script completion of verifications Outsource captcha Create accounts across virtual devices Distribute creation of accts using botnet Scrape identities from public sites Age accounts, then reactivate Use stolen credentials Defraud verification process ... Require email verification Test for human behind keyboard Rate limit by device ID Rate limit by IP/location Look for similarities across accounts Require reputation level to proceed Filter for content / context, add auth challenge Require manual verification Manual review of account/event ...
  • 26.
    Except one smallthing... ...what kind of game is this?
  • 27.
    Multi-player Mode Offense Attempt Success Defense Deflect 4,4 0, 10 Ignore 10, 0 1, 1 Offense Attempt Success Defense Deflect 4, 4 0, 10 Ignore 10, 0 1, 1 Offense Attempt Success Defense Deflect 4, 4 0, 10 Ignore 10, 0 1, 1 Offense Attempt Success Defense Deflect 4, 4 0, 10 Ignore 10, 0 1, 1 Attackers are not the only players in the game Legitimate users that are also affected by added friction
  • 28.
    Team Dynamics So thisadds another factor into the appropriate level of friction question, which is: - Disincentive value of each layer of friction for an innocent - Likelihood the disincentive will be incorrectly applied to an innocent - Likelihood the disincentive value > payoff value for the innocent (go find a new game)
  • 29.
    Decisions, Decisions Authorize Block Good false positive Bad false negative RESPONSE POPULATION Incorrectdecisions have a cost Correct decisions are free (usually) Good Action Gets Blocked Bad Action Gets Through Downstream Impacts
  • 30.
  • 31.
    Why are westill playing? Economic/mathematical models depend on rational participants Free will doesn’t imply rationality Economics studies what should happen, behavioral economics studies what does happen
  • 32.
    Example of rationalirrationality Ultimatum Game - Player A given $1000 Player A needs to split the $ with Player B Player A gets to choose the split - Player B receives offer If B accepts, both get $ If B rejects, both get 0
  • 33.
    Take it orleave it Outcomes - Player A’s usually offer ~50% - Player B’s often reject if offered <30% - This behavior occurs across cultures, levels of wealth Emotions matter - Heightened brain activity in Bilateral antierior insula (disgust) w/low offers Dorsolateral prefrontal cortext (cognitive decision making) w/high offers - Fairness, Fear, Punishing the mean
  • 34.
    Therefore: Winning strategies dependon understanding behavior Both attackers and defenders may exhibit bias when making decisions - about the game and other players Retrofit conceptual models to actual experiences Fill in the blanks on player costs/payoffs Risk controls still either need to - Change friction (cost), or - Change expected value of pay-off Continue to analyze game dynamics over time - Low-risk, high frequency interactions (data) - High-risk, low frequency interactions (negotiation)
  • 35.
    Prediction is verydifficult, especially about the future Niels Bohr Allison Miller @selenakyle
  • 36.
    Some references Axelrod, Robert.The Evolution of Cooperation. Dixit, Avinash and Nalebuff, Barry. The Art of Strategy: A Game Theorist’s Guide to Success in Business and in Life. Fisher, Len. Rock, Paper, Scissors: Game Theory in Everyday Life. Gibbons, Robert. Game Theory for Applied Economists. Meadows, Donella. Thinking in Systems: A Primer. Wikipedia’s sections on Game Theory, Economics, & Probability.