I will present in this talk the basic ideas leading to the current framework for Adversarial Risk Analysis (ARA) as it was conceived at SAMSI around 2006-07. In particular, I will discuss the differences between ARA and the Game Theoretic approach to model adversaries behavior. This presentation will be useful to set the stage for Thursday’s working group session on ARA in which we will explore future research directions on ARA.

1. 1
Adversarial Risk Analysis
Jesus Rios
IBM Research
Raleigh, NC August 7, 2019
Program on Games, Decisions, Risk and Reliability
Opening Workshop

2. 2
Outline
• Traditional vs. Adversarial Risk Analysis
• Critique of previous approaches
• The ARA framework
• How to predict actions from intelligent others
• A few basic but illustrative examples
• Discussion on possible future research directions

3. 3
Probabilistic Risk Analysis
• Identify potential adverse events
– Risk = uncertainty & unwanted consequences
• Quantify
– their probabilities of occurrence, and
– magnitude of their consequences
• Construct and evaluate
– strategies to reduce or eliminate risks in a cost-efficient manner
• Hazard (source of risk) governed by chance
– Events from nature
– Accidents (non-intentional)
– Failure in engineered systems (like e.g. nuclear reactors)
Not by deliberate actions of intelligent actors

4. 4
Risk analysis of project costs
• Forecast costs under normal circumstances
• Risk Assessment
– Identify disruptive events
– Estimate their
• probabilities of occurrence
• impact on costs
– Compute optimal decision
– Forecast costs under risk assessment

5. 5
Risk Management
• If difference in EU (costs) before and after risk assessment too big
• Identify mitigation actions (interventions) that
– lower the chances of disruptions, and/or
– lower the extra costs from disruptions
– but they also entail a cost
Adversarial Risks
stemming from
malicious (or self-interested)
actions of other intelligent agents

6. 6
Counterbioterrorism
• “Biological Threat Risk Assessment” for DHS (Battelle, 2006)
– based on Probability Event Trees (PET)
• Government & Terrorists’ decisions treated as random events
• Methodological improvements study (NRC)
– PET appropriate for risk assessment of
• Random failures in engineering systems
• Random hazards from Nature
but not for adversarial risk assessment
• Terrorists are intelligent adversaries
trying to achieve their own objectives
• Their decisions (if rational) can be somehow anticipated
– PET cannot be used for a full risk management analysis
• Government is a decision maker not a random variable

7. 7
Banks-Anderson (2006)
• Case study
– Exploring how to defend US against a possible smallpox attack
• Stockpiling vaccine
• Stockpiling and increasing biosurveillance
• Stockpiling, surveilling, and inoculating first responders
• Mass inoculation
• Random costs (payoffs)

8. 8
Game theoretic solution
• Minimax defence (Nash equilibrium)
– The U.S. should choose the (randomized) defense with
smallest row-wise-max expected cost
• Assumptions and limitations
– Simultaneous moves game
– Zero-sum game
• Attacker’s objectives are known
– Both defender and terrorists share
• same beliefs about random costs for defender
– Impossibility to accommodate all kind of available information
• e.g. intelligence about what the attacker might do

9. 9
Critiques to the
Game Theoretic approach
• All players are expected utility maximizers
• Unrealistic assumptions
– Full and common knowledge
• e.g. opponent’s utilities are known
– Common prior for games with private information
• Symmetric predictive/normative approach
– What if multiple equilibria
– Passive understanding
• Equilibria does not provide partisan advise

10. 10
Decision analytic approach
• One-sided prescriptive support
– Use a prescriptive model (SEU) for supporting one side
– Treat opponent's decisions as uncertainties
• Assess probs over his possible actions
– Compute action of maximum expected utility for our DM
• The ‘real’ bayesian approach to games (Kadane & Larkey 1982)
– Weaken common (prior) knowledge assumption
• Asymmetric prescriptive/descriptive approach (Raiffa 2002)
– Prescriptive advice to one party conditional on
a (probalistic) description of how others will behave

11. 11
Decision analytic approaches
Decision tree + sensitivity analysis on (problematic) probabilities
• Von Winterfeldt and O’Sullivan (2006)
– Should We Protect Commercial Airplanes Against
Surface-to-Air Missile Attacks by Terrorists?

12. 12
Methods to predict adversary’s behavior
• Role playing (simulations of adversaries’ thinking)
• Adversary-preference models
– Examine decision from the adversary’s viewpoint
• Elicit adversary’s probs and utilities (point estimates)
– Solve adversary’s decision problem
• Treat adversary as an EU maximizer ( = rationality?)
• Find his action of max. EU
– Assuming we know the adversary’s true probs and utilities
• We can anticipate with certitude what the adversary will do

13. 13
Parnell (2007): Adversary-preference models
• Terrorist’s influence diagram
Deaths
Economic
Impact
Terrorist
Value
Weight
Deaths
Mitigation
Effectiveness
Weight
Economic
Impact
Max
Deaths
Max
Economic
Impact
Detect
Pre-attack
Obtain
Agent
Attack
Success
Bioterrorism
Target
Bioterrorism
Agent
Acquire
Agent
Terrorist Influence Diagram

14. Bioterrorism_Agent
Location_X [0.023709]
Acquire_Agent
Agent_A [0.0353835]
Acquire_Agent
Agent_B [0.03008]
Yes
0.400
[0]
No
0.300
[0]
Not_successful
0.250
[0]
Low
0.10003.500
[0.10003]
High
0.2515.250
[0.2515]
Attack_Success
Yes
.700
[0.11289]
Obtain_Agent
No
.600
[0.079023]
Detect_Pre_attack
Produce [0.0474138]
Detect_Pre_attack
Procure [0.0406404]
Acquire_Agent
Agent_C [0.0474138]
Bioterrorism_Agent
Location_Y [0.0474138]
Bioterrorism_Target
[0.0474138]
14
Parnell (2007)
• Terrorist’s decision tree

15. 15
Paté-Cornell & Guikema (2002)
Attacker Defender

16. 16
Paté-Cornell & Guikema (2002):
Probabilistic prediction models
• Assessing probabilities of terrorist’s actions
– From the Defender viewpoint
• Model the Attacker’s decision problem
• Estimate Attacker’s probs and utilities (point estimates)
• Calculate expected utilities of attacker’s actions
– Prob of attacker’s actions proportional to their perceived EU
• Feed these probs into the Defender’s decision problem
– uncertainty nodes with Attacker’s decisions
• Choose defense of maximum EU
• Shortcoming
– If the (idealized) adversary is an EU maximizer
he would certainly choose the attack of max EU

17. 17
How to assess probabilities over
the actions of an intelligent adversary??
• Raiffa (2002) asymmetric prescriptive/descriptive approach
– Assess probabilities from experimental data (observed behavior)
• Lab role simulation experiments
• Rios Insua, Rios & Banks (2009)
– Assessment based on analysis of adversary’s decision reasoning
• Assuming the opponent is a SEU maximizer
– Model his decision problem
– Assess his probabilities and utilities
– Find his action of maximum expected utility
– Uncertainty in the Attacker’s decision stems from
• our uncertainty about his probabilities and utilities
– Sources of information
• Available past statistical data of adversary’s decision behavior
• Expert knowledge / Intelligence

18. 18
Adversarial Risk Analysis (ARA)
• A framework to manage adversarial risks
• Distinction between risks from
– nature/accidents/failures
– actions from intelligent adversaries
• One-sided prescriptive support
– SEU model for the supported party
– Treat the adversary’s decision as uncertainties
• New methods to predict adversary’s actions
– We assumed the adversary is an expected utility maximizer
– but other descriptive models may be possible
• Uncertainty on adversary’s decision stems from
– our uncertainty about his decision analysis

19. 19

20. 20
Some basic ARA models
• Supporting a Defender against an Attacker
• Game Theory vs. Bayesian Decision Analysis
• How to assess probabilities over Attacker’s actions
– No infinity regress
• sequential Defend-Attack model
– Infinity regress
• simultaneous Defend-Attack model
• sequential Defend-Attack-Defend model

21. 21
Sequential Defend-Attack model
• Two intelligent decision makers
– Defender and Attacker
• Sequential moves
– First Defender, afterwards Attacker knowing Defender’s decision
( | , )Ap S d a
( , )Du d S ( , )Au a S
( | , )Dp S d a

22. 22
Game Theoretic Analysis
Solution:
Expected utilities at node S
Best Attacker’s decision at node A
Assuming Defender knows Attacker’s analysis
Defender’s best decision at node D

23. 23
ARA: supporting the Defender
Defender’s problem Defender’s solution of maximum SEU
Modeling input:
??

24. 24
Example: Banks-Anderson (2006)
• How to defend US against a possible smallpox attack
– Random costs
– Conditional probabilities of each kind of smallpox attack
given terrorists know what defence has been adopted
– Compute expected cost of each defence
• Solution: defence of minimum expected cost
Problematic
step
of the analysis

25. 25
Predicting Attacker’s decision: .
Defender problem Defender’s view of Attacker problem

26. 26
Solving the assessment problem
Defender’s view of
Attacker problem
Elicitation of
Assuming Attacker is an EU maximizer
D’s beliefs about
MC simulation

27. 27
Simultaneous Defend-Attack model
• Decision are taken without knowing each other’s decisions

28. 28
Game Theory Analysis
• Common knowledge
– Both know each other expected utility for every pair (d,a)
– Nash equilibrium
• When some information is not common knowledge
– Private information
• Modeled using Defender and Attacker type’s
– Common prior over private information
– Compute Bayes-Nash Equlibrium

29. 29
ARA: supporting the Defender
• Defender’s decision analysis
How to elicit
Weaken common (prior) knowledge assumption

30. 30
Assessing
• Attacker's decision analysis
as seen by the Defender
•
– : Attacker’s uncertainty about Defender’s decision
– Defender’s uncertainty about the model used by the Attacker
to predict the defense chosen by the Defender
– Its elicitation may require a further level of analysis (recursive thinking)

31. 31
The assessment problem
• To predict Attacker’s decision
– The Defender needs to solve Attacker’s decision problem
– She needs to assess
• Her beliefs about
• The assessment of requires further analysis
– D’s analysis of A’s analysis of D’s problem
• “Thinking about what-the-other-is-thinking-about…”
• It leads to a hierarchy of nested decision models

32. 32
Infinite regress within the assessment problem
• Where to stop this hierarchy of recursive analysis?
– Accommodate as much information as we can
– Stop when the Defender has no more information about u’s and p’s
– Non-informative or reference model
– Sensitivity analysis
2 2u p G.T. (Full and common knowledge)
Asymmetric
prescriptive/descriptive
approach
1 1u p
2 2
ˆ ˆu p
1 1u p
2 2
ˆ ˆu p
1 1
ˆ ˆˆ ˆu p
Where to stop?
!
1 1
ˆ ˆu p
2 2u p

33. 33
The Defend–Attack–Defend model
• Two intelligent players
– Defender and Attacker
• Sequential moves
– First, Defender moves
– Afterwards, Attacker knowing Defender’s move
– Afterwards, Defender again responding to attack outcome

34. 34
Defend–Attack–Defend model

35. 35
Standard Game Theory Analysis
• Under common knowledge of utilities and probs
• At node
• Expected utilities at node S
• Best Attacker’s decision at node A
• Best Defender’s decision at node
• Nash Solution:

36. 36
ARA:
Supporting the Defender against the Attacker
• At node
• Expected utilities at node S
• At node A
• Best Defender’s decision at node
• ??

37. 37
Predicting
• Attacker’s problem as seen by the Defender

38. 38
Assessing
Given

39. 39
Monte-Carlo approximation of
• Drawn
• Generate by
• Approximate

40. 40
The assessment of
• The Defender may want to exploit information about
how the Attacker analyzes her problem
• Hierarchy of recursive analysis
– Stop when there is no more information to elicit

41. 41
Discussion
• We assumed that the Adversary is an EU maximizer
– other (non expected utility) models of rationality are possible
• Rios Insua, Banks, and Rios (2016)
• Possible research directions
– Applications to real problems
• with more complex dynamic interactions
– Implementation issues
• Elicitation of valuable judgmental input from Defender
• Computational issues
– More than one party in both sides
• defender and attacker’s
• cooperation and competition approches
– Better attacker’s rationality models
– Adversarial Machine Learning
– Multiagent Reinforcement Learning
– Solving the assessment problem under infinite regress

42. 42
References
• Banks, D. and S. Anderson (2006) Game theory and risk analysis in the context of the
smallpox threat, in A. Wilson, G. Wilson and D. Olwell (ed) Statistical Methods in
Counterterrorism, 9-22.
• Kadane, J.B. and P.D. Larkey (1982) Subjective probability and the theory of games,
Management Science, 28, 113-120.
• Parnell, G. (2007) Multi-objective Decision Analysis, in Voeller (ed) Handbook of Science
and Technology for Homelan Security, Wiley.
• Parnell, G., Banks, D., Borio, L., Brown, G., Cox, L. A., Gannon, J., Harvill, E., Kunreuther,
H., Morse, S., Pappaioanou, M., Pollack, S., Singpurwalla, N., and Wilson, A. (2008).
Report on Methodological Improvements to the Department of Homeland Security’s
Biological Agent Risk Analysis, National Academies Press.
• Pate-Cornell, E. and S. Guikema (2002) Probabilistic modeling or terrorist threats: a
systematic analysis approach to setting priorities among countermeasures, Military
Operations Research, 7, 5-23.
• Raiffa, H. (2002) Negotiation Analysis, Harvard University Press.
• Rios Insua, D. J. Rios, and D. Banks (2009) Adversarial risk analysis, Journal of the
American Statistical Association, 104, 841-854.
• Rios Insua, D. D. Banka, and J. Rios (2016) Modeling opponents in adversarial risk
analysis. Risk Analysis, 36(4), 742-755.
• von Winterfeldt, D. and T.M. O’Sullivan (2006) Should we protect commercial airplanes
against surface-to-air missile attacks by terrorists? Decision Analysis, 3, 63-75.