Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Robust Adversarial Risk Analysis: A level-k Approach

894 views

Published on

Adversarial risk analysis is an active and important area of decision analytic research. Both single-actor decision analysis and multiple-actor game theory have been applied to this problem, with game theoretic methods being particularly popular. While game theory models do explicitly capture strategic interactions between attackers and defenders, two of the key assumptions—decision making based on subjective expected utility maximization and common knowledge of rationality—are known to be descriptively inaccurate in some situations. This paper addresses these shortcomings by proposing, formulating, and illustrating the application of robust optimization methodologies to a level-k game theory model for adversarial risk analysis. Level-k game theory provides a practical method for modeling bounded rationality. Robust optimization provides an alternative way to model the actions of conservative players facing “deep” uncertainties about their environment—uncertainties that are possible to bound but which are difficult or impossible to represent using probability distributions. Our approach thus combines level-k and robust optimization insights to provide a computationally tractable model of boundedly rational players who are faced with significant and difficult to quantify uncertainties.

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

Robust Adversarial Risk Analysis: A level-k Approach

  1. 1. Robust Adversarial Risk Analysis: A level-k approach LAURA MCLAY CASEY ROTHSCHILD SETH GUIKEMAVirginia Commonwealth University Wellesley College Johns Hopkins University Statistical Sciences & Operations Economics Geography and Environmental Research crothsch@wellesley.edu Engineering lamclay@vcu.edu sguikema@jhu.edu Paper to appear in Decision Analysis This paper was supported by research funded by DHS S&T under contract HSHQDC-10-C-00105.
  2. 2. Summary• Robust optimization methodologies can be combined with Adversarial Risk Analysis (ARA) – Examines how an attacker is likely to process partial information about defensive postures – Considers defenders with various levels of strategic information – Maintains computational efficiency of prior ARA algorithms• Models actions of conservative players facing “deep” uncertainties about their environment – Uncertainties that are possible to bound – Uncertainties that are difficult or impossible to represent using probability distributions• Apply our approach to a Defender-Attacker game
  3. 3. Motivation• National Academy of Science 2008 report concerned that the Bioterrorism Risk Assessment program fails to adequately model adaptive U.S. adversaries• Apply advanced analytical methods to model an intelligent adversary seeking to cause harm in situations characterized by imperfect information• Expected utility maximization may not always be an appropriate choice paradigm to describe adversary behavior
  4. 4. Robust optimization (RO)• Concerned with uncertain data in optimization models• RO finds the optimal solution that satisfies all constraints given any possible distribution of constraint data• RO methods allow uncertainties in input parameters to be distribution-free• RO models can be solved efficiently by standard algorithms if uncertain inputs have a structure that does not make the problem harder – Reformulate semi-infinite RO models as standard math programs• RO models are a game against nature – Zero-sum, perfect and complete information
  5. 5. Robust players• Robust players are more “conservative”• A robust paradigm is not a minimax paradigm – Both minimax and robust paradigms are distribution-free and optimize over the worst-case scenario – Minimax: player minimizes the maximum possible loss over both the uncertainty set and opponent actions – Robust: allows player to optimally respond to opponent while robust against non-strategic uncertainties• Game against nature, where nature sets worst possible values of uncertain inputs with perfect information
  6. 6. Robust games• Multiple players, each of whom is a robust decision maker – “nature” is a notional player• Can be modeled as 1. stochastic programming models that consider data uncertainty within an optimization problem 2. stochastic games, where players determine the transition probabilities• Equilibria exist for robust games under a wide range of assumptions• Rectilinearity of the uncertainty sets does not make robust games “harder” than non-robust games – No dependencies between the uncertainty setsAghassi, M., D. Bertsimas. 2006. Robust game theory, Mathematical Programming 107, 231 – 273.Kardes, E. Ordóñez, F., Hall, R.W. 2011. Discounted robust stochastic games and an application to queuing control, Operations Research 59(2), 365 – 382.
  7. 7. Level-k games• Models players who are boundedly rational who play against a level-(k-1) opponent• k = level of strategic sophistication – k = 0: player acts randomly – k = 1: player acts optimally but is not strategic – k = 2: player acts optimally by assuming a level-1 opponent – k = : player is rational• Level-k games do not necessarily converge to a Nash equilibrium as k• Level-k is a good heuristic – Accounts for behavior in a wide range of experiments• Level-k have computational advantages – Solution is recursive
  8. 8. Approach• Extend a level-k Bayesian ARA solution algorithm for a sequential Defender-Attacker model with imperfect observability of the Defender’s move*• Two boundedly rational players: – Defender (Daphne) chooses d D – Attacker (Apollo) chooses a A• Daphne assesses Apollo’s level of strategic sophistication k• Daphne’s beliefs of Apollo’s preferences  A – CDF FA ( A ) with expected utility  A (a, d ; A )• What Daphne believes Apollo believes about her preferences D – CDF FD ( D ) with expected utility  D (a, d ;D ) *• Daphne’s true preferences  DRios Insua, I., J. Rios, and D. Banks. 2009. Adversarial risk analysis, Journal of the American Statistical Association 104, 841-854.Rothschild, C., L.A. McLay, S. Guikema. 2012. Adversarial risk analysis: A level-k approach. Risk Analysis (to appear).
  9. 9. Approach, cont’d.• Uncertain information structure about Daphne’s decisions ~ – Apollo’s signal    with probabilities q ( | d )• Daphne assesses Apollo’s decisions  – Probability of selecting actions p D (a |  ) • (Daphne estimates of) Apollo’s priors p A (d ), d  D• Daphne’s expected utility E D   ~ D (a | d ) D (a, d ; D ) p * aA
  10. 10. Robust ARA• Robustness with respect to the (finite) information structure ~ q ( | d ) – These are the decisions of a notional player (nature) who selects them to be as bad as possible for a player j j• qD ( | d ) and q A,a ( | d ) denote the selections from Daphne and Apollo’s level-j decision problems, respectively – independent whenever ( j ,  )  (m,  ) – Treats each player at each level as being independently robust – Different levels of same player at different levels have different realizations of the uncertain parameters• Interval parameter uncertainty: j j qD ( | d ) and q A,a ( | d ) lie in L( | d ),U ( | d ) – can be computed for each stage in the game by solving a simple linear program
  11. 11. ARA algorithm: a level-k approach 7 steps *1. Assess preferences and basic perceptions  D for Daphne ~ (and the information structure q ( | d ) )2. Assess Daphne’s perceptions of Apollo via the distribution FA ( A )3. Assess Daphne’s beliefs about Apollo’s perceptions of Daphne via the distribution FD ( D )4. Assess Daphne’s strategic reasoning level-k (with k > 1)5. “Initiate” the recursion via uniform level-1 priors:  a) p1 (d )  1 A d  D D  b) p1 (a |  )  1 D a  A A
  12. 12. ARA algorithm, cont’d.6. Solve recursively for priors up to level-k: For j=1,…,k – 1 a) Solve for Daphne’s level-(j+1) priors: • Randomly draw a large number of  A from FA • For each draw, compute Apollo’s optimal action assuming Apollo is level-j:  j j  p A (d )q A,a ( | d )     j j dD  p A (d )q A,a ( | d ) A  (a, d ; A )   * a j ( A |  )  arg max a   j min   q A ,a ( | d ),d D d D   j    subject to L( | d )  q A,a ( | d )  U ( | d ), d  D     • Estimate pD (a |  ),   , via the fraction of  A with a * ( A |  )  a. j • Set  pD1 (a |  )  pD (a |  ),  .   [non-robust case] a ( |  )  arg max     j *  p (d )q ( | d ) A   ( a, d ; A )  ,    j A  a  p (d )q ( | d ) d D j A A   d D 
  13. 13. ARA algorithm, cont’d.6. b) Solve for Apollo’s level-(j+1) priors: • Randomly draw a large number of  D from FD • For each draw, compute Daphne’s optimal action assuming Daphne is level-j: j j   j min  qD ( | d ) pD (a |  ) D (a, d ; D )  qD ( |d ),    a A   j  d * ( D )  arg max dD subject to   qD ( | d )  1 j   j   L ( | d )  qD ( | d )  U ( | d ),    • Estimate  via the fraction of the with  ~ (d ) pA D d * ( D )  d  • Set p j 1 (a |  )  p (a |  ),  . D D[non-robust case]  j  d * ( D )  arg max d D   q ( | d )  pD (a |  ) D ( a, d ; D )  j   a A 
  14. 14. ARA algorithm, cont’d.7. Solve for (level-k) Daphne’s optimal action: j k   k min  qD ( | d ) pD (a |  ) D (a, d ; D )  qD ( |d ),    a A  *  k  d k ( D )  arg max dD subject to   qD ( | d )  1   k   L ( | d )  qD ( | d )  U ( | d ),      [non-robust case]  k  * *  d ( )  arg max d D   q ( | d )  pD ( a |  ) D ( a, d ; D )  k D   a A 
  15. 15. Example• Apollo can either (actively try to) initiate a Smallpox attack (a1) or No Attack (a2) with A={a1,a2}• Daphne can either install an array of smallpox Detectors (d1) or No Detectors (d2), with D={d1,d2}• Information structure   { 1 ,  2 ,}    q ( i | di )  qi , q ( | d i )  1  qi and q ( i | d j )  0 if i  j• The parameters qi are uncertain, with a range of possible values [0.3, 0.7]Compare three cases:• Baseline robust model = both Daphne and Apollo are robust decision-makers• Hybrid model = Daphne is a robust decision-maker but Apollo is not• Non-robust model = Neither Daphne nor Apollo are robust decision-makers
  16. 16. Example: Utilities• Daphne  D (1), D (2) with real values ( * D * (1), D (2))  (4,15) –D (1) = cost of installing detectors – D (2) = benefit of having Smallpox detectors in place if an attack is launched – Daphne’s beliefs about Apollo’s beliefs about her preferences taken to be uniformly distributed on [0, 10] x [5, 15]• Apollo  A (1), A (2)  –  A (1) = direct cost from mounting an attack –  A (2) = additional cost if attack occurs in the presence of detectors – Daphne’s beliefs are uniformly distributed on [0, 20] x [0,10] Detector None Detector None  D (2)   D (1)  10  10  A (2)   A (1) 10   A (1) Smallpox No Attack  D (1) 0 0 0  D (a, d ; D )  A (a, d ; A )
  17. 17. Results as a function of k• Daphne’s optimal baseline robust decision is not to install detectors (k = 2, 3,…, 10 ) • Non-robust model: install detectors (k = 4,5,…, 10 ) • Hybrid model: install detectors (k = 2, 3,…, 10 )• Daphne’s baseline robust expected utilities do not unilaterally decrease when compared tothe non-robust base case• Solutions to all three models converge to a perfect Bayesian equilibrium with k.
  18. 18. Sensitivity of Daphne’s optimal level-4 decisions to her preference parameters Gray = install detectors, white = do not install detectors 15 15 15 14 14 14 13 13 13 12 12 12 11 11 11D*(2) D*(2) D*(2) 10 10 10 9 9 9 8 8 8 7 7 7 6 6 6 5 3 3.2 3.4 3.6 3.8 4 4.2 4.4 4.6 4.8 5 5 5 3 3.2 3.4 3.6 3.8 4 4.2 4.4 4.6 4.8 5 3 3.2 3.4 3.6 3.8 4 4.2 4.4 4.6 4.8 5 D*(1) D*(1) D*(1) Baseline robust Hybrid Non-robust In baseline robust model, Daphne’s decisions are relatively insensitive to  D (2) than in the other two models – This is a result of having lower priors for a non-robust Apollo attacking Daphne’s decisions in the baseline robust and the non-robust models differ in 20 scenarios Daphne’s decisions in the hybrid and the non-robust models differ in 14 scenarios
  19. 19. Conclusions• Extend level-k adversarial risk analysis model and algorithm to consider robust optimization methodologies• A computational examples indicates that – optimal decisions can be significantly different when considering robust decision-making as compared to non-robust decision-making – robust decision-making does not always unilaterally decrease the expected utility of a given agent (vis a vis the parallel model with expected utility maximizers) – uncertainty with regard to informational requirements set by nature may play out in different, unexpected ways between the players• Posit that robust level-k approach offers promise for terrorism risk assessment and management – Addresses NAS concerns• Could be used in a plural modeling approach, since no one model may be effective for modeling adversaries

×