Successfully reported this slideshow.
Your SlideShare is downloading. ×

Is robustness really robust? how different definitions of robustness impact decision-making under climate change

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 35 Ad
Advertisement

More Related Content

Slideshows for you (20)

Advertisement

Similar to Is robustness really robust? how different definitions of robustness impact decision-making under climate change (20)

Recently uploaded (20)

Advertisement

Is robustness really robust? how different definitions of robustness impact decision-making under climate change

  1. 1. Is robustness really robust? How different definitions of robustness impact decision-making under climate change M. Giuliani and A. Castelletti NRM Polimi
  2. 2. EWRI2016 NRM What is robustness? “the insensitivity of system design to errors, random or otherwise, in the estimates of those parameters affecting design choice” (Matalas and Fiering, 1977)
  3. 3. EWRI2016 NRM Why searching robustness? “robust strategies perform well across a wide range of deeply uncertain scenarios” (Lempert, 2002)
  4. 4. EWRI2016 NRM Why searching robustness? “robust strategies perform well across a wide range of deeply uncertain scenarios” SPM Summary for Policymakers 6.0 4.0 2.0 −2.0 0.0 (o C) 42 32 39 historical RCP2.6 RCP8.5 Global average surface temperature change(a) RCP2.6 RCP4.5 RCP6.0 RCP8.5 Mean over 2081–2100 1950 2000 2050 2100 Northern Hemisphere September sea ice extent(b) 10.0 8.0 6.0 4.0 2.0 (106 km2 ) 29 (3) 37 (5) 39 (5)
  5. 5. EWRI2016 NRM Robustness in the literature: first works RISK, AMBIGUITY, AND THE SAVAGE AXIOMS* By DANIEL ELLSBERG I. Are there uncertainties that are not risks? 643. II. Uncertainties that are not risks, 647.- JII. Why are some uncertainties not risks? - 656. I. ARE THERE UNCERTAINTIES THAT ARE NOT RISKS? There has always been a good deal of skepticism about the behavioral significance of Frank Knight's distinction between "meas- urable uncertainty" or "risk," which may be represented by numeri- cal probabilities, and "unmeasurable uncertainty" which cannot. Knight maintained that the latter "uncertainty" prevailed - and hence that numerical probabilities were inapplicable - in situations when the decision-maker was ignorant of the statistical frequencies of events relevant to his decision; or when a priori calculations were impossible; or when the relevant events were in some sense unique; or when an important, once-and-for-all decision was concerned.' Yet the feeling has persisted that, even in these situations, people tend to behave "as though" they assigned numerical probabilities, or "degrees of belief," to the events impinging on their actions. How- ever, it is hard either to confirm or to deny such a proposition in the absence of precisely-defined procedures for measuring these alleged "degrees of belief." What might it mean operationally, in terms of refutable predic- tions about observable phenomena, to say that someone behaves "as if" he assigned quantitative likelihoods to events: or to say that he does not? An intuitive answer may emerge if we consider an example proposed by Shackle, who takes an extreme form of the Knightian Oxford University Press [1961]. “willingness of decision makers to sacrifice expected performance to improve robustness to uncertainty” (Maas et al. 1962)
  6. 6. EWRI2016 NRM Robustness in the literature A Comparison of Robustness Metrics for Scheduling DAGs on Heterogeneous Systems Louis-Claude Canon and Emmanuel Jeannot LORIA, INRIA, Nancy University, CNRS Campus Scientique – BP 239 54506 Vandoeuvre-l`es-Nancy Cedex, France {louis-claude.canon,emmanuel.jeannot}@loria.fr Abstract— A schedule is said robust if it is able to absorb some degree of uncertainty in tasks duration while maintaining a stable solution. This intuitive notion of robustness has led to a lot of different interpretations and metrics. However, no comparison of these different metrics have ever been preformed. In this paper, we perform an experimental study of these different metrics and show how they are correlated to each other in the case of task scheduling, with dependencies between tasks. I. INTRODUCTION Research in scheduling has gathered a lot of different solutions depending on the pursued objective. For instance, if the objective function to minimize is the makespan (the total execution time of the application) different heuristics have been proposed in the literature such as HEFT [17], CPOP [17], hybrid remapper [11], BIL [12], hybrid method [13] or GDL [16]. However, there are a lot of other possible objectives than minimizing the makespan. Among these objectives the robustness has recently received a lot of attention [1], [3], [5], [7], [14], [15]. A schedule is said robust if it is able to absorb some degree of uncertainty in the task duration while maintaining a stable solution. Thus, it is important to note that the robustness alone is not a metric but it gives an idea of the stability of the solution with regards to another performance metric such as schedule length, load balance of an application, variations in their duration. Moreover, we try to see to which extend optimizing the makespan can help in optimizing the robustness. In other words, we try to answer the following question: are short schedules more robust that long ones? In this work we also test some makespan-centric scheduling heuristics of the literature (BIL, HEFT, Hyb.BMCT) and see on different scenarios how they perform in terms of robustness. Therefore, the contribution of this paper is the following: we provide a comprehensive study of different robustness metrics in the case of task graph scheduling. We study how they are correlated to each other and whether robustness and makespan are conflicting objectives or not. Finally, we compare the robustness of three different makespan-centric scheduling heuristics. The remaining of the paper is organized as follows. In Section II we present the problem and the notations used in this paper. Several works dealing with robustness are detailed in Section III. The robustness metrics we use are described in Section IV. In Section V we present the experimental setup we used for testing and comparing the different metrics. Results are shown in Section VI and discussed in Section VII. Finally, conclusion and future works are given in Section VIII II. MODELS [5], [7], [14], [15]. A schedule is said robust if it is able to absorb some degree of uncertainty in the task duration while maintaining a stable solution. Thus, it is important to note that the robustness alone is not a metric but it gives an idea of the stability of the solution with regards to another performance metric such as schedule length, load balance of an application, queue waiting time of batch scheduler, etc. The reason why robustness is becoming an important objective is the recent focus on large systems that can be dynamic and where uncertainty in terms of workload or resource usage can be very important. Moreover, a brief look at the literature shows that despite the fact that robustness is a very intuitive notion there is no consensus on a single metric. Conversely, almost each paper uses its own metric depending on the studied problem and the general context of the work. Furthermore, there does not exist a comparison between these different metrics, hence it is not possible to decide which metric to use when designing a heuristic. In this paper we focus on comparing different metrics of robustness in the context of scheduling task graph on heterogeneous systems: we model an application as a set of tasks having precedence constraints and a task as a set of statements. The performance metric we use is the makespan (the completion time of the application) and therefore, we look at the robustness of the makespan when tasks may have Section IV. In Section V we present the experimental setup we used for testing and comparing the different metrics. Results are shown in Section VI and discussed in Section VII. Finally, conclusion and future works are given in Section VIII II. MODELS We model the parallel application by a directed acyclic graph (DAG) G = (V, E, C), where V is a set of nodes that represent tasks and E is a set of edges that represent dependencies between tasks (often due to communications). C is the set of communication volume between tasks. The target platform is composed of a set of heterogeneous resources each having different capacities in terms of network speed. When there is no uncertainty we use two matrices to model communication speed: T = (τi,j)1≤i≤m,1≤j≤m and L = (li,j)1≤i≤m,1≤j≤m, where m is the number of machines. τi,j is the time to send one data element from processor i to processor j and li,j is the network latency from processor i to processor j. To model the fact that communications are way faster between two tasks mapped on the same processor and thus negligible, we put ∀i ∈ [1, m], τi,i = li,i = 0. Hence, if task 1 is mapped to processor i and task 2 is mapped to processor j then the communication time between these two tasks will be: li,j + c1,2 × τi,j, where c1,2 ∈ C is the communication volume between task 1 and task 2. As we 1-4244-1388-5/07/$25.00 © 2007 IEEE 2007 IEEE International Conference on Cluster Computing558 SIAM REVIEW c⃝ 2011 Society for Industrial and Applied Mathematics Vol. 53, No. 3, pp. 464–501 Theory and Applications of Robust Optimization∗ Dimitris Bertsimas† David B. Brown‡ Constantine Caramanis§ Abstract. In this paper we survey the primary research, both theoretical and applied, in the area of robust optimization (RO). Our focus is on the computational attractiveness of RO approaches, as well as the modeling power and broad applicability of the methodology. In addition to surveying prominent theoretical results of RO, we also present some recent results linking RO to adaptable models for multistage decision-making problems. Finally, we highlight applications of RO across a wide spectrum of domains, including finance, statistics, learning, and various areas of engineering. Key words. robust optimization, robustness, adaptable optimization, applications of robust optimiza- tion AMS subject classifications. 90C31, 93B40, 93D21 rcopyright;seehttp://www.siam.org/journals/ojsa.php
  7. 7. EWRI2016 NRM Robustness in the WATER literature RESEARCH ARTICLE 10.1002/2014WR015338 Beyond optimality: Multistakeholder robustness tradeoffs for regional water portfolio planning under deep uncertainty Jonathan D. Herman1, Harrison B. Zeff2, Patrick M. Reed1, and Gregory W. Characklis2 1 Department of Civil and Environmental Engineering, Cornell University, Ithaca, New York, USA, 2 Department of Environmental Sciences and Engineering, University of North Carolina, Chapel Hill, North Carolina, USA Abstract While optimality is a foundational mathematical concept in water resources planning and man- agement, ‘‘optimal’’ solutions may be vulnerable to failure if deeply uncertain future conditions deviate from those assumed during optimization. These vulnerabilities may produce severely asymmetric impacts across a region, making it vital to evaluate the robustness of management strategies as well as their impacts for regional stakeholders. In this study, we contribute a multistakeholder many-objective robust decision making (MORDM) framework that blends many-objective search and uncertainty analysis tools to discover key tradeoffs between water supply alternatives and their robustness to deep uncertainties (e.g., population pressures, climate change, and financial risks). The proposed framework is demonstrated for four intercon- nected water utilities representing major stakeholders in the ‘‘Research Triangle’’ region of North Carolina, U.S. The utilities supply well over one million customers and have the ability to collectively manage drought via transfer agreements and shared infrastructure. We show that water portfolios for this region that com- pose optimal tradeoffs (i.e., Pareto-approximate solutions) under expected future conditions may suffer sig- nificantly degraded performance with only modest changes in deeply uncertain hydrologic and economic factors. We then use the Patient Rule Induction Method (PRIM) to identify which uncertain factors drive the individual and collective vulnerabilities for the four cooperating utilities. Our framework identifies key stake- holder dependencies and robustness tradeoffs associated with cooperative regional planning, which are critical to understanding the tensions between individual versus regional water supply goals. Cooperative demand management was found to be the key factor controlling the robustness of regional water supply planning, dominating other hydroclimatic and economic uncertainties through the 2025 planning horizon. Results suggest that a modest reduction in the projected rate of demand growth (from approximately 3% per year to 2.4%) will substantially improve the utilities’ robustness to future uncertainty and reduce the potential for regional tensions. The proposed multistakeholder MORDM framework offers critical insights into the risks and challenges posed by rising water demands and hydrological uncertainties, providing a planning template for regions now forced to confront rapidly evolving water scarcity risks. 1. Introduction Cooperative regional water portfolio planning, in which adaptive management strategies are coordinated across multiple water utilities, is a core component of the ‘‘soft path’’ approach for utilizing existing infra- structure more efficiently [Gleick, 2002, 2003]. Such portfolios may combine conservation measures [e.g., Renwick and Green, 2000], water transfers [Lund and Israel, 1995; Wilchfort and Lund, 1997; Hadjigeorgalis, 2008], and financial instruments [Brown and Carriquiry, 2007; Zeff and Characklis, 2013; Zeff et al., 2014] to diversify the management of scarcity risks in a flexible manner. Regional water portfolios provide an innova- tive approach to offset future demand and climate risks, yet their potential vulnerabilities to deep uncertain- ties must be recognized. Deep uncertainty acknowledges that decision makers may not be able to enumerate all sources of uncertainty in a system nor their associated probabilities [Langlois and Cosgel, 1993; Lempert, 2002; Lempert et al., 2003; Polasky et al., 2011; Kasprzyk et al., 2013]. Also referred to as Knigh- tian uncertainty [Knight, 1921], a core concern is providing appropriate risk management actions despite the uncertainties in correctly specifying all probability distributions [Friedman, 1976]. Deep uncertainty is especially prevalent in complex economic and environmental systems, where rapidly evolving systems may cause management policies to produce severe unintended consequences on stakeholders. These traits have long been recognized in water supply planning, making it a classic example of a ‘‘wicked’’ problem Key Points: We advance many-objective robust decision making for multiple stakeholders Stakeholders’ robustness exhibits dependencies, vulnerabilities, and tradeoffs A modest reduction in demand growth rate insulates against future uncertainty Supporting Information: Readme Supplement Correspondence to: J. Herman, jdh366@cornell.edu Citation: Herman, J. D., H. B. Zeff, P. M. Reed, and G. W. Characklis (2014), Beyond optimality: Multistakeholder robustness tradeoffs for regional water portfolio planning under deep uncertainty, Water Resour. Res., 50, doi:10.1002/2014WR015338. Received 21 JAN 2014 Accepted 19 AUG 2014 Accepted article online 23 AUG 2014 HERMAN ET AL. VC 2014. American Geophysical Union. All Rights Reserved. 1 Water Resources Research PUBLICATIONS Dynamic adaptive policy pathways: A method for crafting robust decisions for a deeply uncertain world Marjolijn Haasnoot a,b,d, *, Jan H. Kwakkel c , Warren E. Walker c , Judith ter Maat d a Utrecht University, Department of Geosciences, P.O. Box 80115, 3508 TC Utrecht, The Netherlands b Twente University, Department of Water Engineering Management, P.O. Box 217,7500 AE Enschede, The Netherlands c Delft University of Technology, Faculty of Technology, Policy and Management, P.O. Box 5015, 2600 GA Delft, The Netherlands d Deltares, P.O. Box 177, 2600 MH Delft, The Netherlands 1. Introduction Nowadays, decisionmakers face deep uncertainties about a myriad of external factors, such as climate change, population growth, new technologies, economic developments, and their impacts. Moreover, not only environmental conditions, but also societal perspectives and preferences may change over time, including stakeholders’ interests and their evaluation of plans (Offermans, 2010; van der Brugge et al., 2005). Traditionally, decisionmakers in many policy domains, including water manage- ment, assume that the future can be predicted. They develop a static ‘optimal’ plan using a single ‘most likely’ future (often based on the extrapolation of trends) or a static ‘robust’ plan that will produce acceptable outcomes in most plausible future worlds (Dessai and Hulme, 2007; Dessai and Van der Sluijs, 2007; Hallegatte et al., 2012). However, if the future turns out to be different from the hypothesized future(s), the plan is likely to fail. McInerney et al. (2012) liken this to ‘‘dancing on the top of a needle’’. But, as the future unfolds policymakers learn and usually respond to the new situation by adapting their plans (ad hoc) to the new reality. Adaptation over the course of time is not only determined by what is known or anticipated at present, but also by what is experienced and learned as the future unfolds (Yohe, 1990) and by the policy responses to events (Haasnoot et al., 2012). Thus, policymaking becomes part of the storyline, and thereby an essential component of the total uncertainty – in fact, Hallegatte et al. (2012) include the adaptation of decisions over time in an updated definition of ‘deep uncertainty’. To address these deep uncertainties, a new planning paradigm has emerged. This paradigm holds that, in light of the deep uncertainties, one needs to design dynamic adaptive plans (Albrechts, 2004; de Neufville and Odoni, 2003; Haasnoot et al., 2011; Hallegatte, 2009; Hallegatte et al., 2012; Ranger et al., 2010; Schwartz and Trigeorgis, 2004; Swanson et al., 2010). Such plans contain a strategic vision of the future, commit to short-term actions, and establish a framework to guide future actions (Albrechts, 2004; Ranger et al., 2010). The seeds for this planning paradigm were planted almost a century ago. Dewey (1927) argued Global Environmental Change 23 (2013) 485–498 A R T I C L E I N F O Article history: Received 15 June 2012 Received in revised form 3 December 2012 Accepted 18 December 2012 Keywords: Uncertainty Policymaking Adaptation pathways Adaptive policies Water management Rhine delta A B S T R A C T A new paradigm for planning under conditions of deep uncertainty has emerged in the literature. According to this paradigm, a planner should create a strategic vision of the future, commit to short-term actions, and establish a framework to guide future actions. A plan that embodies these ideas allows for its dynamic adaptation over time to meet changing circumstances. We propose a method for decisionmaking under uncertain global and regional changes called ‘Dynamic Adaptive Policy Pathways’. We base our approach on two complementary approaches for designing adaptive plans: ‘Adaptive Policymaking’ and ‘Adaptation Pathways’. Adaptive Policymaking is a theoretical approach describing a planning process with different types of actions (e.g. ‘mitigating actions’ and ‘hedging actions’) and signposts to monitor to see if adaptation is needed. In contrast, Adaptation Pathways provides an analytical approach for exploring and sequencing a set of possible actions based on alternative external developments over time. We illustrate the Dynamic Adaptive Policy Pathways approach by producing an adaptive plan for long-term water management of the Rhine Delta in the Netherlands that takes into account the deep uncertainties about the future arising from social, political, technological, economic, and climate changes. The results suggest that it is worthwhile to further test and use the approach. ß 2012 Elsevier Ltd. * Corresponding author at: Deltares, P.O. Box 177, 2600 MH Delft, The Netherlands. Tel.: +31 88 335 81 75. E-mail addresses: Marjolijn.Haasnoot@deltares.nl (M. Haasnoot), J.H.Kwakkel@tudelft.nl (J.H. Kwakkel), W.E.Walker@tudelft.nl (W.E. Walker), Judith.TerMaat@deltares.nl (J. ter Maat). Contents lists available at SciVerse ScienceDirect Global Environmental Change journal homepage: www.elsevier.com/locate/gloenvcha 0959-3780 ß 2012 Elsevier Ltd. http://dx.doi.org/10.1016/j.gloenvcha.2012.12.006 Open access under CC BY-NC-ND license. Open access under CC BY-NC-ND license. Robust Decision Making and Info-Gap Decision Theory for water resource system planning Evgenii S. Matrosov, Ashley M. Woods, Julien J. Harou ⇑ Department of Civil, Environmental and Geomatic Engineering, University College London, Chadwick Building, Gower Street, London WC1E 6BT, UK a r t i c l e i n f o Article history: Received 5 August 2011 Received in revised form 5 March 2013 Accepted 9 March 2013 Available online 29 March 2013 This manuscript was handled by Konstantine P. Georgakakos, Editor-in-Chief, with the assistance of Aris P. Georgakakos, Associate Editor Keywords: Water resources planning Robust Decision Making (RDM) Info-Gap Decision Theory (IGDT) Uncertainty Infrastructure planning s u m m a r y Stationarity assumptions of linked human–water systems are frequently invalid given the difficult-to- predict changes affecting such systems. In this case water planning occurs under conditions of deep or severe uncertainty, where the statistical distributions of future conditions and events are poorly known. In such situations predictive system simulation models are typically run under different scenarios to evaluate the performance of future plans under different conditions. Given that there are many possible plans and many possible futures, which simulations will lead to the best designs? Robust Decision Mak- ing (RDM) and Info-Gap Decision Theory (IGDT) provide a structured approach to planning complex sys- tems under such uncertainty. Both RDM and IGDT make repeated use of trusted simulation models to evaluate different plans under different future conditions. Both methods seek to identify robust rather than optimal decisions, where a robust decision works satisfactorily over a broad range of possible futures. IGDT efficiently charts system performance with robustness and opportuneness plots summaris- ing system performance for different plans under the most dire and favourable sets of future conditions. RDM samples a wider range of dire, benign and opportune futures and offers a holistic assessment of the performance of different options. RDM also identifies through ‘scenario discovery’ which combinations of uncertain future stresses lead to system vulnerabilities. In our study we apply both frameworks to a water resource system planning problem: London’s water supply system expansion in the Thames basin, UK. The methods help identify which out of 20 proposed water supply infrastructure portfolios is the most robust given severely uncertain future hydrological inflows, water demands and energy prices. Mul- tiple criteria of system performance are considered: service reliability, storage susceptibility, capital and operating cost, energy use and environmental flows. Initially the two decision frameworks lead to differ- ent recommendations. We show the methods are complementary and can be beneficially used together to better understand results and reveal how the particulars of each method can skew results towards par- ticular future plans. Ó 2013 Elsevier B.V. All rights reserved. 1. Introduction Water resource systems are sensitive to climate and population changes, making supply infrastructure planning difficult. Planning et al., 2006; Lempert and Collins, 2007) rather than optimality. A ‘robust’ system performs satisfactorily, or satisfices (Simon, 1959) performance criteria, over a wide range of uncertain futures rather than performing optimally over the historical period or a few Journal of Hydrology 494 (2013) 43–58 Contents lists available at SciVerse ScienceDirect Journal of Hydrology journal homepage: www.elsevier.com/locate/jhydrol Many objective robust decision making for complex environmental systems undergoing change Joseph R. Kasprzyk a,*, Shanthi Nataraj b , Patrick M. Reed a , Robert J. Lempert b a Department of Civil and Environmental Engineering, Penn State University, 212 Sackett Building, University Park, PA 16802, USA b RAND Corporation, 1776 Main Street, Santa Monica, CA, USA a r t i c l e i n f o Article history: Received 5 July 2012 Received in revised form 12 December 2012 Accepted 13 December 2012 Available online 23 January 2013 Keywords: Multiobjective evolutionary algorithms Robust decision making Interactive visual analytics Uncertainty analysis Water supply Environmental management a b s t r a c t This paper introduces many objective robust decision making (MORDM). MORDM combines concepts and methods from many objective evolutionary optimization and robust decision making (RDM), along with extensive use of interactive visual analytics, to facilitate the management of complex environmental systems. Many objective evolutionary search is used to generate alternatives for complex planning problems, enabling the discovery of the key tradeoffs among planning objectives. RDM then determines the robustness of planning alternatives to deeply uncertain future conditions and facilitates decision makers’ selection of promising candidate solutions. MORDM tests each solution under the ensemble of future extreme states of the world (SOW). Interactive visual analytics are used to explore whether so- lutions of interest are robust to a wide range of plausible future conditions (i.e., assessment of their Pareto satisficing behavior in alternative SOW). Scenario discovery methods that use statistical data mining algorithms are then used to identify what assumptions and system conditions strongly influence the cost-effectiveness, efficiency, and reliability of the robust alternatives. The framework is demon- strated using a case study that examines a single city’s water supply in the Lower Rio Grande Valley (LRGV) in Texas, USA. Results suggest that including robustness as a decision criterion can dramatically change the formulation of complex environmental management problems as well as the negotiated selection of candidate alternatives to implement. MORDM also allows decision makers to characterize the most important vulnerabilities for their systems, which should be the focus of ex post monitoring and identification of triggers for adaptive management. Ó 2012 Elsevier Ltd. All rights reserved. 1. Introduction This paper contributes the many objective robust decision making (MORDM) framework, which combines many objective evolutionary optimization, robust decision making (RDM), and interactive visual analytics to facilitate the management of complex environmental systems. The MORDM framework seeks to address several key challenges for environmental systems undergoing change. The first is how to evaluate the performance of alternative planning and management strategies. To make these evaluations, planners have traditionally used cost benefit analysis, in which are rewarded and penalized in ways that cannot be predicted a priori (Franssen, 2005). The approach has also been shown to inadequately compensate for non-monetary benefits (Bromley and Beattie, 1973) especially when multiple policies are considered (Hoehn and Randall, 1989). Multiobjective approaches instead seek to quantify the large number of conflicting objectives that charac- terize planning. In addition to cost, it has been recognized that complex planning efforts often reveal additional critical perfor- mance objectives (Hitch, 1960), such as maximizing reliable per- formance, minimizing environmental damages, and improving system efficiency. The Harvard Water Program was one of the Contents lists available at SciVerse ScienceDirect Environmental Modelling Software journal homepage: www.elsevier.com/locate/envsoft Environmental Modelling Software 42 (2013) 55e71 LETTERS PUBLISHED ONLINE: 20 JULY 2015 | DOI: 10.1038/NCLIMATE2721 Selection of climate policies under the uncertainties in the Fifth Assessment Report of the IPCC L. Drouet1,2 *, V. Bosetti1,2,3 and M. Tavoni1,2,4 Strategies for dealing with climate change must incorporate and quantify all the relevant uncertainties, and be designed to manage the resulting risks1 . Here we employ the best available knowledge so far, summarized by the three working groups of the Fifth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC AR5; refs 2–4), to quantify the uncertainty of mitigation costs, climate change dynamics, and economic damage for alternative carbon budgets. We rank climate policies according to dierent decision-making The literature on the role of uncertainty in climate policy making has mostly relied on either analytical or simplified integrated assessment models (IAMs), such as DICE (ref. 13). In such contexts, di erent decision-making criteria and preferences over risks have been shown to have a significant impact on the optimal abatement strategy14,15 . However, these exercises lack detail in the representation of the mitigation options and the climate dynamics. Larger-scale models, which capture the main interrelationships between human and natural systems, have How Should Robustness Be Defined for Water Systems Planning under Change? Jonathan D. Herman, S.M.ASCE1 ; Patrick M. Reed, Ph.D., A.M.ASCE2 ; Harrison B. Zeff3 ; and Gregory W. Characklis, Ph.D., M.ASCE4 Abstract: Water systems planners have long recognized the need for robust solutions capable of withstanding deviations from the conditions for which they were designed. Robustness analyses have shifted from expected utility to exploratory bottom-up approaches which identify vulnerable scenarios prior to assigning likelihoods. Examples include Robust Decision Making (RDM), Decision Scaling, Info-Gap, and Many-Objective Robust Decision Making (MORDM). We propose a taxonomy of robustness frameworks to compare and contrast these approaches based on their methods of (1) alternative generation, (2) sampling of states of the world, (3) quantification of robustness measures, and (4) sensitivity analysis to identify important uncertainties. Building from the proposed taxonomy, we use a regional urban water supply case study in the Research Triangle region of North Carolina to illustrate the decision-relevant consequences that emerge from each of these choices. Results indicate that the methodological choices in the taxonomy lead to the selection of substantially different planning alternatives, underscoring the importance of an informed definition of robustness. Moreover, the results show that some commonly employed methodo- logical choices and definitions of robustness can have undesired consequences when ranking decision alternatives. For the demonstrated test case, recommendations for overcoming these issues include: (1) decision alternatives should be searched rather than prespecified, (2) dominant uncertainties should be discovered through sensitivity analysis rather than assumed, and (3) a carefully elicited multivariate satisficing measure of robustness allows stakeholders to achieve their problem-specific performance requirements. This work emphasizes the importance of an informed problem formulation for systems facing challenging performance tradeoffs and provides a common vocabulary to link the robustness frameworks widely used in the field of water systems planning. DOI: 10.1061/(ASCE)WR.1943-5452.0000509. © 2015 American Society of Civil Engineers. 02/13/15.CopyrightASCE.Forpersonaluseonly;allrightsreserved. Editorial Coping with the Wickedness of Public Policy Problems: Approaches for Decision Making under Deep Uncertainty Jan H. Kwakkel Faculty of Technology, Policy and Management, Delft Univ. of Technol- ogy, Jaffalaan 5, 2628 BX Delft, Netherlands (corresponding author). E-mail: j.h.kwakkel@tudelft.nl Warren E. Walker Faculty of Technology, Policy and Management, Delft Univ. of Technology, Jaffalaan 5, 2628 BX Delft, Netherlands. E-mail: w.e .walker@tudelft.nl wicked problems can be profound, difficult if not impossible to re- verse, and result in lock-ins for future decision making. Planning and decision making in wicked problem situations should, there- fore, be understood as an argumentative process: in which the prob- lem formulation, a shared understanding of system functioning and how this gives rise to the problem, and the set of promising solutions, emerge gradually through debate among the involved decision makers and stakeholders (Dewulf et al. 2005). When even the problem formulation itself is uncertain and con- llrightsreserved. Editorial Coping with the Wickedness of Public Poli Approaches for Decision Making under Dee Jan H. Kwakkel Faculty of Technology, Policy and Management, Delft Univ. of Technol- ogy, Jaffalaan 5, 2628 BX Delft, Netherlands (corresponding author). E-mail: j.h.kwakkel@tudelft.nl Warren E. Walker Faculty of Technology, Policy and Management, Delft Univ. of Technology, Jaffalaan 5, 2628 BX Delft, Netherlands. E-mail: w.e .walker@tudelft.nl wicked problems can be profo verse, and result in lock-ins fo and decision making in wicke fore, be understood as an argum lem formulation, a shared un and how this gives rise to the solutions, emerge gradually t decision makers and stakehol When even the problem for allrightsreserved. Editorial Coping with the Wickedness of P Approaches for Decision Making Jan H. Kwakkel Faculty of Technology, Policy and Management, Delft Univ. of Technol- ogy, Jaffalaan 5, 2628 BX Delft, Netherlands (corresponding author). E-mail: j.h.kwakkel@tudelft.nl Warren E. Walker Faculty of Technology, Policy and Management, Delft Univ. of Technology, Jaffalaan 5, 2628 BX Delft, Netherlands. E-mail: w.e .walker@tudelft.nl Marjolijn Haasnoot Deltares, P.O. Box 177, 2600 MH Delft, Netherlands; Faculty of Technology, Policy and Management, Delft Univ. of Technology, Jaffalaan 5, 2628 BX Delft, Netherlands. E-mail: Marjolijn.Haasnoot@deltares.nl wicked verse, a and de fore, be lem fo and ho solution decisio Wh tested, that fac and lea pursuit (Herma facilita ersonaluseonly;allrightsreserved. Editorial Coping with the Wick Approaches for Decisi Jan H. Kwakkel Faculty of Technology, Policy and Management, Delft Univ ogy, Jaffalaan 5, 2628 BX Delft, Netherlands (correspon E-mail: j.h.kwakkel@tudelft.nl Warren E. Walker Faculty of Technology, Policy and Management, Del Technology, Jaffalaan 5, 2628 BX Delft, Netherlands. .walker@tudelft.nl Marjolijn Haasnoot Deltares, P.O. Box 177, 2600 MH Delft, Netherlands; Technology, Policy and Management, Delft Univ. of Technolo 5, 2628 BX Delft, Netherlands. E-mail: Marjolijn.Haasnoot DOI: 10.1061/(ASCE)WR.1943-5452.0000626 In many planning problems, planners face major ch coping with uncertain and changing physical condition rightASCE.Forpersonaluseonly;allrightsreserved. vulnerable scenarios prior to assigning likelihoo Many-Objective Robust Decision Making (MOR approaches based on their methods of (1) alternati and (4) sensitivity analysis to identify important case study in the Research Triangle region of No choices. Results indicate that the methodological underscoring the importance of an informed defi logical choices and definitions of robustness can test case, recommendations for overcoming the (2) dominant uncertainties should be discovered satisficing measure of robustness allows stakeho the importance of an informed problem formulatio to link the robustness frameworks widely used © 2015 American Society of Civil Engineers. Introduction Decision makers in water resources systems aim ple performance objectives in the projected future robust to deviations from these projections. R context broadly refers to “the insensitivity of errors, random or otherwise, in the estimates of affecting design choice” (Matalas and Fiering specific mathematical implementations of this c matically. Extensive studies have demonstrated t decision makers to sacrifice expected performan bustness to uncertainty (Hitch 1960; Maass et 1979; Schneller and Sphicas 1983; Walker et a 2002; Clímaco 2004; Walker et al. 2013; DiFran 2014), signaling a departure from traditional requiring a priori aggregation of costs and benef of simultaneously navigating these goals in com led to the rise of a posteriori decision supp 1 School of Civil and Environmental Engineering, Cornell Univ., Ithaca, NY 14853 (corresponding author cornell.edu 2 Professor, School of Civil and Environme 211 Hollister Hall, Cornell Univ., Ithaca, NY 14853. 3 Dept. of Environmental Sciences and Engineeri Carolina, Chapel Hill, NC 27599. 4 Professor, Dept. of Environmental Sciences and E North Carolina, Chapel Hill, NC 27599. Note. This manuscript was submitted on September 1 December 3, 2014; published online on February 10, 201 open until July 10, 2015; separate discussions must be sub papers. This paper is part of the Journal of Water Reso Management, © ASCE, ISSN 0733-9496/04015012(14 © ASCE Downloadedfromascelibrary.orgbyLi.Co.Sa8181901/mi/155985on02/13/15.CopyrightASCE.Forpersonaluseonly;allrights vulnerable scenarios prior to assigning likelihoods. Examples include Robust Decision Making (RDM), Decision Scaling, Info-Gap, and Many-Objective Robust Decision Making (MORDM). We propose a taxonomy of robustness frameworks to compare and contrast these approaches based on their methods of (1) alternative generation, (2) sampling of states of the world, (3) quantification of robustness measures, and (4) sensitivity analysis to identify important uncertainties. Building from the proposed taxonomy, we use a regional urban water supply case study in the Research Triangle region of North Carolina to illustrate the decision-relevant consequences that emerge from each of these choices. Results indicate that the methodological choices in the taxonomy lead to the selection of substantially different planning alternatives, underscoring the importance of an informed definition of robustness. Moreover, the results show that some commonly employed methodo- logical choices and definitions of robustness can have undesired consequences when ranking decision alternatives. For the demonstrated test case, recommendations for overcoming these issues include: (1) decision alternatives should be searched rather than prespecified, (2) dominant uncertainties should be discovered through sensitivity analysis rather than assumed, and (3) a carefully elicited multivariate satisficing measure of robustness allows stakeholders to achieve their problem-specific performance requirements. This work emphasizes the importance of an informed problem formulation for systems facing challenging performance tradeoffs and provides a common vocabulary to link the robustness frameworks widely used in the field of water systems planning. DOI: 10.1061/(ASCE)WR.1943-5452.0000509. © 2015 American Society of Civil Engineers. Introduction Decision makers in water resources systems aim to achieve multi- ple performance objectives in the projected future while remaining robust to deviations from these projections. Robustness in this context broadly refers to “the insensitivity of system design to errors, random or otherwise, in the estimates of those parameters affecting design choice” (Matalas and Fiering 1977), although specific mathematical implementations of this concept differ dra- matically. Extensive studies have demonstrated the willingness of decision makers to sacrifice expected performance to improve ro- bustness to uncertainty (Hitch 1960; Maass et al. 1962; Bonder 1979; Schneller and Sphicas 1983; Walker et al. 2001; Lempert 2002; Clímaco 2004; Walker et al. 2013; DiFrancesco and Tullos 2014), signaling a departure from traditional decision theory requiring a priori aggregation of costs and benefits. The challenge of simultaneously navigating these goals in complex systems has led to the rise of a posteriori decision support, reviewed by Tsoukias (2008), in which the identification of decision alternatives and vulnerable states is preceded by data gathering, numerical modeling, and optimization. This process represents constructive learning with stakeholder feedback (Roy 1971, 1990) in which problem formulations compete as multiple working hypotheses (Chamberlin 1890). Given a set of alternatives shown to be near-optimal in the best available projections of the expected future state of the world, it remains a challenge to perform a posteriori alternative selection according to a defensible robustness criterion. Several competing criteria have been proposed, and their consequences for decision making warrant comparison. The term a posteriori decision support is drawn from the multi- objective optimization literature, referring to the generation of de- cision alternatives through computational search before imposing stakeholder preference on the problem (Cohon and Marks 1973, 1975), also known as generate-first-choose-later (GFCL) in the systems engineering literature (e.g., Hwang et al. 1979; Crossley et al. 1999; Balling and Richard 2000; Messac and Mattson 2002; Reynoso-Meza et al. 2014). Historically, this approach has stood in contrast to a priori weighted preference aggregation (Reuss 2003; Banzhaf 2009), which allows simpler solution tech- niques but requires decision makers to accurately assign prefer- ence weighting prior to reviewing alternatives (Bond et al. 2008, 2010). The salient implication for robustness analysis is that a sin- gle decision alternative produced by weighted aggregation, while promising in the projected future, may fail under deviations from this projection. A multiobjective, a posteriori approach provides a flexible set of alternatives that may be evaluated according to their robustness. In a related context, the concept of a posteriori decision support can be extended to the selection of scenarios, or states of the world, for decision making under uncertainty. In a traditional scenario 1 School of Civil and Environmental Engineering, 207 Hollister Hall, Cornell Univ., Ithaca, NY 14853 (corresponding author). E-mail: jdh366@ cornell.edu 2 Professor, School of Civil and Environmental Engineering, 211 Hollister Hall, Cornell Univ., Ithaca, NY 14853. 3 Dept. of Environmental Sciences and Engineering, Univ. of North Carolina, Chapel Hill, NC 27599. 4 Professor, Dept. of Environmental Sciences and Engineering, Univ. of North Carolina, Chapel Hill, NC 27599. Note. This manuscript was submitted on September 19, 2014; approved on December 3, 2014; published online on February 10, 2015. Discussion period open until July 10, 2015; separate discussions must be submitted for individual papers. This paper is part of the Journal of Water Resources Planning and Management, © ASCE, ISSN 0733-9496/04015012(14)/$25.00. © ASCE 04015012-1 J. Water Resour. Plann. Manage. J. Water Resour. Plann. Manage. Downloadedfromascelibrary.orgbyLi.Co.Sa8181901/mi/155985on02/13/15.CopyrightASCE.Forpersonaluseonly;allrights How Should Robus Plan Jonathan D. Herman Harrison B. Zeff Abstract: Water systems planners have long reco for which they were designed. Robustness analy vulnerable scenarios prior to assigning likelihoo Many-Objective Robust Decision Making (MOR approaches based on their methods of (1) alternati and (4) sensitivity analysis to identify important case study in the Research Triangle region of No choices. Results indicate that the methodological underscoring the importance of an informed defi logical choices and definitions of robustness can test case, recommendations for overcoming the (2) dominant uncertainties should be discovered satisficing measure of robustness allows stakeho the importance of an informed problem formulatio to link the robustness frameworks widely used © 2015 American Society of Civil Engineers. Introduction Decision makers in water resources systems aim ple performance objectives in the projected future robust to deviations from these projections. R context broadly refers to “the insensitivity of errors, random or otherwise, in the estimates of affecting design choice” (Matalas and Fiering specific mathematical implementations of this c matically. Extensive studies have demonstrated t decision makers to sacrifice expected performan bustness to uncertainty (Hitch 1960; Maass et 1979; Schneller and Sphicas 1983; Walker et a 2002; Clímaco 2004; Walker et al. 2013; DiFran 2014), signaling a departure from traditional requiring a priori aggregation of costs and benef of simultaneously navigating these goals in com led to the rise of a posteriori decision supp 1 School of Civil and Environmental Engineering, Cornell Univ., Ithaca, NY 14853 (corresponding author cornell.edu 2 Professor, School of Civil and Environme 211 Hollister Hall, Cornell Univ., Ithaca, NY 14853. 3 Dept. of Environmental Sciences and Engineeri Carolina, Chapel Hill, NC 27599. 4 Professor, Dept. of Environmental Sciences and E North Carolina, Chapel Hill, NC 27599. Note. This manuscript was submitted on September 1 December 3, 2014; published online on February 10, 201 open until July 10, 2015; separate discussions must be sub papers. This paper is part of the Journal of Water Reso Management, © ASCE, ISSN 0733-9496/04015012(14 © ASCE Downloadedfromascelibrary.orgbyLi.Co.Sa8181901/mi/155985on02/13/15.CopyrightASCE.Forpersonaluseonly;allrightsreserved. How Should Robustness Be Defined for Water Systems Planning under Change? Jonathan D. Herman, S.M.ASCE1 ; Patrick M. Reed, Ph.D., A.M.ASCE2 ; Harrison B. Zeff3 ; and Gregory W. Characklis, Ph.D., M.ASCE4 Abstract: Water systems planners have long recognized the need for robust solutions capable of withstanding deviations from the conditions for which they were designed. Robustness analyses have shifted from expected utility to exploratory bottom-up approaches which identify vulnerable scenarios prior to assigning likelihoods. Examples include Robust Decision Making (RDM), Decision Scaling, Info-Gap, and Many-Objective Robust Decision Making (MORDM). We propose a taxonomy of robustness frameworks to compare and contrast these approaches based on their methods of (1) alternative generation, (2) sampling of states of the world, (3) quantification of robustness measures, and (4) sensitivity analysis to identify important uncertainties. Building from the proposed taxonomy, we use a regional urban water supply case study in the Research Triangle region of North Carolina to illustrate the decision-relevant consequences that emerge from each of these choices. Results indicate that the methodological choices in the taxonomy lead to the selection of substantially different planning alternatives, underscoring the importance of an informed definition of robustness. Moreover, the results show that some commonly employed methodo- logical choices and definitions of robustness can have undesired consequences when ranking decision alternatives. For the demonstrated test case, recommendations for overcoming these issues include: (1) decision alternatives should be searched rather than prespecified, (2) dominant uncertainties should be discovered through sensitivity analysis rather than assumed, and (3) a carefully elicited multivariate satisficing measure of robustness allows stakeholders to achieve their problem-specific performance requirements. This work emphasizes the importance of an informed problem formulation for systems facing challenging performance tradeoffs and provides a common vocabulary to link the robustness frameworks widely used in the field of water systems planning. DOI: 10.1061/(ASCE)WR.1943-5452.0000509. © 2015 American Society of Civil Engineers. Introduction Decision makers in water resources systems aim to achieve multi- ple performance objectives in the projected future while remaining robust to deviations from these projections. Robustness in this context broadly refers to “the insensitivity of system design to errors, random or otherwise, in the estimates of those parameters affecting design choice” (Matalas and Fiering 1977), although specific mathematical implementations of this concept differ dra- matically. Extensive studies have demonstrated the willingness of decision makers to sacrifice expected performance to improve ro- bustness to uncertainty (Hitch 1960; Maass et al. 1962; Bonder 1979; Schneller and Sphicas 1983; Walker et al. 2001; Lempert 2002; Clímaco 2004; Walker et al. 2013; DiFrancesco and Tullos 2014), signaling a departure from traditional decision theory requiring a priori aggregation of costs and benefits. The challenge of simultaneously navigating these goals in complex systems has led to the rise of a posteriori decision support, reviewed by Tsoukias (2008), in which the identification of decision alternatives and vulnerable states is preceded by data gathering, numerical modeling, and optimization. This process represents constructive learning with stakeholder feedback (Roy 1971, 1990) in which problem formulations compete as multiple working hypotheses (Chamberlin 1890). Given a set of alternatives shown to be near-optimal in the best available projections of the expected future state of the world, it remains a challenge to perform a posteriori alternative selection according to a defensible robustness criterion. Several competing criteria have been proposed, and their consequences for decision making warrant comparison. The term a posteriori decision support is drawn from the multi- objective optimization literature, referring to the generation of de- cision alternatives through computational search before imposing stakeholder preference on the problem (Cohon and Marks 1973, 1975), also known as generate-first-choose-later (GFCL) in the systems engineering literature (e.g., Hwang et al. 1979; Crossley et al. 1999; Balling and Richard 2000; Messac and Mattson 2002; Reynoso-Meza et al. 2014). Historically, this approach has stood in contrast to a priori weighted preference aggregation (Reuss 2003; Banzhaf 2009), which allows simpler solution tech- niques but requires decision makers to accurately assign prefer- ence weighting prior to reviewing alternatives (Bond et al. 2008, 2010). The salient implication for robustness analysis is that a sin- gle decision alternative produced by weighted aggregation, while promising in the projected future, may fail under deviations from this projection. A multiobjective, a posteriori approach provides a flexible set of alternatives that may be evaluated according to their robustness. In a related context, the concept of a posteriori decision support can be extended to the selection of scenarios, or states of the world, for decision making under uncertainty. In a traditional scenario 1 School of Civil and Environmental Engineering, 207 Hollister Hall, Cornell Univ., Ithaca, NY 14853 (corresponding author). E-mail: jdh366@ cornell.edu 2 Professor, School of Civil and Environmental Engineering, 211 Hollister Hall, Cornell Univ., Ithaca, NY 14853. 3 Dept. of Environmental Sciences and Engineering, Univ. of North Carolina, Chapel Hill, NC 27599. 4 Professor, Dept. of Environmental Sciences and Engineering, Univ. of North Carolina, Chapel Hill, NC 27599. Note. This manuscript was submitted on September 19, 2014; approved on December 3, 2014; published online on February 10, 2015. Discussion period open until July 10, 2015; separate discussions must be submitted for individual papers. This paper is part of the Journal of Water Resources Planning and Management, © ASCE, ISSN 0733-9496/04015012(14)/$25.00. © ASCE 04015012-1 J. Water Resour. Plann. Manage. J. Water Resour. Plann. Manage. Downloadedfromascelibrary.orgbyLi.Co.Sa8181901/mi/155985on02/13/15.CopyrightASCE.Forpersonaluseonly;allrightsreserved.
  8. 8. EWRI2016 NRM How robustness is implemented? A metric 𝛙 is introduced for filtering the uncertainty in the scenarios w∈𝚵
  9. 9. EWRI2016 NRM A metric 𝛙 is introduced for filtering the uncertainty in the scenarios w∈𝚵 How robustness is implemented? • Maximin • Optimism-pessimism rule • Principle of insufficient reason • Regret - deviation from best • Regret - deviation from baseline • Signal to noise • Mean and standard deviation separately • Fraction of cases that passes thresholds the definition of the robustness metric introduces another uncertain parameter in the problem
  10. 10. 1. How to decide how to decide?
  11. 11. 1. How to decide how to decide? 2. What are the impacts of mis-defining the robustness metric?
  12. 12. 1. How to decide how to decide? 2. What are the impacts of mis-defining the robustness metric? 3. What happens if these metrics evolve in time?
  13. 13. EWRI2016 NRM The Lake Como system Como Adda River Milano Lake Como 160 33 66 100 km 4000 m 8 agricultural districts
  14. 14. EWRI2016 NRM The Lake Como system Como Adda River Milano Lake Como 160 33 66 100 km 4000 m 8 agricultural districts Flood control in Como
  15. 15. EWRI2016 NRM The Lake Como system Como Adda River Milano Lake Como 160 33 66 100 km 4000 m 8 agricultural districts Flood control in Como Irrigation supply
  16. 16. EWRI2016 NRM Como Adda River Milano Lake Como 160 33 66 100 km 4000 m 8 agricultural districts Flood control in Como Irrigation supply The Lake Como system
  17. 17. EWRI2016 NRM Como Adda River Milano Lake Como 160 33 66 100 km 4000 m 8 agricultural districts Flood control in Como Irrigation supply The Lake Como system critical period with water deficit
  18. 18. EWRI2016 NRM critical period with water deficitClimate change scenarios
  19. 19. Numerical results
  20. 20. EWRI2016 NRM Policy performance under historical climate flooding (storage reliability) 0.7 0.75 0.8 0.85 0.9 0.95 1 irrigation(volumetricreliability) 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 40% 14% 46% co-benefit degradation single-objective benefit Pareto approximate set over history Re-evaluation over the 28-scenarios ensemble Legend
  21. 21. EWRI2016 NRM Policy performance under climate change flooding (storage reliability) 0.7 0.75 0.8 0.85 0.9 0.95 1 irrigation(volumetricreliability) 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 40% 14% 46% co-benefit degradation single-objective benefit Pareto approximate set over history Re-evaluation over the 28-scenarios ensemble Legend
  22. 22. EWRI2016 NRM The impacts of misdefining robustness: modification of tradeoffs
  23. 23. EWRI2016 NRM The impacts of misdefining robustness: underestimation of system performance
  24. 24. EWRI2016 NRM Multi-robustness policy design (a): flooding (storage reliability) (b): irrigation (volumetric reliability) insufficient reason maximin (pessimism) maximax (optimism) optimism/pessimism rule minimax regret directionofpreference maximincriterion directionofpreference 0.997 0.985 1.00 0.990 0.005 0.550 0.270 0.860 0.484 0.726 0.806 0.620 0.970 0.719 0.167 0.712 0.505 0.854 0.633 0.283 a⇤ = arg max a [ 1 ⌅(f(a, w)), . . . , N ⌅ (f(a, w))]
  25. 25. EWRI2016 NRM Robustness evolution in time policy P1 level[m] J F M A M J J A S O N D0.7 0.72 0.74 0.76 0.78 0.8 0.82 0.5 0.55 0.6 0.65 P1 P2 P3 P5 P4 irrigation (vol. rel.) insufficient reason irrigation(vol.rel.) maximin(pessimism) policy P3 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 1.2 -0.4 0.0 0.4 0.8 policy P2 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 policy P1 level[m] J F M A M J J A S O N D0.76 0.78 0.8 0.82 P1 P2 P3 P5 P4 on (vol. rel.) cient reason 1.2 -0.4 0.0 0.4 0.8 J F M A M J J A S O N D0.7 0.72 0.74 0.76 0.78 0.8 0.82 0.5 irrigation (vol. rel.) insufficient reason policy P3 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 policy P4 J F M A M J J A S O N D 1.2 0.4 0.0 0.4 0.8 policy P5 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 -0.4 policy P2 J F M A M J J A S O N D 1.2 0.4 0.0 0.4 0.8 release [m3/s] 0 143 water demand Legend
  26. 26. EWRI2016 NRM irrigation (vol. rel.) insufficient reason policy P4 1.2 policy P2 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 Robustness evolution in time policy P1 level[m] J F M A M J J A S O N D0.7 0.72 0.74 0.76 0.78 0.8 0.82 0.5 0.55 0.6 0.65 P1 P2 P3 P5 P4 irrigation (vol. rel.) insufficient reason irrigation(vol.rel.) maximin(pessimism) policy P3 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 1.2 -0.4 0.0 0.4 0.8 policy P2 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 J F M A M J J A S O N D0.7 0.72 0.74 0.76 0.78 0.8 0.82 0.5 irrigation (vol. rel.) insufficient reason policy P3 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 policy P4 J F M A M J J A S O N D 1.2 0.4 0.0 0.4 0.8 policy P5 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 -0.4 policy P2 J F M A M J J A S O N D 1.2 0.4 0.0 0.4 0.8 release [m3/s] 0 143 water demand Legend
  27. 27. EWRI2016 NRM on (vol. rel.) cient reason policy P3 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 olicy P4 policy P5 1.2 olicy P2 J J A S O N D Robustness evolution in time policy P1 level[m] J F M A M J J A S O N D0.7 0.72 0.74 0.76 0.78 0.8 0.82 0.5 0.55 0.6 0.65 P1 P2 P3 P5 P4 irrigation (vol. rel.) insufficient reason irrigation(vol.rel.) maximin(pessimism) policy P3 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 1.2 -0.4 0.0 0.4 0.8 policy P2 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 J F M A M J J A S O N D0.7 0.72 0.74 0.76 0.78 0.8 0.82 0.5 irrigation (vol. rel.) insufficient reason policy P3 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 policy P4 J F M A M J J A S O N D 1.2 0.4 0.0 0.4 0.8 policy P5 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 -0.4 policy P2 J F M A M J J A S O N D 1.2 0.4 0.0 0.4 0.8 release [m3/s] 0 143 water demand Legend
  28. 28. EWRI2016 NRM policy P4 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 J F M A M J J A S O N D -0.4 Legend Robustness evolution in time policy P1 level[m] J F M A M J J A S O N D0.7 0.72 0.74 0.76 0.78 0.8 0.82 0.5 0.55 0.6 0.65 P1 P2 P3 P5 P4 irrigation (vol. rel.) insufficient reason irrigation(vol.rel.) maximin(pessimism) policy P3 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 1.2 -0.4 0.0 0.4 0.8 policy P2 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 J F M A M J J A S O N D0.7 0.72 0.74 0.76 0.78 0.8 0.82 0.5 irrigation (vol. rel.) insufficient reason policy P3 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 policy P4 J F M A M J J A S O N D 1.2 0.4 0.0 0.4 0.8 policy P5 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 -0.4 policy P2 J F M A M J J A S O N D 1.2 0.4 0.0 0.4 0.8 release [m3/s] 0 143 water demand Legend
  29. 29. EWRI2016 NRM J F M A M J J A S O N D -0.4 olicy P4 J J A S O N D policy P5 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 J J A S O N D egend Robustness evolution in time policy P1 level[m] J F M A M J J A S O N D0.7 0.72 0.74 0.76 0.78 0.8 0.82 0.5 0.55 0.6 0.65 P1 P2 P3 P5 P4 irrigation (vol. rel.) insufficient reason irrigation(vol.rel.) maximin(pessimism) policy P3 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 1.2 -0.4 0.0 0.4 0.8 policy P2 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 J F M A M J J A S O N D0.7 0.72 0.74 0.76 0.78 0.8 0.82 0.5 irrigation (vol. rel.) insufficient reason policy P3 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 policy P4 J F M A M J J A S O N D 1.2 0.4 0.0 0.4 0.8 policy P5 level[m] J F M A M J J A S O N D 1.2 -0.4 0.0 0.4 0.8 -0.4 policy P2 J F M A M J J A S O N D 1.2 0.4 0.0 0.4 0.8 release [m3/s] 0 143 water demand Legend
  30. 30. EWRI2016 NRM Is robustness really robust?
  31. 31. EWRI2016 NRM Is robustness really robust? NO, because • the definition of the robustness metric represents an additional uncertain parameter of the problem
  32. 32. EWRI2016 NRM Is robustness really robust? NO, because • the definition of the robustness metric represents an additional uncertain parameter of the problem • the robustness metric should be calibrated to capture the attitude of the real DM or multiple metrics should be considered
  33. 33. EWRI2016 NRM Is robustness really robust? NO, because • the definition of the robustness metric represents an additional uncertain parameter of the problem • the robustness metric should be calibrated to capture the attitude of the real DM or multiple metrics should be considered • possible dynamic changes in DM attitudes and preferences might further impact on decisions under uncertainty
  34. 34. EWRI2016 NRM Is robustness really robust? NO, because • the definition of the robustness metric represents an additional uncertain parameter of the problem • the robustness metric should be calibrated to capture the attitude of the real DM or multiple metrics should be considered • possible dynamic changes in DM attitudes and preferences might further impact on decisions under uncertainty Comprehensive analysis including additional metrics and testing on different case study applications
  35. 35. thank you http://giuliani.faculty.polimi.it www.nrm.deib.polimi.it @MxgTeo @NRMPolimi Matteo Giuliani matteo.giuliani@polimi.itREFERENCES: M, Giuliani and A. Castelletti (2016). Is robustness really robust? How different definitions of robustness impact decision-making under climate change, Climatic Change, 136: 409-424.

×