Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Policy Making using Modelling in a Complex world

483 views

Published on

A talk given at the CECAN workshop, London July 2016

Abstract:

The consequences of complexity in the real world are discussed together with some meaningful ways of understanding and managing such situations. The implications of such complexity are that many social systems are fundamentally unpredictable by nature, especially when in the presence of structural change (transitions). This implies consequences for the way we model, but also for the way models are used in the policy process.

I discuss the problems arising from a too narrow focus on quantification in managing complex systems, in particular those of optimisation. I criticise some of the approaches that ignore these difficulties and pretend to approximately forecast using the impact of policy options using over-simple models. However, lack of predictability does not automatically imply a lack of managerial possibilities. We will discuss how some insights and tools from "Complexity Science" can help with such management. Managing complex systems requires a good understanding of the dynamics of the system in question - to know, before they occur, some of the real possibilities that might occur and be ready so they can be reacted to as responsively as possible. Agent based simulation will be discussed as a tool that is suitable for this task, especially in conjunction with model-informed data visualisation.

Published in: Science
  • Be the first to comment

Policy Making using Modelling in a Complex world

  1. 1. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 1 Policy Making using Modelling in a Complex world Bruce Edmonds Centre for Policy Modelling Manchester Metropolitan University
  2. 2. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 2 or… towards what might be in a ‘Cyan Book’ (halfway between the aqua and green books)
  3. 3. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 3 Introduction Part 0
  4. 4. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 4 Simple systems… … may be complicated but behave in predictable ways, allowing them to be represented by models... • where one can use them to numerically forecast • where uncertainty can be analytically estimated • where one can get rough estimates cheaply, and better estimates with increasing investment • which one can sensibly plan and execute ot systematically • where there is a basically one right way of doing it • so that one can fully understand the model
  5. 5. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 5 A double pendulum Even with only two bits of wood the result can be complex An Example Video is at: http://www.youtube.com/watch?v=czLIj-4suOk A trace of the motion:
  6. 6. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 6 The Main Point of the Talk… …is that complex systems need to be dealt with in a different way to that of simple systems... ...not only modelled in a different way using different techniques but also how models about complex systems are used in any policy development process needs to change. • Complex modelling will be increasingly important as we try to develop better policies and deal with complex and fast moving situations • But it can not be ‘business as usual’ – just doing better modelling with the same modeller–policy actor relationship will not work well
  7. 7. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 7 Structure of the (rest of the) Talk 1. A bit about modelling context, purposes and tensions 2. Some of the underlying assumptions and habits that need to change 3. An example model – Stefano Picascia’s Modelling of the Housing Rental Market 4. Some suggestions as to ways forward
  8. 8. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 8 Modelling, its context, purposes and tensions Part 1
  9. 9. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 9 Some of the Context • An increasingly professionalised coterie of skilled modellers within government (GORS etc.) • Leading to new standards for modelling and analysis within government – the ‘Aqua Book’ – which lays down very sensible guidelines for model development and quality assurance • Within the context of the ‘Green Book’ – a framework for policy development and evaluation • Along side a set of academic and other experts increasingly willing to be involved in helping government model and analyse complex stuff
  10. 10. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 10 Different modelling purposes Models can be used for a wide variety of different purposes, and these impact upon the kind of techniques needed and its difficulties, e.g. • Forecasting – predicting unknown (e.g. future) situations and outcomes • Explanation – understanding how known outcomes might have come about • Theoretical Exploration – understanding a complex model by exploring some of its properties and behaviours • Analogy – using a model as a way of thinking about something else
  11. 11. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 11 A picture of modelling whatisobservedor measured themodel themodellers themodelusers
  12. 12. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 12 Model Scope • The scope of a model is the conditions under which it is useful for its planned purpose • Whilst this is implicit and stable for many simple systems, this is not the case for many complex ones • Thus trying to make scope explicit is important, and these relate to model assumptions • A process not modelled (and hence outside its scope) can overwhelm the results… • ..but in complex systems internal processes of change can also emerge, and some of these can be usefully modelled (but only in more complex ways)
  13. 13. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 13 Possible modelling trade-offs • Some desiderata for models: validity, formality, simplicity and generality • these are difficult to obtain simultaneously (for complex systems) • there is some sort of complicated trade-off between them (for each modelling exercise) simplicity generality validity formality Analogy Solvable Mathematical Model Data What Policy Actors Want
  14. 14. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 14 Some underlying assumptions and habits Part 2
  15. 15. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 15 The Green Book • Basically a formulation of cost-benefit analysis • As if an economist had written a manual for policy actors in how to think (i.e. as their theory states) • It does have some useful advice on generating and considering alternatives and trying to judge uncertainty But does assume that one can: 1. list the main alternative options 2. forecast the results of these 3. put meaningful numerical values on these
  16. 16. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 16 Quantification • Implicit throughout the Green Book approach • Makes life much easier for policy actors • Especially when asked to justify an approach • But can be more misleading than helpful because it gives a false impression of accuracy • And implicitly leads to a focus on the measurable and that things will ‘average out’ etc. • Was a limitation of purely mathematical approaches, but computer simulation does not have to be focused on these aspects • Collecting qualitatively different possible outcomes might be much more useful
  17. 17. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 17 No gradual approximation, but scope-limited usefulness It is often assumed that as time and effort increase the accuracy of the results improve, but this is not the case with complex systems and models Rather in order for the outcomes to be within scope enough iterative development has to occur Before this the results are worse than nothing Time and cost Error
  18. 18. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 18 That (real, complex) systems ever get near an equilibrium • From economics, there is an obsession with equilibrium in much modelling • Where, even when it is known equilibria are not observed, they are assumed in order to forecast • Most macro economic models (and many cost-based planning models) do this • In complex systems, even when they are in near- equilibrium state, they can move away from this • Good when things are not changing very much, almost useless for turning points or when structural change is occurring • (See forthcoming open-access book “Non-Equilibrium Social Science”)
  19. 19. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 19 The Aqua Book • Is a standard for the development and quality assurance of modelling/analysis for policy • Has a lot of very sensible advice (I wish that my academic colleagues had similar standards) • It distinguishes different roles involved in the modelling process, which clarify the chain of command/assurance and recommends extensive and frequent communication between them • However, it only talks about the responsibilities of the modelling team, as if modelling can be an ‘on demand service’ for policy actors • Mostly it is written for the cases where the model or situation is basically simple • See my recent review in JASSS (jasss.surrey.ac.uk)
  20. 20. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 20 That different models should agree • Sometimes policy actors complain that different models of the same phenomena disagree • But this is inevitable where different models are taking different approaches and making different assumptions – the results are relative to these • One can try and force models to agree, but in the process one eliminates the variety of modelling assumptions (in return for an illusion of certainty) • In risk-analyses one wants variety so that it is more likely that future possibilities can be anticipated
  21. 21. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 21 Planning and Managing Modelling • In a simple case one can apply an approach (as described in the Aqua book) where one carefully plans, manages and evaluates models • As if this was like building a bridge! • But in complex cases complications about what needs to be included or not requires a more iterative approach… • ...where models are repeatedly built for a purpose and the lessons learnt as you go along... • Becuase the difficulties can not be predicted in complex cases!
  22. 22. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 22 Compartmentalism • That some problems can be separated into smaller sub-problems which can be modelled more simply • Not true in many complex cases, where the scope of modelling is dependent on having enough of the key processes represented • Sometimes several different modelling approaches with different (but overlapping) assumptions can be more helpful • Just fiddling, incrementally expanding an existing (and failing) model will probably not help here
  23. 23. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 23 Instrumental Institutionalisation • Even a wrong or clearly inadequate model can be a useful part of a policy development process if the model is flexible enough • Because when used repeatedly its limitations and oddities can be fixed with institutional ‘kludges’ • Becoming something more like a consistent ‘base line’ from which policy can be debated • e.g. the standard transport models • But as these are instutionalised, they are increasingly difficult to change because the other policy processes have adapted to it • Resulting in a situation similar to the apocryphal ‘slowly boiled frog’ story – at each time the motivation to change is less than the perceived advantage of doing so, in which case wait for a crisis to occur!
  24. 24. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 24 An Example: A Model of the Rental Housing Market Part 3
  25. 25. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 25 The model • By Stefano Picascia, an PhD student of mine, now at Sienna University, Italy • Is an agent-based simulation that represents both tenants and developers co-adapting • Is geographically based with tenants making decisions as where to move to based on location as well as quality of housing and price • Developers put in captial to build/rennovate housing for tenants • Rents are determined by the quality and prices of surrounding housing
  26. 26. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 26 The Manchester Case Waves of price changes can spread Can have different outcomes each time it is run Has also been applied to London and Beirut Video of model running is at: http://www.youtube.com/watch?v=PtYTtkPrACM
  27. 27. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 27 Average prices in a run
  28. 28. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 28 Different Sectors of the City in a run
  29. 29. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 29 What it does and does not tell us In the model (which is the private rental sector only): • That change is fundamentally internally driven as well as due to outside events • Price oscillations are endemic to the system • That some regions of cities will be stuck as low quality housing for long periods of time • The very high price regions stay that way • That under certain conditions sudden ‘gentrification’ may occur to some degree • For poorer districts decline is gradual and continual between any such periods
  30. 30. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 30 Some ways forward and conclusions for complex cases Part 4
  31. 31. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 31 From Probabilistic to Possibilistic • When outcomes can not be sensibly forecast… • And especially numerically forecast… • …where even probability zones or 90% bounds are misleading • Then moving to an approach that models and understand (more of) underlying processes... • ...in terms of the different kinds of outcome might be much more informative • Each outcome tagged with its own assumptions and scopes (if they differ)
  32. 32. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 32 From Forecasting to Risk Analysis • However much one might like forecasting, often it is simply not possible… • ...let alone in a way such that the outcomes from different options can be compared! • Predicting outcomes can be more misleading than helpful • Rather it may be more approapriate to use models for risk analysis – finding all the ways a policy might go wrong (or right!) • Techniques are available to help discover and understand how endogenous processes might result in different future possibilities • Which can then inform the design of ‘early warning’ monitors giving the most immediate feedback to policy makers
  33. 33. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 33 Informing the adaptive ‘driving’ of policy • Complex models are no good for policy makers! • Because they have to make decisions on grounds they understand and know the reliability of • They can not (and should not) delegate this to ‘experts’ and their inscrutable models • Rather modellers should use their modelling to understand the key emergent kinds of outcome • To inform: – the consideration of these kinds of outcome – the design of appropriate data visalisations – the design of ‘earl warning indicators • …So that policy can adapt to changing trends and events as quickly and fluidly as possible
  34. 34. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 34 Conclusions • Modelling of complex phenomena is not cheap or quick and requires iterative development • It will not forecast the impact of potential policies or events, but can anticipate possible future outcomes in a way intuition can not • There will always be a ‘scope’ – a set of conditions/assumptions a model depends upon • But a good model can repay its investment in terms of cost and improving people’s lives many, many times over
  35. 35. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 35 Summary It is no good wishing that the world or modelling is simple and trying to ‘force’ it to be so, one has to adapt to suit reality… …this includes how models and modelling are used by the policy process
  36. 36. Policy Making using Modelling in a Complex world, Bruce Edmonds, CECAN, London, 20th July 2016. slide 36 The End The Centre for Policy Modelling: http://cfpm.org These slides will be available at: http://slideshare.net/BruceEdmonds Stefano’s work was developed under this project, funded by the EPSRC, grant number EP/H02171X

×