Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May ...
Upcoming SlideShare
Loading in …5
×

Towards Integrating Everything (well at least: ABM, data-mining, qual&quant data, networks and complexity science)

498 views
397 views

Published on

A talk given at the SKIN3 workshop in Budapest, May 2014 (http://cress.soc.surrey.ac.uk/SKIN/events/third-skin-workshop)

Innovation or other policy-orientated research has tended to take one of two strategies: (a) work with high-level abstractions of macro-level variables or (b) focus on micro-level aspects/areas with simpler mechanisms. Whilst (a) may provide some comfort in the form of forecasts, these are almost useless for policy since they can only be relied upon if nothing much has changed. Although approach (b) may produce some interesting studies which show how complex even small aspects of the involved processes are, with maybe interesting emergent effects, it provides only a small part of the overall picture and little to guide decision making.

Rather, I (with others) suggest a different approach. Instead of aiming to produce some kind of "adequate" theory (usually in the form of a model along with its interpretation), that instead we aim at integrating different kinds of evidence and find the best ways to present these to policy makers in order to help policy-makers 'drive' by providing views of what is happening. Thus (1) utilising the greatest possible range of evidence and (2) providing rich, relevant but synthetic views of this evidence to the policy makers. Any projections should be 'possibilistic' rather than 'probabilistic' - showing the different ways in which social processes might unfold, and help inform the analysis of risks. The talk looks at some of the ways in which this might be done, to integrate micro-level narrative data, time-series data, survey data, network data, big data using a variety of techniques. In this view, models do not disappear, but rather have a different purpose and hence be developed and checked differently.

This shift will involve a change in attitude and approach from both researchers and those in the policy world. Researchers will have to give up the playing for general or abstract theory, satisfying themselves with more gentle and incremental abstraction, whilst also accepting and working with a greater variety of kinds of evidence. They will also have to stop 'conning' the policy world with forecasts, and refuse to provide these as more dangerous than helpful. The policy world will have to stop looking for a magic 'crutch' that will reduce uncertainty (or provide justification for chosen policies) and move towards greater openness with both data and models.

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
498
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Erik’s talk
  • From experience and Break-out session in a Complexity Science and Policy Conference in london
  • van de leew
  • Bert Droste-Franke’s indicates some of these difficulties
  • Many of the talks here
  • Not the Nancy Cartwright that does the voice of Marge in the Simpsons
  • like economic indicators: GDP, science score card of Muhamed and Matthias
  • Stephan Linger
  • Petra
  • qual2rule sig, SI etc.
  • Cara
  • CNR Pisa
  • Paper on validating networks
  • IRSTEA, CIRAD
  • Scott Moss
  • Kuhn’s spectacles
  • Towards Integrating Everything (well at least: ABM, data-mining, qual&quant data, networks and complexity science)

    1. 1. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 1 Towards Integrating Everything… Bruce Edmonds Centre for Policy Modelling Manchester Metropolitan University
    2. 2. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 2 …well at least: ABM, data-mining, qualitative & quantitative data, networks and complexity science
    3. 3. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 3 Complexity Science and Policy Do Not Mix Well.. Because (among other reasons): • The phenomena of interest to policy makers is complicated, complex and changing all the time and thus very hard to “understand” • Complexity Scientists and Policy People have very different goals (understanding vs. action) • They have to work in very different ways with respect to very different sub-cultures/peers • They are (mostly) unwilling to understand each others’ world • There is a tussle between them in terms of status and power (control)
    4. 4. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 4 Some example problems • “You have 3 months to give me your best forecast, however preliminary” • Policy makers are unwilling to outsource any control over the process to researchers • Important data is not available to researchers • Policy makers already know what they will do, they are just looking for a justification or story • Complex models make policy debate difficult • The outsourcing of blame: “The decision was made based on the best scientific advice” • The worry of researchers that the caveats underlying their models will be lost/ignored
    5. 5. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 5 This talk is about some different strategies for dealing with this • That is, to step back, and look at what we are doing in complexity science and how this might (or might not) ‘fit’ into the world of policy making and hence usefully inform policy • In other words, can our models and understanding be useful to society and, if so, how • Now, unfortunately we have only a little “data” about this relationship – there has been relatively little observational work on this • Thus I will proceed by looking at some of the issues and discussing possibilities only • This is a synthetic talk, amplifying some trends and issues already raised at this workshop then extrapolated to a conclusion (that not all will be happy with)
    6. 6. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 6 Strategy 0: A natural language discourse • Issue is discussed in rich, meaningful language • Any models are implicit and/or analogical • Meaning of terms is often vague or ambiguous – Advantages in terms of building consensus – Disadvantages when working out what went wrong • Debate is accessible to everyone • But lack of cumulative knowledge development • Depends upon “conceptual framework” and hence can be influenced by fashion • Not much good for anticipating radical change • Good for integrating diverse evidence albeit informally • But poor at dealing with complication/complex detail
    7. 7. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 7 Strategy 1: “predictive” “black-box” macro-level models • Model the system relating macro-level properties of the whole system (using differential equations, rate equations, systems dynamics etc.) • With a view to predicting the effects of different policy options (albeit with large error bounds) • It is possible to make such models, of social phenomena but understanding is often not the main way of doing this but trial&error – in other words model adaption (Nate Silver) • But this only works if nothing essential changes – these models only give “surprise free” projections
    8. 8. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 8 Strategy 2: partial, complex, micro- level models used as an analogy • Model some processes/aspects at the micro-level system to observe the emergence of outcomes • Does not easily relate to data, does not predict, and remains quite abstract from the observed • Can be used to understand/explore the possible model “trajectories” of outcomes • But the model remains more of an analogy, because the mapping to any observed case is unclear and remade by each interpreter • This gives a “story” why, but is difficult to relate to particular policy options/questions • Tends to give “negative” conclusions w.r.t. policy
    9. 9. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 9 Kinds of Laws • “How the Laws of Physics Lie” N. Cartwright, 1980 • Strategy 1 corresponds to “Phenomenological Laws”, they match/predict the observed data but do not explain • Strategy 2 corresponds to “Explanatory Laws”, they explain why things occur but do not predict • You need both kinds of model but crucially also the “bridging rules” learnt via acculturation connect the two – this is often not explicit • In a mature science formal derivations/theories can be made between the two kinds of model
    10. 10. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 10 An illustration: Modelling a gas T V P µT V The gas laws: A Phenomenological Model Picture of randomly moving, weakly interacting elastic spheres An Explanatory Model Bridging rules
    11. 11. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 11 Back to Complex Social Science • People are applying these two types separately, indeed they are often in different academic fields • The bridging rules have not been developed • Broadly the policy world wants the properties of strategy 1 models • …whilst complexity science likes strategy 2 • This reflects the difference in their goals and both of their wishes to retain control and avoid blame • But neither, on their own, will satisfy the joint needs of complexity and policy worlds, they are either: unable to deal with structural change or they only provide a kind of analogy as to what might happen
    12. 12. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 12 Robots in Uncertain Environments • AI/robotics in the 60s/70s applied an approach where the entity kept a model of its world and tried to evaluate different alternative actions via predicting their effects using the model • However these were not good at coping with unknown and uncertain environments • Rather what turned out to work much better was: – Using the world as a model of itself – frequent sampling – Fast adaption in response to immediate conditions – Different levels of abstraction and/or control (subsumption architecture)
    13. 13. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 13 Strategy 3: “consistent” views of what is happening • Not general “theories” of social phenomena • Rather well-understood and “consistent” abstraction from the data that gives a “view” that is designed for a particular purpose • Behind this there will be a (maybe implicit) model which explains how the resulting view is generated from the source data • But this model does not have to capture the processes within the observed phenomenon • And, crucially, it is not a projection forward in time but a view of the current situation
    14. 14. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 14 This can involve different kinds of data manipulation Including: • abstraction: – losing information to make patterns/trends more visible – selecting aspects relevant to the present purpose • synthesis: integrating data from different sources to produce: – a more complete picture – a more meaningful picture • re-expression: manipulating the data to: – suit the interpretative abilities of humans – show how key aspects/properties relate/change
    15. 15. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 15 Desiderata for such “views” • Timely: to be able to be produced reasonably quickly (once designed!), so it is practical to use • Adequate: to give enough information about the current state and (maybe) how it is changing • Consistent: in the sense that the same kind of situation is reliably recognisable from its presentation (given its purpose) • Transparent: The meaning of what they show needs to be clear enough to be learnable • Well-founded: in the sense that it is clear how to adapt it to new/changed data
    16. 16. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 16 Use of these “views” • The algorithm for constructing these views might well be derived from the results of extensive analysis, simulation modelling etc. • However, they are themselves a more straight- forward representation of the data with a clear and comprehensible account of their nature • They are used to help “steer” policy by giving a richer but timely sense of what is happening • Policy makers (maybe initially with advice from researchers) learn how to interpret the views • They “own” the view machinery and its use, not the researchers • The views might be publically available, so that their import may be discussed
    17. 17. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 17 An illustration of Strategy 3 in action Modelling micro- aspects Data analysis Expert opinion ABM and other analysis Understanding processual possibilities Measures, tools and visualisations Policy Decisions Consequences Wider Public Policy WorldResearch World
    18. 18. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 18 In order to realise this strategy… …we need to: • integrate the streams of evidence/data • discover what the possibilities in terms of social processes are (using data mining, simulation modelling etc.) • use this to focus on what would indicate the early onset of these processes occurring and their progress • present tools for these using a variety of means (visualisations, statistics, graphs, interactives etc.) • but in a targeted and relevant manner • then be willing to let go an others develop their use ABM and other techniques What is delivered to the policy world
    19. 19. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 19 Example: Understanding Innovation Innovation is a multi-level phenomena, including: • Individuals creating new things/ideas/innovations • The spread of these over networks • How innovations are received – how they ‘fit’ into the conceptual and pragmatic frames of the potential recipients • How products/ideas combine to deliver services • How some innovations are catalytic/tools – facilitate the creation of many more innovations • How innovations can change the affordances and habits of a whole society and hence influence its culture • etc. etc.
    20. 20. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 20 Narrative Data • Observational or personal accounts of what people do and their motivations are valuable source material, especially for the specification of the micro-level (agent behaviour) • There is a wealth of such evidence within qualitative research, albeit often “smothered” within (what seem to us as outsiders) obscure jargon, issues, debates and pretentions • This can be used to suggest some of the menu of strategies people use and when they use them • Are often relatively mundane and context-specific • They might be mistaken, but are often a much better starting point than the theories of academics • Methods for using such narrative data to inform the specification of agents are currently being worked upon • These can, to some extent, bring some of the “mess” of observed social life into ABM models
    21. 21. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 21 Psychological Evidence • Often the constructs we use to make our agents are ad hoc, coming from a “folk psychology” • They are not checked for consistency against what is known in psychology • Unfortunately (for us) the psychological literature is concerned with its own issues and debates and does not often give the answers we need • And it tends to look at different aspects in isolation and gives no clue as to how they relate/integrate • However it does provide weak constraints as to the specification of agent behaviour and a language within which to critique suggestions
    22. 22. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 22 big Data • can provide fine-grained but noisy and incomplete information about actions • data-mining these can provide insights about how people behave • but provide richer insights when analysed with, or compared against, other evidence and hypotheses • in particular, clustering of such data might indicate some of the different kinds of behaviour • each of which might be then abstracted to suggest some different behavioural rules
    23. 23. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 23 Survey Data • Existing surveys are often not very informative or useful, only providing veiled hints for what people might be doing/thinking • The detail data from the survey is more useful than the particular summaries presented in paper • However, once the relevant behavioural dimensions and strategies are known directed surveys can be useful to determine proportion of occurrence and conditions of occurrence
    24. 24. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 24 Network Data • Direct network data is only sparsely available and then often very incomplete • Or derived from big Data where the connection between links suggested by that data and social connections in the target phenomena is weak • But can be a useful validation check on the networks that emerge from ABMs • The relevant aspects of these networks could be compared with those in available data • However, comparing networks is non-trivial • Even when sampled in odd ways, the sampling can be mimicked within the ABM then compared
    25. 25. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 25 Participatory Validation • When results are presented/developed with stakeholders and/or subjects • Can provide valuable input as to the meso-level output/behaviours • Context is important to provided –histories and situations wherein the behaviour might occur so that participants can give meaningful responses • Often useful in getting indications of where a model might be wrong • Animations/visualisations/graphs are important to give a fuller picture to participants
    26. 26. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 26 Time Series Data • Are usually at an aggregate or group level • Gives an idea of the dynamics one is looking for • Are often quite derived/abstract Aggregate Statistics • Aggregate summaries of measures • Give different projections of the data • Are most useful when most closely linked to policy inputs or (un)desirable outcomes
    27. 27. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 27 Integration via ABMs to “Views” Micro-level Narrative data Psychology Data-mining Survey data Network data Participatory input Meso-level Macro-level Time-series Aggregate Statistics Survey summaries ABM etc. Archetypal Stories of Individuals Complex Visualisations Key Global Indicators DeliveredtoPolicyWorld Scenarios
    28. 28. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 28 Side Note: Social Context • An obsession of mine • Word “Context” is problematic, but roughly the perceived kind of situation (e.g. keynote lecture) • Context is not necessarily accessible to conscious thought, reifiable, consistent or abstractable • But some are socially entrenched and obvious • People behave differently in different contexts! • Identifying social contexts (via data mining, observation etc.) is key to integrating many data, particularly qualitative or specific data with more generic/abstract patterns
    29. 29. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 29 Being A Modeller • Status is largely accorded to higher end abstractions rather than the concrete/mundane A more balanced science Social Simulation/Complex ity Science Abstract constructions Concrete constructions • We like to maintain control over our own constructions including their subsequent use • We inevitably see the world through our models and so distort as we make progress
    30. 30. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 30 Some things I am suggesting to avoid • Giving policy makers/advisers predictions is as unwise as giving a sharp knife to a child – at best it will not be useful to them and at worst it could cause a horrible accident and it will be your fault • Just because a model provides a good way of thinking about things does not make it true. • Policy makers expect scientists to only present only (basically) correct models to them – not unvalidated speculations or computational analogies (ways of thinking about something). • Even though you find a model really interesting and revealing, do not attempt to share the model with those involved in policy – they have fundamentally different concerns and goals to you. This is not just a matter of expressing things clearly/simply enough.
    31. 31. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 31 Conclusions • The product should not be a model so much as a representation of what is happening • That is clear and the result of informed research • But then ceded into the policy development/assessment cycle, losing control over it • Thus to avoid relieving policy makers of the burden decision making and future uncertainty • In particular not to predict, even with error bars • but to inform them better as to what is happening • this will involve us shifting to data integration,
    32. 32. Towards Integrating Everything, Bruce Edmonds, Joining Complexity Science And Social Simulation For Policy, Budapest, May 2014. slide 32 The End! Bruce Edmonds: http://bruce.edmonds.name Centre for Policy Modelling: http://cfpm.org The slides will be/are uploaded at: http://slideshare.net/BruceEdmonds

    ×