Four different views of a policy model : an analysis and some suggestions

3,434 views

Published on

A policy model has (at least) four different interpretations: (a) intention: the intention/interpretation of the simulation designer/programmer, (b) validation: the meaning established by the validation of the model in terms of the mapping(s) to sets of evidence, (c) use: the meaning established as a result of the use of a model in a policy making/advice context and (d) interpretation: the narrative interpretation of the policy maker/advisor when justifying decisions made where this refers to a policy model.
These four different interpretations are loosely connected via social processes. The relation between intention and validation is relatively well discussed in the context of “scientific” model specification and development. The relation between use and interpretation has been discussed in a number of specific contexts. However when and how a relationship between the scientific world of intention/validation and the policy world of use/interpretation are established in practice is an area with little active research.
Both personal experience and philosophical considerations suggest that these two worlds are very different in terms of both purpose and method. However this does not mean that there cannot be any well-founded connection between them. The key question is understanding the social processes of how this can happen, what are the conditions that facilitate it happening and what is the nature of the relationship between the four views when it does happen.
Interestingly these issues have been faced and extensively discussed in the field of Artificial Intelligence, which has confronted the distinction between meaning of internal models (loosely, the beliefs of an agent about its environment) in these four ways. The field of AI has not come up with a final solution to these problems, and is itself divided into those that inhabit separate approaches that adopt a subset of these approaches to model meaning. However it is suggestive of some ways forward, namely:
• a recognition of the problem that there are these different ways of attributing meaning to a policy model (and hence avoid some common errors derived from conflating these four views);
• symbol grounding in the sense of learning meanings through repeated use and adjustment (either in response to validation or interpretation views or both);
• and the observation of scientific-policy interaction as it actually occurs (e.g. an ethnographic study of scientist/policy advisor interaction).
Some developments in the area of participatory policy modelling can be seen as forays into this arena, albeit without structured assessment.

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,434
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
28
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Tale of me and simulations and understanding them
  • Four different views of a policy model : an analysis and some suggestions

    1. 1. Four different views of a policy modelan analysis and some suggestions<br />Bruce EdmondsCentre for Policy ModellingManchester Metropolitan University<br />
    2. 2. Two Worlds<br />Research<br />Ultimate Goal is Agreement with Observed (Truth)<br />Modeller also has an idea of what the model is and how it works<br />Policy<br />Ultimate Goal is in Final Outcomes (Usefulness)<br />Decisions justified by a communicable causal story<br />Policy Advisor<br />Modeller <br />Policy Model<br /><ul><li>Labels/Documentation may be different from all of the above!</li></ul>Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 2<br />
    3. 3. Four Meanings (of the PM)<br />Research World<br />The researcher’s idea/intention for the PM<br />The fit of the PM with the evidence/data<br />The ideavalidation relation extensively discussed within research world<br />Policy World<br />The usefulness of the PM for decisions<br />The communicable story of the PM<br />The goalinterpretation relation extensively discussed within policy world<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 3<br />
    4. 4. Exemplars of the Divide between Researchers and Policy Makers/Advisors<br />Policy makers try to get researchers to predict the unpredictable<br />Researchers can’t get the input (time, money, data) they need from policy makers<br />Policy advisors want support for a policy (their best guess that it’s good) and researchers reply with caveats and complications<br />Policy Makers (over) simplify the results<br />Researchers make the model so complex nobody but they can understand it<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 4<br />
    5. 5. Two Models of Truth<br />Realism<br />Truth through relationship of output to observations<br />Feedback through correspondence (RMSE error, stylized facts, knowledge about processes)<br />ForecastObserve loop<br />Often associated with hard sciences, numbers and analytic models<br />Instrumentalism<br />Truth through trying something and assessing how well it works<br />Feedback through success (cost, votes, pleasure, pain)<br />ActionAssess loop<br />Often associated with the humanities, narratives, politics, perspective and aggregate statistics<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 5<br />
    6. 6. The Anti-AnthropocentricAssumption<br />That the universe is not arranged for our benefit(as academics trying to understand it)<br />That assumptions as these are likely to be wrong:<br />Our planet is the centre of the universe<br />Planetary orbits are circles<br />Risky events follow a normal distribution<br />But here… that the social phenomena we study happen to be such that numerical and other easy ‘surface’ methods of understanding will be sufficient<br />In other words, we can not side-step the difficulties with some clever proxies/mathematics/statistics/etc.<br />Simplicity (what is easier) is not any guide to truth <br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 6<br />
    7. 7. Consequences of the AAA<br />There is no reason to suppose that a model that is adequate to modelling policy issues will be simple enough for us to understand<br />(e.g. that it will be analytically solvable)<br /><ul><li>We may well have to abandon hope of genericmodels and settle for context-specific approaches
    8. 8. We will need to use a combination of different approaches
    9. 9. We will need to model our models etc. to have a change of understanding them</li></ul>Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 7<br />
    10. 10. Some modelling tensions<br />precision (model not vague)<br />Economic Models<br />Agent-basedmodels<br />generalityof scope<br />Scenarios<br />Stats/regressionmodels<br />Wanted forpolicy decisions<br />realism(reflects knowledge of processes)<br />Reality?<br />Lack of error (accuracy of results)<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 8<br />
    11. 11. But this is the same as what confronts anyorganism/robot…<br />An adaptive entity faces an environment which is overwhelmingly complex, uncertain, unknown etc.<br />Thus this is a dilemma that AI/Robotics also confronts… whether to make robots:<br />Intelligent, learning knowledge about its world and reasoning (i.e. a realist approach)<br />Or “hard-wired” with some imperfect but effective tricks to enable it to survive and do the essential (i.e. an instrumental approach)<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 9<br />
    12. 12. AI/ML/Robotics/Adaptive Behaviour etc.<br />They are deeply split about the best approach, with competing traditions that do communicate<br />Started with reasoning and problem solving, assuming correct knowledge was at hand (a strictly realist limited-context approach)…<br />…but in more challenging domains, quick and dirty instrumentalist approaches (following Brooks) seemed to do better but these had limitations and were specific to a certain goal<br />They have not resolved this issue…<br />…but at least the competing approaches are recognised and much discussed.<br />So here are some outcomes from that discussion…<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 10<br />
    13. 13. 1: the Symbol Grounding Problem<br />Harnad (1990) Physica D<br />Formal representation can not gain meaning from internal relations with other part of the formal representation <br />In order to attach internal symbols to their meaning one needs a repeated and frequent interaction in context<br />By repeatedly changing a model with respect to the available evidence <br />By repeatedly using the model for decision making and seeing the results<br />Via frequent participatory methods, involving policy makers/advisors/stakeholders in the modelling<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 11<br />
    14. 14. 2: Starting “small”<br />Elman (1993) Cognition<br />Start with easy and specific situations and learn about them<br />Does not matter if these are instrumental and largely wrong, but the incremental building up to more useful domains essential<br />Onlygeneralising to harder and more general situations slowly<br />In other words, a commitment to a continual and lengthy modelling process<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 12<br />
    15. 15. 3: Multi-Level Modelling<br />Brooks (1991) Artificial Intelligence<br />Has fast local and specific reactive circuits for control of short-term behaviour<br />With layers that coordinate these lower level control strategies at progressively higher levels<br />Lower levels continue what they are doing feeding back to higher levels<br />Higher levels occasionally adjust or interrupt lower level units<br />Resulted in robots that could navigate rough and unknown domains like insects<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 13<br />
    16. 16. A Layered approach to modelling<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 14<br />
    17. 17. A tighter loop involving stakeholders...<br />Stakeholders are involved in parts of the “modelling” loop: criticising model, providing data, specifying model, determining goal etc.<br />Involvement comes from: relevance to their goals, having some effect/control, quickly seeing the results, feeling involved, not onerous, being situated in their lives<br />This inevitably means a loss of control by modellers! This is unavoidably political.<br />A radical move: giving this power more directly to people rather than their representatives<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 15<br />
    18. 18. Two Worlds – <br />Empirical<br />Ultimate Goal is Agreement with Observed (Truth)<br />Modeller also has an idea of what the model is and how it works<br />Instrumental<br />Ultimate Goal is in Final Outcomes (Usefulness)<br />Decisions justified by a communicable causal story<br />Tighter loop = participatory modelling<br />Model<br /><ul><li>Labels/Documentation may be different from all of the above!</li></ul>Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 16<br />
    19. 19. A Multi-Level Vision of PM <br />Feedback on Use<br />Empirical Input<br />Top-level PM<br />Top-level Policy Advisors, Academics, Official Stats.<br />Top-level Policy Makers<br />ContinualDocking<br />Pressure groups, Qual. Data, Academics<br />Region Specific PM<br />Topic Specific PM<br />Regional-level Policy Makers, Stakeholders<br />ContinualDocking<br />ContinualDocking<br />Citizen-level PM<br />Citizen-level PM<br />Local Stakeholders, activists<br />Crowd sourcing/input, individuals <br />Citizen-level PM<br />Citizen-level PM<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 17<br />
    20. 20. Consequences of Multi-Level PM<br />Different models being “pulled” in different ways by different groups, inputs and needs<br />Continual re-modelling to keep models ‘docked’ with each other and to incorporate new observed processes (maybe with a distributed ‘wiki’-like structure)<br />A lot of work by stakeholders as well as researchers<br />A lot of data of ALL levels and kinds: textual, anecdotal, network, aggregate statistical, mass data, time-series etc.<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 18<br />
    21. 21. Main Conclusions<br />The Policy/Research divide is deep, not just a matter of inclination, stubbornness or training, but rooted in their different goals<br />Lessons from the AI/Robotics world imply:<br />Need for tight and sustained iteration of modelling processes<br />Start with easier problems, work up later<br />Multi-level approach, with different levels of model, generalising only when possible and necessary<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 19<br />
    22. 22. Auxiliary Conclusions<br />A need for techniques (human and computer aided) for “translating” qualitative evidence from conversations and texts to the agent rules in simulations<br />The need for an ethnographic observation and study of researchers and policy people interacting<br />Possibly writing “manuals” for policy advisors “Researchers: how to deal with them, their advantages and bugs” <br />A similar manual for researchers about policy advisors/makers!<br />Four different views of a policy model: an analysis and some suggestions, Bruce Edmonds, Policy Workshop@ECCS, Vienna, September 2011, slide 20<br />
    23. 23. The End <br />Bruce Edmonds<br />http://bruce.edmonds.name<br />Centre for Policy Modelling<br />http://cfpm.org<br />Slides uploaded to http://slideshare.com<br />

    ×