Risks Revealed: Understanding Models' Role in Dutch Flood Management

DOI : 10.1002/bewi.201701823
Risks in the Making: The Mediating Role of Models in Water
Management and Civil Engineering in the Netherlands
Matthijs Kouw
Zusammenfassung: Wie Risiken entstehen: Zum medialen
Charakter von Modellen
am Beispiel von Wasserwirtschaft und Ingenieurbau der
Niederlande. Der Umgang
mit Vorannahmen, Unsicherheiten und blinden Flecken gehçrt
zur Praxis der Modellie-
rung. Dabei wird das den Modellen inh-rente Wissen um
Risiken in unterschiedlichem
Maße sichtbar, was fer technologische Kulturen, die auf
Modelle rekurrieren, folgenreich
sein kann. Um Kollaborationen zwischen unterschiedlichen
sozialen Gruppen wie Inge-
nieuren und politischen Entscheidern zu ermçglichen, werden in
„trading zones“ (Peter
Galison) gemeinsame Sprachregelungen gestiftet. W-hrend
solche Sprachregelungen tat-
s-chlich soziale Gruppen verbinden und deren Zusammenarbeit
befçrdern kçnnen, geht
die Rolle von Modellen als Medium h-ufig nicht in diese
Sprachregelungen mit ein. Die
Sprachregelungen der trading zones kçnnen somit dazu
beitragen, den Einfluss der Mo-
dellierungspraxis auf Risikoeinsch-tzungen zu verdecken.
Aufbauend auf Einsichten aus
den Science and Technology Studies (STS) und empirischer

Forschung in hydrologi-
scher, hydrodynamischer und geotechnischer Ingenieursarbeit
sowie der :kologie unter-
streicht dieser Beitrag die Netzlichkeit des medialen Charakters
von Modellen fer den
Nachvollzug unterschiedlicher Risikoeinsch-tzungen von
Akteuren im Wasserschutz.
Die F-higkeit, Vorannahmen, Unsicherheiten und blinde Flecken
von Computersimula-
tionen und Modellierungspraxen zu erkennen und einzusch-tzen,
erweist sich bei man-
gelnder Werdigung des medialen Charakters von Modellen als
eingeschr-nkt.
Summary: Risks in the Making: The Mediating Role of Models
in Water Manage-
ment and Civil Engineering in the Netherlands. Reliance on
models can make techno-
logical cultures susceptible to risks through the assumptions,
uncertainties, and blind
spots that may accompany modeling practices. Historian of
science Peter Galison has
described computer modeling practices as “trading zones”,
conceptual spaces in which
a shared language is hammered out in an attempt to facilitate
collaboration between dif-
ferent social groups, such as engineers and policymakers.
Although such a shared lan-
guage may enable collaboration between diverse groups, it may
also make the relation
between modeling practices and knowledge of risks less visible,
since the shared language
does not necessarily acknowledge how models produce
knowledge about their world. In
that respect, models have a ‘mediating’ role, since they are not
straightforward representa-

tions of the world, but involve a process of translating
phenomena into formalized repre-
sentations that enable experimentation. Drawing on insights
from Science and Technolo-
gy Studies (STS) and empirical work in the domains of
hydrology, hydrodynamics, geo-
technical engineering, and ecology, this paper emphasizes the
importance of understand-
ing the mediating role of models in shaping the understanding
that various actors hold
with regard to water-related risks. Failing to appreciate the
mediating role of models
M. Kouw, Dr. , SIM-CI, Zuid Hollandlaan 7, NL-2596 AL The
Hague, E-Mail: [email protected]
160 T 2017 WILEY-VCH Verlag GmbH & Co. KGaA,
Weinheim
Ber. Wissenschaftsgesch. 40 (2017) 160 – 174 www.bwg.wiley-
vch.de
www.bwg.wiley-vch.de
hampers the ability of actors to understand the assumptions,
uncertainties, and blind
spots that accompany simulation practices, and may put
technological cultures at risk.
Keywords: trading zone, risk, hydrology, hydrodynamics,
geotechnical engineering,
ecology, immersion, uncertainty, exclusion
Schlesselwçrter: Aushandlungszone, Risiko, Hydrologie,
Hydrodynamik, Geotechnik,
:kologie, Immersion, Unsicherheit, Exklusion

1 Introduction: increasing reliance on models
Although the history of the Netherlands is marked by
catastrophe due to the nation’s
propensity to flooding, the Dutch have become renowned for
their ability to deal with
the various challenges presented by the water which surrounds
them. Climate change
poses further risks in the form of rapid rises in sea level and
peak river discharges, as well
as surface water quality and biodiversity decreases due to
ongoing water temperature in-
creases.1 The history of the Netherlands features many
examples of successful interven-
tions against flood risk, the apex of which is a series of flood
barriers known as the Delta
Works, situated in the southeast of the Netherlands. In 1953,
after a dramatic flood in
the Zeeland province, which led to both significant loss of life
and capital, a set of flood
risk management measures materialized in the form of a
‘Deltaplan’, a strategy to shorten
the Dutch coastline, thereby reducing the number of dykes that
needed to be raised, and
economically improving the safety of the Zeeland province.
This ambitious construction
commenced in 1954, and was only completed in 1997. The
American Society of Civil
Engineers declared the Delta Works, along with the ‘Zuiderzee
Works’, a set of dams,
dykes, and land reclamation works in the north of the
Netherlands, as one of the ‘Seven
Wonders of the Modern World’. The success of such large-scale
engineering projects in
the Netherlands has granted the Dutch the reputation of being

able to master Earth’s ele-
ments.
The ideal of manipulability and control over the elements is
problematized by persis-
tent uncertainties posed by water and soil, which may not bode
well for the idea flood
risks can be nullified by constructing new dykes, improving
their construction, or imple-
menting some other technical measure, such as using sensor
technology to monitor the
structural integrity of dykes. Policymakers working for the
Dutch government ac-
knowledge that floods may occur despite their world-renowned
system of flood protec-
tion, and that the country must continue to prepare for dealing
with floods and their af-
termath. A position that grates with engineers dedicated to the
idea that safety can be en-
gineered by either implementing or improving flood barriers.2
As a result of climate
change, sea levels may rise up to four metres by the year 2200
and river discharges will
increase.3 However, the Dutch appear to be confident in their
ability to deal with these
challenges. Although, alongside these direct impacts, the
secondary impacts of climate
change on biodiversity and ecological resilience must be
acknowledged as an important
risk as well. More widely, climate change will impact
geopolitical stability and may have
a significant impact on the Dutch economy, which is heavily
reliant on international
trade. As such, the challenges concomitant with climate change
cannot be confined to
a ‘can-do’ mentality, based solely on implementing ever-greater

technological solutions.
Ber. Wissenschaftsgeschichte 40 (2017) 160 – 174 T 2017
WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim 161
Risks in the Making
Activities aimed at ensuring the safety, habitability, economic
welfare, and environ-
mental sustainability of the Netherlands make extensive use of
models, ranging from
scale models of rivers and coastal structures to computer
simulations of water flow and
ecological aspects of bodies of water. Such models are used to
define, monitor, predict,
counter, and communicate water-related risks. This has
established a ‘social reliance’4 on
modeling, which makes it necessary to understand the
Netherlands as a ‘technological
culture’.5 A technological culture is one which “highlights that
the modes of inhabitation
and signification (culture) that make up our world are
technologically mediated.”6 In
this paper, I approach the vulnerability of the Netherlands to
water-related challenges by
focusing on the ‘mediating’ role of models, by which I refer to
the ways in which models
produce knowledge about the world. Models are not mute
instruments that represent
the world as it is. Rather, models provide a formal
representation of the world that can
contain assumptions, uncertainties, and blind spots. As a result,
risks can be rendered
more and less visible. Failing to understand this mediating role

of models may be an un-
accounted source of risk if left unacknowledged.
The extent to which the mediating aspect of models is
acknowledged depends on the
actors involved, who can attribute different meanings to models
and model output. So-
ciologist of technology Wiebe Bijker7 has suggested differences
between the meaning at-
tributed to technological artefacts by different “relevant social
groups,” groups of actors
that have a particular interest in a particular technological
artefact, can be analysed in
terms of “inclusion” and “technological frames.” The latter are
composed of “concepts
and techniques deployed by a community in its problem
solving” and consist of “a com-
bination of current theories, tacit knowledge, engineering
practice (such as design
methods and criteria), specialized testing procedures, goals, and
handling and using prac-
tice.”8 The notion of technological frame applies to the
interactions between different
actors, who may hold divergent opinions about the meaning of a
particular technological
artefact. Technological frames “can be used to explain how the
social environment struc-
tures an artefact’s design” and “how existing technology
structures the social environ-
ment.”9 Thus, actors’ ‘inclusion’ in a particular technological
frame can shape how they
view technological artefacts.
This paper first interrogates the extent by which different social
groups, modelers and
civil engineers, involved in water management in the

Netherlands assess the mediating
role of models. Secondly, it asks to what extent these various
technological frames,
regarding the mediating role of models, do put the Netherlands
at risk. In answering
these questions, I draw on an extended period of participant
observation between 2009
and 2011, as well as interviews conducted in the same
timeframe at Deltares in Delft, an
institute for applied research in the field of water, subsurface
and infrastructure. In addi-
tion, interviews were held at relevant software development
companies working on
model development and maintenance, governmental
organisations, research institutes,
and universities.10
My research questions are addressed at three water-related
modeling practices, all of
which involve a trading zone in which different groups of actors
interact. First, I look at
the impact of the rise of computer simulations in hydrology, the
science of the proper-
ties, distribution and effects of water on Earth’s surface, and
hydrodynamics, the science
of the dynamics of fluids. The models used in these sciences are
noted for their com-
plexity, which makes it more difficult for modeling actors to
understand the knowledge
instruments they use on a daily basis. Second, I address how
models are used in geotech-
Weinheim Ber. Wissenschaftsgeschichte 40 (2017) 160 – 174
Matthijs Kouw

nical engineering to study the onset of seepage erosion, a form
of erosion whereby water
flowing through a dyke or its foundations causes structural
damage, which can lead to
the failure of dykes. I address how such models function in a
more general context of
flood risk management, and are increasingly brought to bear on
possibilities attributed
to the analysis of large amounts of measurement data. Third, I
examine the use of
models in ecology, where they are used to enable participatory
processes of water gover-
nance, with the aim of improving and maintaining water quality
in accordance with
guidelines for water quality governance established by the
European Commission.
In discussing these cases, I indicate who is involved with
modeling in water manage-
ment and civil engineering, and the extent to which modeling
practices can become
hidden from view by the actors involved in each modeling
trading zone. The title of this
paper, ‘Risks in the making’, refers not only to the constructive
role of models in
producing knowledge about risks, it also hints towards the
possible detrimental effects of
failing to understand how models establish an understanding of
risks, yielding risks ‘in
the making’, which may loom behind seemingly innocuous
modeling practices.
2 Modeling and trading zones

As argued in the introduction, models are not merely mute
instruments. They play an
inscriptive role in the process of defining, monitoring,
predicting, countering, and com-
municating water-related risks. Although models will bear
varying degrees of semblance
to their objects of study, their ‘target systems’, they are not
straightforward representa-
tions of a presupposed world ‘out there’. Sociological and
philosophical studies of mod-
eling practices have demonstrated that the translation of target
systems into models,
upon which simulations can be run, as experiments, is not
straightforwardly ‘representa-
tional’.11 In other words, the world is ‘staged’ in a model in
order to facilitate experimen-
tation. The ability of models to stage the world through
simplification should not imme-
diately be lamented, since it is also what makes them of great
value. Models give some
idea of what may happen in a probable future, or in extreme
circumstances, such as at
un-
likely or unfeasibly high or low water levels. Moreover, models
enable experimentation
in a way that does not interfere with target systems, and can be
more economic and effi-
cient than experimentation in such systems. Models allow the
temporality of risks to be
reimagined in various ways, potentially opening up new
opportunities for fruitful inter-
vention in real-world systems. However, this pragmatic
‘staging’ also suggests the use of
models can be accompanied by assumptions, uncertainties, and
blind spots, meaning

modeling practices risk shaping the understanding of target
systems. As a result, once
our understanding of target systems is understood as something
that is shaped by model-
ing, it can be appreciated that these shortfalls may have an
effect on the practices that
draw on model output.
Work on “trading zones”12 looks at how various actors engage
in practices around
a particular object or technology, and can thereby aid analysts
in affirming the role of
models as mediators between social groups, such as engineers
and physicists. In trading
zones, rules of exchange between different social groups are
established despite differ-
ences between these social groups. It should be noted that these
differences are not over-
come by means of some uniform translation of various interests.
Rather, trading zones
involve partial communication, and can be identified as
“locations in which communi-
Risks in the Making
ties with a deep problem of communication manage to
communicate. If there is no
problem of communication, there is simply ‘trade’, ‘not a
trading zone’.”13 Galison dis-
cusses how simulations act as intermediaries between different
actors: They are objects

that are pivotal in activities related to a number of different
groups.14 Echoing the ideas
about models as far from innocent instruments of knowledge,
Galison describes how
modeling practices can form a site where differences are not so
much overcome, but
rather bridged in a way deemed satisfactory to all involved.
Trading zones involve a practical solution to the existence of
multiple perspectives on
modeling by establishing a shared language that ‘works’.
However, as I argue in more
detail below, those involved in modeling may be able to
exchange ideas and insights
without addressing the mediating effects of models themselves.
In this paper, I see
models as a socio-material substrate of water governance that
consists of technological,
organisational, institutional, and social elements, and shapes the
understanding actors
may have of various water-related risks, some of which were
mentioned in the introduc-
tion. However, since the actors involved have varying degrees
of inclusion and occupy
different technological frames, potential assumptions,
uncertainties, and blind spots may
go unnoticed or un-interrogated. This may introduce or
exacerbate water-related risks,
since potentially harmful mediating effects of models are not
addressed, leading to risks
that are glossed over until disaster potentially appears. Failing
to understand how models
can obscure as well as make risks apparent can render
technological cultures vulnerable.
Accordingly, it is argued that models, as part of a particular
trading zone, should be un-

derstood as mediating instruments contributing to both the
elucidation and production
of risks.
3 The rise of computer simulation in hydrology and
hydrodynamics
During the 20th century, research in hydrology and
hydrodynamics in the Netherlands
gradually abandoned scale modeling, such as small-scale
physical models of bridges,
dams, sluices, and harbours in water bodies or entire river
deltas. In their place, re-
searchers began using computational models, consisting of
equations that describe the
behaviour of water that is described formally as interactions
between points on a grid.
This led to the establishment of computer simulation as the
dominant knowledge instru-
ment in modeling hydrological and hydrodynamic
phenomena.15 The predominance of
computer simulation has led to a greater dependence on
increasingly complex computer
code, making it more and more difficult for hydraulic engineers
to fully grasp the design
and impact of the simulations and models they use. The
increasing complexity of code is
due to a variety of factors: Models are developed by greater
numbers of people, leading
to a more complex underlying computer code. In addition, old,
or ‘legacy’ code may no
longer be understood because the human expert enabling its use
is no longer available.
Finally, modelers may have the tendency to leave parts of the
model unquestioned due to
successful applications in the past. In this way, computer

simulations may become ‘black
boxed’, in so far as their successful outcome leaves the
structure, data, and computation
of the model unexamined.16
In computer simulations, the expertise of modelers is inscribed
in the form of com-
puter code, allowing modeling expertise to travel beyond its
development site. As
a result, simulation practice is distributed over a larger and
more varied group of actors
that is no longer limited to the engineers who actively worked
on the development of the
Matthijs Kouw
specific code. For example, models’ code may travel to
consultancies, software develo-
pers, decision makers working for city governments, scientists,
amongst many others.
Contemporary developments have enabled the distribution of
simulation practice across
a multitude of actors. First, what are termed ‘modeling
interfaces’ are becoming more
and more popular. These interfaces are standards that allow
model components from dif-
ferent developers to exchange data. Thus, modeling interfaces
enable a modular ap-
proach to modeling, in which code written by a variety of
parties is coupled together to
create a working model, provided these model components meet

the constraints of the
interface. On such platforms, model components can be
combined to create a complete
model the user deems reliable, even though extensive
knowledge of the underlying com-
ponents is not required. Second, the increasing popularity of
simulations in water gover-
nance has created a demand for simulations that can be used by
non-experts. Such simu-
lations need to be interactive, which can mean underlying
calculation rules, the set of
mathematical operations performed by a model, are simplified
to enable immersive in-
teraction with the model. Users end up using a computer
simulation that meets the crite-
ria of a governance context, but such simulations may not have
the scientific rigor
needed to convey a more complete understanding of
hydrological and hydrodynamic
phenomena.
The increasing complexity of models, together with their
computational codification
in the form of software, which enables a larger and more varied
audience, amplify issues
related to ‘epistemic opacity’,17 a term which describes how
those involved in modeling
practices may be unable to understand the computational
principles underlying compu-
ter simulations, or may not have the desire to understand these
underlying computatio-
nal principles. A reflexive or critical approach to computer
simulations, which is some-
thing engineers often emphasize as being of crucial importance,
would involve a persis-
tent questioning of the components underlying the model. This

could be achieved by
examining its design, underlying equations, and the time-
consuming process of getting
to know a model by working with it for extended periods of
time in different projects.
When such a questioning attitude is absent, those working with
computer simulations
straddle discovery and manipulation : Simulations may inform
the understanding of hy-
drological and hydrodynamic phenomena, but the way in which
they inform such an
understanding may not be subject to reflection. In other words,
the mediating role of the
computer simulation is left unquestioned. According to
sociologist of science Sherry
Turkle, simulation increasingly involves manipulation rather
than discovery, since simu-
lations are becoming increasingly opaque and more convincing
thanks to interactive ele-
ments and captivating graphics.18
Although epistemic opacity is prevalent when computer
simulations are used, it is not
given that reflexivity regarding models is decreasing amongst
all social groups involved in
modeling work. As a result of epistemic opacity, these social
groups can fall prey to ‘im-
mersion’, which can be defined more generally in terms of
engrossing, enticing, or capti-
vating influence of technologically mediated practices and
experiences.19 Hydraulic engi-
neers do not object to codification of knowledge in the form of
code per se, but do stress
it should not lead to uncritical adoption of model output. A
reflexive approach to
modeling fosters engagement with the epistemic opacity

characteristic of computer simu-
lations. Hydraulic engineers attempt to ‘tease out’ knowledge
from simulations and
models, either by using them as ‘sparring partners’, or by
‘tinkering’. In the latter case,
hydraulic engineers thoroughly examine a model’s code and try
to familiarize themselves
Risks in the Making
with the model by feeding it different sets of data. The answer
to epistemic opacity and
the danger of immersion is then not necessarily the mastery of
simulations and models,
but rather reflexive practices. ‘Tinkering’ is a promising form
of reflexivity that leads to
critical engagement with epistemically opaque computer
simulations. This way of deal-
ing with such simulations is promising because it is unlikely
that the trends that have es-
tablished opacity can be reversed. The dangers of opacity were
signalled in the early days
of software development by mathematician and computer
scientist Edsger Dijkstra.
However, contemporary challenges of hydraulic engineering do
not bode well for Dijk-
stra’s suggestion that we “confine ourselves to the design and
implementation of intel-
lectually manageable programs.”20 As code grows in
complexity, as discussed above, epis-
temic opacity cannot be ignored, as it may lead to immersion in

the absence of reflexiv-
ity. Studies of computer simulations in hydrology and
hydrodynamics should not focus
on epistemic opacity exclusively, they should also inquire into
the reflexivity of those in-
volved in simulation work. After all, when model output is
considered to be reliable,
those involved in simulation may be less inclined to question
the authority of simula-
tions and models.
Questioning the extent by which those engaged in a particular
modeling practice ad-
dress epistemic opacity can help to make explicit the
technological frames of those in-
volved. In this way, points of insufficient reflexivity can be
pointed out and the crucial
instrumental role of computer simulations can become subject to
further scrutiny. Per-
haps it is a characteristic of a particular trading zone that the
actors involved fail to ad-
dress the instrumental role of computer simulations? A language
that works may very
well be established, but may also limit reflexive engagement
with the problem of episte-
mic opacity. This issue is of importance since models play a
crucial instrumental role in
imagining possible solutions to climate-related issues in
hydrology and hydrodynamics.
Without suitable reflexive engagement, the use of computer
simulations risks steering
the imaginary of possible solutions toward the problems of
climate change without the
scepticism with which engineers should be required to apply to
a model’s output.

There are strong indications that a reflexive attitude to
computer simulations is preva-
lent among engineers responsible for model development.
Actors outside the realm of
model development, such as consultants, decision makers, and
policymakers, appear to
have a more substantial confidence in computer simulations,
and have less of a tendency
to take a model’s output with a degree of skepticism, which is
something lamented by
engineers involved in model development. Inclusion therefore
has a relationship with re-
flexivity, since those directly involved in model development
appear more likely to adopt
a critical stance towards computer simulations.
4 Wrestling with uncertainties in geotechnical engineering
‘Piping’ is a form of seepage erosion involving the movement of
water under or through
a dyke that provokes instability, in some cases leading to
breaches. The flow of water
under or through a dyke may build channels that can eventually
form a shortcut between
the structure’s water and land facing sides, a so-called ‘pipe’,
that runs through its foun-
dations. Pipes dramatically increase the speed of erosion, which
may damage the dyke or
its foundations to such an extent that it either collapses or
breaches. In the Netherlands
many dykes can consist of either clay or peat, or a combination
of the two, which sit on
foundations of sand. Whereas clay and peat are cohesive and
relatively impermeable,

Matthijs Kouw
sand is relatively permeable, which means the foundations of
many dykes are prone to
seepage erosion of their foundations. As sea levels rise and
river discharges increase as
a result of climate change, ‘piping’ is becoming a more pressing
issue in the Netherlands.
Sociologist Matthias Gross defines uncertainty as “a situation in
which, given current
knowledge, there are multiple possible future outcomes.”21
This idea of uncertainty can
be aligned with multiple possible outcomes of soil behaviour:
will a dyke hold or will it
become unstable? The composition of dykes and their
foundations, as well as the interac-
tions between different types of soil in dykes and their
foundations, are a source of persis-
tent uncertainties in geotechnical engineering. The composition
of soil may be known at
the locations where measurements have been taken, but soil can
be rather heterogeneous,
implying large differences between measuring points, even
those relatively close to each
other. Since the behaviour of soil is dependent on its
composition, this leads to difficul-
ties in predicting the likelihood and possible impact of seepage
erosion. In addition, geo-
technical engineers stress the difficulties imposed by the
complexity of interactions be-
tween different kinds of soil. Such interactions are not

understood very well yet, and
remain a source of uncertainty. The lack of data about soil can
be solved in principle,
but certainly not in practice given the limited amount of
resources available for measur-
ing and the fact that some locations may not be accessible, as a
result of the position of
roads or buildings. Uncertainty about soil behaviour precludes
certainty of prediction.
Geotechnical engineers deal with the uncertainties posed by soil
morphologies in var-
ious ways. Firstly, the geotechnical engineers at Deltares are
committed to an elaborate
process of research which is intended to reduce uncertainties,
and to develop state of the
art quantitative methods that can be used in safety assessments.
To do so, a variety of
scale models and computational techniques are used to
understand ‘piping.’ This in-
volves modeling cross-sections of dykes at various scales, so
the occurrence and develop-
ment of ‘piping’ can be studied. In some cases, engineers have
built 1 : 1 scale models
and simulated dyke breaches caused by ‘piping’ with these full-
scale models. In addition,
computational models of soil composition have been created
that attempt to describe the
interactions between different soil types within a dyke.
However, since experiments with
physical scale models are expensive and ‘piping’ takes place
largely out of sight, within
a dyke, seepage erosion remains a phenomenon that engineers
struggle to understand,
since empirical observations that help to validate computational
models are difficult to

acquire. In the laboratory, physical models and computational
models are assessed in
terms of their ability to produce relevant knowledge about
‘piping’, such as details about
the development of pipes and the circumstances that influence
pipe formation. Outside
of the laboratory, models need to fulfil a different demand,
since policymakers wish to
use state of the art model-based assessments of seepage erosion
to assess the safety of
dykes and determine where expensive dyke reinforcements need
to be constructed. Such
assessments ultimately lead to a binary choice: either a dyke
needs to be reinforced or it
does not.
More recently, flood risk management in the Netherlands has
adopted data-intensive
methods which require large amounts of measurement data to
monitor the safety of the
dyke, documenting everything from registrations of water levels
inside a dyke, to mea-
surements of vibrations that may indicate ‘piping’. This
assessment also involves the de-
velopment of software aimed at a ‘non-expert’ audience, such as
local decision makers
and organisations in charge of water safety, who need to be able
to monitor dyke safety
and act on reliable information in times of possible flooding.
Thus, geotechnical engi-
Risks in the Making

neering is not only about doing rigorous science, such as
understanding the dynamics of
seepage erosion, but increasingly also involves the development
of technologies that
enable audiences outside of the scientific community to act in
times of need, for example
by developing evacuation plans. In other words, knowledge
about ‘piping’ needs to be
not only epistemically robust, produced by ever more accurate
calculation rules, it also
needs to meet requirements related to ‘social robustness’. That
is, this knowledge needs
to be perceived as innovative and reliable in the eyes of
policymakers, or aligned with po-
litical commitments to flood preparedness.22 According to
geotechnical engineers,
models can only present the ‘piping’ in a more or less
satisfactory manner that provides
insights into how complex soil morphologies may behave. As
such, geotechnical models
only represent geotechnical phenomena in an exploratory
manner. However, geotechni-
cal models tend to function in a representational manner outside
of the laboratory, for
example in policy-making contexts.
Uncertainty can put technological cultures at risk. Attempts to
develop definitive cal-
culation rules, implement ‘innovative’ technologies for
geotechnical modeling and flood
risk management, and develop policies related to dyke safety
can be ‘unsettled’ by un-
certainty. Dyke safety assessments attempt to accommodate
uncertainty in the form of

multiple possible future outcomes. For example, a calculation
rule may turn out to be in
need of improvement, an application developed for flood risk
management may yield
unexpected outcomes or be used in an unforeseen manner, and
policymakers may not
adopt the insights produced by research into dyke failure
mechanisms. This corresponds
with Gross’ aforementioned definition of uncertainty as “a
situation in which, given cur-
rent knowledge, there are multiple possible future outcomes.”23
As a result, uncertainty
can put technological cultures at risk: The methods chosen to
cope with various risks
may be out of step with the multiple possible future outcomes
that various uncertainties
may imply.
Uncertainty can be used to describe ‘knowledge gaps’ or blind
spots in dyke safety as-
sessments. However, uncertainty does not simply entail a lack
of knowledge, it is a by-
product of simulation practices that cannot be wholly excised
from the endeavour. From
the perspective of geotechnical engineers, claims to certain
knowledge need to be ap-
proached with apprehension: The reliability of geotechnical
models does not imply an
objective truth, and calculation rules are always provisional.
These rules have acquired
credibility through worthwhile insights in the laboratory, but
that does not mean geotech-
nical models are complete or ‘finished’. Uncertainties point to
aspects of seepage erosion
that deserve further examination, and thereby act as indicators
of potentially worthwhile

research.
However, uncertainties may not be appreciated as such. As
sociologist Donald Mac-
kenzie argues, uncertainties are approached and valued
differently by various social
groups.24 In that sense it is important to acknowledge that the
outcome of modeling
practice may be disruptive and cause difficulties in the realm of
policy making: Model
output can lead to the insight that calculation rules to assess
dyke safety are no longer re-
levant, and updated calculation rules may yield assessments of
dyke safety that call for
costly investments in order to enhance dyke resilience.
Although uncertainties can put
technological cultures at risk, they also form a potential source
of knowledge about risks.
The settling of knowledge in the form of calculation rules,
software, or policies that are
considered to be both epistemically and socially robust may
gloss over uncertainties. As
a result, technological cultures may be put at risk, since settled
knowledge can imply a di-
Matthijs Kouw
minished ability to evaluate the pros and cons of various
approaches to uncertainty, and
preclude the adoption of uncertainty as a source of knowledge
about risks.

The preceding discussion on ‘piping’, and geotechnical
engineering more generally,
has described a trading zone in which the appreciation of
uncertainty is shaped by the
technological frame of the actors involved: whereas engineers
emphasize the provisional
and exploratory nature of their research on ‘piping’, decision
makers and policymakers
are primarily concerned with developing robust policies for
dyke safety. Models are re-
quired to fulfil different roles, depending on the technological
frames of those involved
and their degree of inclusion in contexts that aim for a more
scientific approach to
seepage erosion. Models are seen as instruments of exploration
or as representations of
dyke safety. Needless to say, maintaining flood defences is of
crucial importance in
a country that earns about a third of its gross national product in
areas prone to flood-
ing.25 However, seeing models as representations glosses over
uncertainties, which can
then no longer be fostered as a source of valuable knowledge.
Although much more can
be said about different kinds of uncertainties involved with
modeling in a policy con-
text,26 my claim here is oriented towards the tensions between
exploration and represen-
tation in modeling dyke failure mechanisms, and the potentially
dangerous side effects
of a preoccupation with models as representations in terms of
the extent to which dyke
failure mechanisms are understood. The way in which flood risk
is imagined changes
when the focus on fundamental research in the laboratory is

abandoned in favour of the
development of techniques for monitoring dyke safety and
dealing with the aftermath of
floods.
5 Participation and exclusion in water quality governance
In present-day water governance in the Netherlands, public
participation is applauded
since it “would improve the quality of decision making by
opening up the decision-
making process and making better use of the information and
creativity that is available
in society.”27 Extended public participation is said to have the
ability to establish more
democratic forms of water governance by allowing various
social groups, ranging from
organisations engaged in environmental protection to those
affected by particular poli-
cies, such as inhabitants of a particular area, to voice their
opinion, which may unsettle
traditional hierarchies in science and politics. However, such
technologies of participati-
on are not celebrated unanimously, since they still determine
how participation takes
place by excluding other forms of knowledge and actors.28
An example of an instrument that was supposed to enable
extended public partici-
pation is the ‘WFD Explorer’: a communication instrument
developed to facilitate dis-
cussions concerning the European Union’s Water Framework
Directive (WFD), which
aims to prevent the deterioration of aquatic ecosystems, protect
or enhance the status of
aquatic ecosystems, promote sustainable water use, and enhance

protection and improve-
ment of the aquatic environment. These objectives, it was
suggested, can be achieved
“through specific measures for the progressive reduction of
discharges, emissions and
losses of priority substances and the cessation or phasing-out of
discharges, emissions
and losses of the priority hazardous substances.”29 Article 14
of the WFD30 stresses that
interested parties can and should be provided with information
amenable to shared deci-
sion making. Active involvement of such parties meets the legal
requirements of the
WFD, improves decision making by attuning governance to the
concerns of those invol-
Risks in the Making
ved, and increases the willingness of these parties to accept and
implement policies so far
developed.31
In this vein, the WFD Explorer was intended as an instrument of
governance that
could provide stakeholders, decision- and policymakers with the
means to collaboratively
explore and understand relationships between the objectives
prescribed by the WFD, the
range of possible measures, and their potential impact. The
WFD Explorer consists of
a calculation core and knowledge database containing

calculation rules related to water
quality. These two components are largely standardized and
authored by the WFD Ex-
plorer’s developers, although users familiar with the design of
the model were initially in-
vited to change parameter values, which is something the
developers later tended to
avoid due to the lack of consensus on the values that needed to
be entered. In a further
attempt at gaining applied relevancy, a structured database was
made an additional com-
ponent of the WFD Explorer, enabling users to enter area-
specific information.
In practice, the WFD Explorer did not reach its intended
audience due to disagree-
ments among the developers of the WFD Explorer, ecologists
and biologists, and its in-
tended users. The developers’ intention to include a multitude
of perspectives in the
WFD Explorer eventually led to a situation in which conflicting
ideas needed to be re-
solved in the models underlying the software. This resulted in a
situation where the
developers took matters into their own hands, and made design
choices on behalf of the
users they aspired to give a voice to. As a result, many users
questioned the value of this
ecological modeling altogether, lamented the WFD Explorer’s
lack of adaptability and
transparency, and these critics did not adopt the WFD Explorer
in their work on water
quality governance.
The WFD Explorer can be interpreted as a trading zone that
involved contestation

between different parties, some of whom had diverging
interests. The aim of establishing
a consensual understanding between the parties involved was
not met, and the developers
tried to ‘hammer out’ a shared language in response to these
difficulties. The developers
were driven to their ill-fated attempts to stabilize the
development process due to two
challenges they faced. First, the attempts of the developers to
meet the requirements of
their user base created a confusing and chaotic process of
implementation, in which the
WFD Explorer suddenly had to function as an instrument for
detailed research rather
than producing estimations with the aim of inspiring dialogues
between those involved
with water quality governance, which had been its original
objective. This reveals a dis-
crepancy between projected and actual use, which only became
apparent after initial at-
tempts to implement the WFD Explorer. Past experiences and
projected benefits of par-
ticipatory governance were a cause for optimism at the time,
particularly on the part of
policymakers at the national level. Secondly, the WFD Explorer
failed to gain the trust
of its user base. Users dismissed its output, lamented the lack of
transparency of the
model’s inner workings, and in some cases even opposed the
idea that ecological phe-
nomena could be modeled at all. Since users had conflicting
ideas concerning the role
the Explorer should fulfil, it became rather difficulth for
developers to meet the afore-
mentioned objections.

Although the WFD Explorer was expected to act as an
instrument for an inclusive
form of participatory governance, requiring it to include a
multitude of knowledge and
social actors, which included not only model developers, but
also ecologists, biologists,
stakeholders, and policymakers, development became restricted
to a group of ‘experts’.
That said, the development of the WFD Explorer required some
uniformity, which
Matthijs Kouw
could not accommodate the interests of all prospective users. In
addition, the viability of
the WFD Explorer in the political arena depended on a degree
of standardization, which
does not necessarily correspond with the ideals of inclusive
politics. Although standard-
ization may imply exclusion, it is also needed in a political
context, to enable policy
making on a national or European level for example.
Instruments of governance there-
fore imply a degree of standardization, and as a result do not
correspond neatly with the
varieties of knowledge and social actors that are expected to be
included in truly inclusive
forms of politics.
In sum, the ‘communication landscape’ opened up by the WFD
Explorer is not

devoid of power, but fraught with power relations that shape the
communication that
takes place. It is crucial to study the effects and sources of
these dimensions of power,
since they shape the knowledge that is included in instruments
of governance and influ-
ence the types of participant. Whether the WFD Explorer
effectively allows for inclusive
forms of politics evokes a tension between standardization and
participation. If water
quality governance leans more heavily towards standardization,
it may reinforce existing
hegemonic approaches to water quality and thereby exclude
knowledge and social actors.
The exclusion of knowledge and actors can put technological
cultures at risk, since
knowledge and actors that are potentially worthwhile are not
included. However, if
water quality governance leans more towards participation, its
legitimacy in the political
arena may be compromised since it cannot meet the
requirements posed by policy
making on a national and European level. Instruments of
governance will involve
a trade-off between standardization and participation, and
should therefore be studied in
terms of exclusion of both knowledge and actors to find out to
what extent they put
technological cultures at risk.
As became clear in the preceding discussion, the WFD Explorer
can be seen as an at-
tempt to furnish an inclusive platform for water quality
governance. In establishing a
‘trading zone’ around water quality governance, the Explorer
hammered out a language

that only met particular interests. This led to the exclusion of
actors whose technological
frames were incompatible with the frames of those responsible
for the stabilization of the
WFD Explorer as an instrument of governance. Thus, water
quality governance became
imagined in a particular manner due to a shared language, which
allowed difficult choi-
ces between inclusion and stability to be made, decisions that
eventually materialized in
the form of the WFD Explorer.
6 Conclusion: the socio-material substrate of trading zones
The three modeling practices discussed above reveal tensions
between different tech-
nological frames that cannot be easily cancelled out. First, the
tension between epistemic
opacity and reflexivity continues to imply those modeling are
immersed : although re-
flexivity and tinkering are indicators of critical engagement
with computer simulations
in practice, epistemic opacity remains a characteristic of
simulation practice. Second, un-
certainty can put technological cultures at risk, but it must not
be ignored as a potential
source of knowledge. The appreciation of uncertainty as a
potential source of knowledge
is presently not yet established firmly in decision making about
risks, which is partly due
to differences between the commitments of engineers and
policymakers that cannot be
easily resolved. Third and finally, even in those cases where
inclusive platforms are de-
signed to engage water-related risks, as my discussion of the
WFD Explorer showed, the

Risks in the Making
standardization needed to make such platforms successful in
practice may be accompa-
nied by exclusion. Thus, immersion, uncertainty, and exclusion
emerge as three issues
pertinent to modeling practice.
These three issues can be seen as side effects of the different
technological frames pres-
ent in the trading zones described above, and cannot be
eradicated due to the fact that
modeling practices are not confined to one particular social
group, such as engineers.
Failing to understand the mediating role of models leads to
immersion, uncertainty, and
exclusion as potential sources of risk. Due to immersion, the
inner workings of models
can be left unquestioned, which makes it more likely model
output will be taken at face
value, even if it contains assumptions, uncertainties, and blind
spots that have problem-
atic effects. Although the epistemic opacity of computer
simulations, and models more
generally, is unlikely to ever go away, reflexivity can lead to
more responsible forms of
modeling practice. Uncertainty, when unaddressed, may cause
discrepancies between
flood risk management and flood-related risks, ultimately
leading to unexpected damages

to flood defences or a failure to act properly in times of need
due to incomplete flood
risk measures. A technological culture that is able to appreciate
uncertainty as a source of
knowledge can adapt to risks that are only provisionally
understood. Exclusion may
cause crises in political representation, since stakeholders and
other relevant actors are
not taken up in water governance. As became clear in the
previous section, exclusion
may be detrimental to the uptake of knowledge instruments
needed using in water
governance. Successfully developing such instruments is crucial
for water governance,
which only maintains political legitimacy if those affected by
policies are included in the
process of policy making. This task requires walking a fine line
between participation
and standardization.
It is clear that the mediating role of models, as knowledge
instruments, have profound
importance in all three cases. Concerning the mediating role of
technologies more gener-
ally, organisation theorist Wanda Orlikowski questions any
rigid divide between social
and technical elements. In her analysis of technologies aimed at
workplace collaboration,
she wishes to advance a ‘sociomaterial perspective’, which
“would highlight how syn-
thetic worlds are not neutral or determinate platforms through
which distributed collab-
oration is facilitated or constrained, but integrally and
materially part of constituting
that phenomenon.”32 For example, users of the WFD Explorer
cannot be considered as

free-floating and fully autonomous individuals since they are
situated in a ‘technical
milieu’ that influences their actions and perspective.
A stronger focus on the construction and effect of this technical
milieu is needed to
avoid depoliticizing the effects of instruments of governance.
Various authors point out
the need for an explicit focus on the technologies of
governance.33 In addition, recent
work on what can broadly be categorized as ‘software
studies’34 forms an emerging area
that is particularly promising in terms of understanding how
software and society are in-
terrelated. However, the broader appreciation and adoption of
this approach will partly
depend on the willingness of social scientists to become
familiar with the principles of
software development. A stronger focus on the mediating role of
technologies can also
deal with the epistemic opacity and perceived success of
simulations and models, which
establish thresholds for participation and negotiation: Users
may be incapable or simply
not interested in opening the black boxes they confront in
simulation practice, or simula-
tions and models may have acquired so much currency in
practice that they cannot be
easily contested.
Matthijs Kouw

In order to understand the relationship between simulation
practice and the vulnerabi-
lity of technological cultures, models need to be analysed as
pragmatic constructions,
developed with particular purposes in mind, and as knowledge
instruments that cannot
function as all-encompassing and exhaustive representations of
target systems. In this
paper, I have shown how the exploratory and pragmatic use of
simulation practice is lo-
cated on a slippery slope towards immersion in increasingly
opaque technological practi-
ces, seemingly socially and epistemically robust explanations of
uncertain phenomena
that may prevent the uptake of uncertainty as a source of
knowledge, and forms of gover-
nance that could exclude knowledge and actors. Immersion,
uncertainty, and exclusion
need to be subjected to careful scrutiny if technological cultures
want to prevent them-
selves from developing risks that loom behind an ignorant
attitude towards the media-
ting role of knowledge instruments.
1 PBL Netherlands Environmental Assessment Agency, The
Effects of Climate Change in the Netherlands:
2012, The Hague : PBL Netherlands Environmental Assessment
Agency, 2013.
2 Han Vrijling, Het water aan de lippen, De Gids 9 (2008), 707
– 720.
3 PBL, The Effects of Climate Change in the Netherlands: 2012
(see note 1), p. 41.
4 Robert B. Pippin, On the Notion of Technology as Ideology,
in : Andrew Feenberg and Alistair Hannay

(eds.), Technology & the Politics of Knowledge, Bloomington:
Indiana University Press 1995, pp. 43 – 61,
here p. 46.
5 Wiebe E. Bijker, The Vulnerability of Technological Culture,
in : Helga Nowotny (ed.), Cultures of Tech-
nology and the Quest for Innovation, New York: Berghahn
Books 2006, pp. 52 – 69.
6 Joost van Loon, Risk and Technological Culture: Towards a
Sociology of Virulence, London: Routledge 2002,
p. 9.
7 Wiebe E. Bijker, The Social Construction of Bakelite: Toward
a Theory of Invention, in: Trevor Pinch
and Wiebe E. Bijker (eds.), The Social Construction of
Technological Systems: New Directions in the Sociology
and History of Technology, Cambridge/MA: MIT Press 1987,
pp.159 – 187; Wiebe E. Bijker, Of Bicycles,
Bakelites, and Bulbs, Cambridge/MA: MIT Press 1995.
8 Bijker, The Social Construction of Bakelite (see note 7), p.
168.
9 Bijker, The Social Construction of Bakelite (see note 7), p.
173.
10 This research was carried out during my time as a PhD
candidate at Maastricht University (Maastricht/the
Netherlands), which culminated in the defence of my PhD thesis
in December of 2012.
11 Gabriele Gramelsberger, From Science to Computational
Sciences : A Science History and Philosophy
Overview, in : Gabriele Gramelsberger (ed.), From Science to
Computational Sciences, Zerich : diaphanes

2011, pp. 19 – 44 ; Stefan Helmreich, Digitizing Development:
Balinese Water Temples, Complexity, and
the Politics of Simulation, Critique of Anthropology 19 (1999),
249 – 265; Genter Keppers, Johannes Len-
hard, and Terry Shinn, Computer Simulation: Practice,
Epistemology, and Social Dynamics, in: Johannes
Lenhard, Genter Keppers, Terry Shinn (eds.), Simulation:
Pragmatic Constructions of Reality, New York:
Springer 2006, pp. 3 – 22 ; Mary Morgan and Margaret
Morrison, Models as Mediating Instruments, in:
Mary Morgan, Margaret Morrison (eds.), Models as Mediators:
Perspectives on Natural and Social Sciences,
Cambridge/MA: Cambridge University Press 1999, pp. 10 – 37;
Eric Winsberg, Science in the Age of Com-
puter Simulation, Chicago: Chicago University Press 2010.
12 Peter Galison, Computer Simulations and the Trading zone,
in: Peter Galison, David J. Stump (eds.), The
Disunity of Science : Boundaries, Contexts, and Power,
Stanford : Stanford University Press 1996,
pp. 118 – 157; Peter Galison, Image and Logic: A Material
Culture of Microphysics, Chicago: University of
Chicago Press 1997.
13 Harry Collins, Robert Evans, and Michael E. Gorman,
Trading zones and Interactional Expertise, in:
Michael E. Gormann (ed.), Trading zones and Interactional
Expertise, Cambridge/MA: MIT Press 2010,
pp. 7 – 23, here p. 8.
14 Galison, Image and Logic (see note 12).
15 Cornelis Disco, Jan van den Ende, ‘Strong, Invincible
Arguments’? Tidal Models as Management Instru-
ments in Twentieth-Century Dutch Coastal Engineering,
Technology and Culture 44 (2003), 502 – 535.

Risks in the Making
16 Bruno Latour, Pandora’s Hope : Essays on the Reality of
Science Studies, Cambridge/MA: Harvard University
Press 1999, p. 304.
17 Paul Humphreys, Extending Ourselves: Computational
Science, Empiricism, and Scientific Method, Cam-
bridge and New York: Oxford University Press 2004; Paul
Humphreys, The Philosophical Novelty of
Computer Simulation Methods, Synthese 169 (2009), 615 – 626.
18 Sherry Turkle, Simulation and Its Discontents,
Cambridge/MA: MIT Press 2009.
19 Gordon Calleja, In-Game : From Immersion to Incorporation,
Cambridge/MA: MIT Press 2011; Matthew
Causey, Theatre and Performance in Digital Culture : From
Simulation to Embeddedness, London: Routledge
2009.
20 Edsger Dijkstra, The Humble Programmer, in: Robert L.
Ashenhurt (ed.), ACM Turing Award Lectures:
The First Twenty Years, 1966 to 1985, New York, NY: ACM
Press 1987, pp. 17 – 32.
21 Matthias Gross, Ignorance and Surprise, Cambridge/MA:
MIT Press 2010, p. 3.
22 Michael Gibbons et al. and Helga Nowotny develop the idea
of social robustness, which they place in

a broader context of an increasing demand for scientific
knowledge that can be valorized and can yield
practical benefits. See: Michael Gibbons, Camille Limoges,
Helga Nowotny, Simon Schwartzmann, Peter
Scott, and Martin Trow, The New Production of Knowledge:
The Dynamics of Science and Research in Con-
temporary Societies, London : Sage 1994; Helga Nowotny,
Democratising Expertise and Socially Robust
Knowledge, Science and Public Policy 30 (2003), 151 – 156.
23 Gross, Ignorance and Surprise (see note 21).
24 Donald A. Mackenzie, Nuclear Missile Testing and the
Social Construction of Accuracy, in: Mario Biagioli
(ed.), The Science Studies Reader, London: Routledge 1999, pp.
342 – 357.
25 Centraal Bureau voor de Statistiek, Milieurekeningen 2008,
Den Haag and Heerlen: Centraal Bureau
voor de Statistiek, 2008, p. 65.
26 Suraje Dessai, Mike Hulme, Does climate adaptation policy
need probabilities?, Climate Policy 4 (2004),
107 – 128.
27 Dave Huitema, Erik Mostert, Wouter Egas, Sabine
Moellenkamp, Claudia Pahl-Wostl, and Resul Yalcin,
Adaptive Water Governance : Assessing the Institutional
Prescriptions of Adaptive (Co-)Management from
a Governance Perspective and Defining a Research Agenda,
Ecology and Society 14/1 (2009), 26.
28 Patrick Feng and Andrew Feenberg, Thinking About Design :
Critical Theory of Technology and the
Design Process, in : Pieter E. Vermaas et al. (eds.), Philosophy

and Design, Berlin: Springer 2008,
pp. 105 – 118; Noortje Marres and Javier Lezaun, Materials and
Devices of the Public : An Introduction,
Economy and Society 40 (2011), 489 – 509.
29 European Union, Directive 2000/60/Ec of the European
Parliament and of the Council of 23 October 2000
Establishing a Framework for Community Action in the Field of
Water Policy, Brussels : European Union
2000, p. 5.
30 European Union, Directive 2000/60/EC (see note 29), p. 16.
31 Erik Mostert, Sandra Junier, Dagmar Ridder, Eduard
Interwies, Gabrielle Bouleau, Pieter Bots, and Pierre
Maurel, Research Report No 1. Innovative instruments and
institutions in implementing the water frame-
work directive; inception report, Limoges: Office International
de l’Eau, 2009, p. 35.
32 Wanda J. Orlikowski, The Sociomateriality of Organisational
Life: Considering Technology in Manage-
ment Research, Cambridge Journal of Economics 34 (2010), 125
– 141, here p. 136.
33 See for example: Andrew Barry, Political Machines:
Governing a Technological Society, London : Athlone
Press 2001; Bruce Braun, Sarah J. Whatmore, The Stuff of
Politics: An Introduction, in : Bruce Braun,
Sarah J. Whatmore (eds.), Political Matter: Technoscience,
Democracy, and Public Life, Minneapolis and
London : University of Minnesota Press 2010, pp. ix – xl ;
Sarah J. Whatmore, Catharina Landstrçm,
Flood Apprentices: An Exercise in Making Things Public,
Economy and Society 40 (2011), 582 – 610.

34 Wendy Hui Kyong Chun, Programmed Visions: Software and
Memory, Cambridge/MA: MIT Press 2011;
Matthew Fuller, Software Studies: A Lexicon, Cambridge/MA:
MIT Press 2008; Adrian Mackenzie, Cutting
Code: Software and Sociality, New York: Peter Lang 2006;
Noah Wardrip-Fruin, Expressive Processing: Digi-
tal Fictions, Computer Games, and Software Studies,
Cambridge/MA: MIT Press 2009.
Matthijs Kouw
Copyright of Berichte zur Wissenschafts-Geschichte is the
property of John Wiley & Sons,
Inc. and its content may not be copied or emailed to multiple
sites or posted to a listserv
without the copyright holder's express written permission.
However, users may print,
download, or email articles for individual use.
ComputationalModellingofPublicPolicy:
ReflectionsonPractice
Nigel Gilbert1, Petra Ahrweiler2, Pete Barbrook-Johnson1,
Kavin
PreethiNarasimhan1,HelenWilkinson3
1Department of Sociology, University of Surrey Guildford, GU2
7XH United Kingdom
2InstituteofSociology,JohannesGutenbergUniversityMainz,Jako

b-Welder-Weg20,55128Mainz,Germany
3Risk
Solution
s, Dallam Court, Dallam Lane, Warrington, Cheshire, WA2
7LT, United Kingdom
Correspondence should be addressed to [email protected]
Journal of Artificial Societies and Social Simulation 21(1) 14,
2018
Doi: 10.18564/jasss.3669 Url:
http://jasss.soc.surrey.ac.uk/21/1/14.html
Received: 11-01-2018 Accepted: 11-01-2018 Published: 31-01-
2018
Abstract: Computationalmodelsare
increasinglybeingusedtoassist indeveloping,
implementingandevalu-
ating public policy. This paper reports on the experience of the
authors in designing and using computational
models of public policy (‘policy models’, for short). The paper
considers the role of computational models in
policy making, and some of the challenges that need to be
overcome if policy models are to make an e�ec-

tive contribution. It suggests that policy models can have an
important place in the policy process because
they could allow policy makers to experiment in a virtual world,
and have many advantages compared with
randomised control trials and policy pilots. The paper then
summarises some general lessons that can be ex-
tractedfromtheauthors’experiencewithpolicymodelling.
Thesegeneral lessonsincludetheobservationthat
o�enthemainbenefitofdesigningandusingamodelisthatitprovides
anunderstandingofthepolicydomain,
rather than the numbers it generates; that care needs to be taken
that models are designed at an appropriate
level of abstraction; that although appropriate data for
calibration and validation may sometimes be in short
supply, modelling is o�en still valuable; that modelling
collaboratively and involving a range of stakeholders
from the outset increases the likelihood that the model will be
used and will be fit for purpose; that attention
needstobepaidtoe�ectivecommunicationbetweenmodellersandsta
keholders;andthatmodellingforpub-
lic policy involves ethical issues that need careful
consideration. The paper concludes that policy modelling
will continue to grow in importance as a component of public
policy making processes, but if its potential is to
befullyrealised,

therewillneedtobeameldingoftheculturesofcomputationalmodelli
ngandpolicymaking.
Keywords: Policy Modelling, Policy Evaluation, Policy
Appraisal, Modelling Guidelines, Collaboration, Ethics
Introduction
1.1 Computationalmodelshavebeenusedtoassist indeveloping,
implementingandevaluatingpublicpoliciesfor
at least threedecades,but theirpotential
remainstobefullyexploited(Johnston&Desouza2015;Anzolaetal.
2017;Barbrook-Johnsonetal.2017).
Inthispaper,usingaselectionofexamplesofcomputationalmodelsus
ed
in public policy processes, we (i) consider the roles of models
in policy making, (ii) explore policy making as a
typeofexperimentationinrelationtomodelexperiments,and(iii)sug
gestsomekeylessonsforthee�ectiveuse
ofmodels. Wealsohighlightsomeof
thechallengesandopportunities facingsuchmodelsandtheiruse
inthe
future.
Ouraimistosupportthemodellingcommunitythatreadsthisjournalin
itse�orttobuildcomputational

models of public policy that are valuable and useful.
1.2
Webelievethise�ortistimelygiventhatcomputationalmodels,ofthe
typethisjournalregularlyreportson,are
now increasingly used by government, business, and civil
society as well as in academic communities (Hauke
etal.2017).
Therearemanyguidestocomputationalmodellingproducedfordi�er
entcommunities, forexam-
ple in UK government the ‘Aqua Book’ (reviewed for JASSS in
Edmonds (2016)), but these are o�en aimed at
practitioner and government audiences, can be highly
procedural and technical, generally omit discussion of
failure and rarely include deeper reflections on how best to
model for public policy. Our aim here is to fill gaps
JASSS, 21(1) 14, 2018
http://jasss.soc.surrey.ac.uk/21/1/14.html Doi:
10.18564/jasss.3669
[email protected]
le�bytheseformalguides, toprovidereflectionsaimedatmodellers,

touseaselectionofexamplestoexplore
issues in an accessible way, and acknowledge failures and
learning from them.
1.3 We focus only on computational models that aim to model,
or include some modelling of, social processes.
Although some of the discussion may apply, we are not directly
considering computational models that are
purelyecologicalortechnical intheir
focus,orsimplermodelssuchasspreadsheetswhichmayimplicitlyco
ver
socialprocessesbuteitherdonotrepresentthemexplicitly,ormakeext
remelysimpleandstrongassumptions.
Although ‘computational models of public policy’ is the full
and accurate term, and others o�en use ‘compu-
tational policy models’, for the sake of brevity, we will use the
term ‘policy models’ throughout the rest of this
paper.
1.4 Based on our experience, our main recommendations are
that policy modelling needs to be conducted with
a strong appreciation of the context in which models will be
used, and with a concern for their fitness for the
purposes for which they are designed and the conclusions drawn
from them. Moreover, policy modelling is

almost always likely to be of low or no value if done without
strong and iterative engagement with the users
of the model outputs, i.e. decision makers. Modellers must
engage with users in a deep, meaningful, ethically
informed and iterative way.
1.5 In the remainder of this paper, Section 2 introduces the role
of policy models in policy making. Section 3 ex-
ploresthe ideaofpolicymakingasatypeofexperimentation
inrelationtopolicymodelexperiments. Wethen
discusssomeexamplesandexperiencesofpolicymodelling(Section
4)anddrawoutsomekeylessonstohelp
make policy modelling more e�ective (Section 5). Finally,
Section 6 concludes and discusses some key next
steps and other opportunities for computational policy
modellers.
TheRoleofModels inPolicyMaking
2.1 Thestandard, butnow somewhatdiscreditedview ofpolicy
making is that itoccurs incycles (forexample, see
the seminal arguments made in Lindblom 1959 and Lindblom
1979; and more recently o�icial recognition in
HM Treasury 2013). A policy problem comes to light, perhaps
through the occurrence of some crisis, a media

campaign, or as a response to a political event. This is the
agenda setting stage and is followed by policy for-
mulation, gathering support for the policy, implementing the
policy, monitoring and evaluating the success of
the policy and finally policy maintenance or termination. The
cycle then starts again, as new needs or circum-
stancesgeneratedemandsfornewpolicies.
Althoughtheideaofapolicycyclehasthemeritofbeingaclearand
straightforwardwayofconceptualisingthedevelopmentofpolicy,
ithasbeencriticisedasbeingunrealisticand
oversimplifying what happens, which is typically highly
complex and contingent on multiple sources of pres-
sureandinformation(Cairney2013;Moran2015),andevenself-
organising (Byrne&Callaghan2014;Teisman&
Klijn 2008).
2.2 The idea of a cycle does, however, still help to identify the
many components that make up the design and
implementationofpolicy. Thereareat
leasttwoareaswheremodelshaveaclearandimportantroletoplay: in
policydesignandappraisal,andpolicyevaluation.
Policyappraisal(asdefinedinHMTreasury2013,sometimes
referred to as ex-ante evaluation, consists of assessing the
relative merits of alternative policy prescriptions in
meetingthepolicyobjectives. Appraisal findingsareakey input

intopolicydesigndecisions. Policyevaluation
either takes a summative approach, examining whether a policy
has actually met its objectives (i.e. ex-post),
or a more formative approach to see how a policy might be
working, for whom and where (HM Treasury 2011).
In the formative role, the key goal is learning to inform future
iterations of the policy, and others with similar
characteristics.
Modellingtosupportpolicydesignandappraisal
2.3 Whenusedex-
ante,apolicymodelmaybeusedtoexploreapolicyoption,helpingtoid
entifyandspecify inde-
tailaconsistentpolicydesign(HMTreasury2013), forexampleby
locatingwherebestapolicymight intervene,
or by identifying possible synergies or conflicts between the
mechanisms of multiple policies. Policy models
can also be used to appraise alternative policies, to see which of
several possibilities can be expected to yield
the best or most robust outcome. In this mode, a policy model is
in essence used to ‘experiment’ with alterna-
tive policy options and assumptions about the system in which
it is intervening, by changing the parameters
or the rules in the model and observing what the outcomes are.

This is valuable because it saves the time and
costassociatedwithhavingtorunexperimentsorpilots
intheactualpolicydomain. Thisconceptofthemodel
as an experimental space is discussed in more detail in Section
3 below.
JASSS, 21(1) 14, 2018
10.18564/jasss.3669
2.4 Thecommonassumptionisthatonebuildscomputationalmodels
inordertomakepredictions. However,pre-
diction,inthesenseofpredictingthefuturevalueofsomemeasure,isin
facto�enimpossibleinpolicydomains.
Socialandeconomicphenomenaareo�encomplex
(inthetechnicalsense,seee.g. Sawyer2005). Thismeans
that how some process evolves depends on random chance, its
previous history (‘path dependence’) and the
e�ect of positive and negative feedback loops. Just as with the
weather, for which exact forecasting is impos-
sible more than a few days ahead, the future course of many
social processes may be literally unknowable in
detail, no matter how detailed the model may be. Secondly, a

model is necessarily an abstraction from real-
ity, and since it is impossible to isolate sections of society,
from outside influences, there may be unexpected
exogenous factors that have not been modelled and that a�ect
the outcome.
2.5 Forthesereasons, theabilitytomake‘pointpredictions’, i.e.
forecastsofspecificvaluesataspecifictimeinthe
future, is rarelypossible. Morepossible
isapredictionthatsomeeventwillorwillnot takeplace,orqualitative
statements about the type or direction of change of values.
Understanding what sort of unexpected outcomes
can emerge and something of the nature of how these arise also
helps design policies that can be responsive
tounexpectedoutcomeswhentheydoarise.
Itcanbeparticularlyhelpful inchangingenvironmentstousethe
model to explore what might happen under a range of possible,
but di�erent, potential futures - without any
commitment about which of these may eventually transpire.
Even more valuable is a finding that the model
shows that certain outcomes could not be achieved given the
assumptions of the model. An example of this is
the use of a whole system energy model to develop scenarios
that meet the decarbonisation goals set by the
EU for 2050 (see, for example, RAENG 2015.)

2.6 Ratherdi�erent fromusingmodels
tomakepredictionsorgeneratescenarios is theuseofmodels to
formalise
and clarify understanding of the processes at work in some
domain. If this is done carefully, the model may be
valuable as a training or communication tool, demonstrating the
mechanisms at work in a policy domain and
how they interact.
Modellingtosupportpolicyevaluation
2.7 To evaluate a policy ex-post, one needs to compare what
happened a�er the policy has been implemented
against what would have happened in the absence of the policy
(the ‘counterfactual’). To do this, one needs
data about the real situation (with the policy evaluation) and
data about the situation if the policy had not
been implemented (the so-called ‘business as usual’ situation).
To obtain the latter, one can use a randomised
control trial (RCT) or quasi-experiment (HM Treasury 2011),
but this is o�en di�icult, expensive and sometimes
impossibletocarryoutduetothenatureoftheinterventionbarringposs
ibilityofcreatingcontrolgroups(e.g. a
schemewhichisaccessibletoall,orapolicy inwhichlocal

implementationdecisionsare impossibletocontrol
and have a strong e�ect).
2.8 Policy models o�er some alternatives. One is to develop a
computational model and run simulations with and
without implementation of a policy, and then compare formally
the two model outcomes with each other and
with reality (with the policy implemented), using quantitative
analysis. This avoids the problems of having to
establish a real-world counterfactual. Once again, the policy
model is being used in place of an experiment.
2.9 Anotheralternative is
tousemorequalitativeSystemMappingtypeapproaches(e.g.
FuzzyCognitiveMapping;
seeUprichard&Penn2016),
tobuildqualitativemodelswithdi�erentstructuresandassumptions(
torepresent
the situation with and without the intervention), and again
interrogate the di�erent outcomes of the model
analyses.
2.10 Finally, another use in ex-post evaluation is to use models
to refine and test the theory of how policies might
have a�ected an outcome of interest, i.e. to support common

theory-based approaches to evaluation such as
Theory of Change (see Clark & Taplin 2012), and Logic
Mapping (see Hills 2010).
2.11 Interrogation of models and model results can be done
quantitatively (i.e. through multiple simulations, sen-
sitivity analysis, and ‘what if’ tests), but may also be done in
qualitative and participatory fashion with stake-
holders, with stakeholders involved in the actual analysis (as
opposed to just being shown the results). The
choice should be driven by the purpose of the modelling
process, and the needs of stakeholders. In both ex-
ante and ex-post evaluation, policy models can be powerful
tools to use as a route for engaging and informing
stakeholders, including the public, about policies and their
implications (Voinov & Bousquet 2010). This may
beby includingstakeholders intheprocess, decisions,
andvalidationofmodeldesign; or itmaybe later in the
process, inusingtheresultsofamodel
toopenupdiscussionswithstakeholders,and/orevenusingthemodel
‘live’ to explore connections between assumptions, scenarios,
and outcomes (Johnson 2015a).
JASSS, 21(1) 14, 2018

10.18564/jasss.3669
Di�iculties intheuseofmodelling
2.12 While, in principle, policy models have all these roles and
potential benefits, experience shows that it can be
di�icult to achieve them (see Taylor 2003; Kolkman et al. 2016,
and Section 4 for some examples). The policy
process has many characteristics that can make it di�icult to
incorporate modelling successfully, including1:
• The need for acceptability and transparency: policy makers
may fall back on more traditional and more
widelyacceptedformsofevidence,especiallywheretherisksassociat
edwiththedecisionarehigh. Mod-
elsmayappeartoactasblackboxesthatonlyexpertscanunderstandoru
se,withoutputshighlyreliant
on assumptions that are di�icult to validate. Analysts and
researchers in government o�en have little
autonomy, and although they may see the value of policy
models, it can be di�icult for them to commu-
nicate this to the decision-makers.

• Changeanduncertainty: theenvironment
inwhichthepolicywillbe implemented,maybehighlyuncer-
tain, this can undermine model development when beliefs or
decisions shi� as a result of the modelling
process (although this is equally an important outcome and
benefit of the modelling process), or other
factors.
• Shorttimescales:
thetimescalesassociatedwithpolicydecisionmakingarealmostalwa
ysrelativelyfast,
and needs can be di�icult to predict, meaning it can be di�icult
for computational modellers to provide
timely support.
• Procurement processes: o�en departments lack the capability
and su�iciently flexible processes to pro-
cure complex modelling.
• Thepoliticalandpragmaticrealitiesofdecisionmaking:
individuals’valuesandpoliticalvaluescanhold
hugesway,eveninthefaceofempiricalevidence(letalonemodelling)
thatmaycontradict theirview,or
point towards policy which is politically impossible.

• Stakeholders: There will be many di�erent stakeholders
involved in developing, or a�ected by, policies.
It will not be possible to engage all of these in the policy
modelling process, and indeed policy makers
may be wary of closely involving them in a participatory
modelling process.
2.13 These characteristics may also apply more widely to
evidence and other forms of research and analysis. It is
not our suggestion that these characteristics are inherently
negative; they may be important and reasonable
parts of the policy making process. The important thing to
remember, as a modeller, is that a model can only,
and should only, provide more information to the process, not a
final decision for the policy process to simply
implement.
PolicyExperimentsandPolicyModels
3.1 Although the roles and uses of policy models are relatively
well-described and understood, our perception is
that there are still many areas where more use could be made of
modelling and that a lack of familiarity with,
and confidence in, policy modelling, is restricting its use.
Potential users may question whether policy mod-

elling in their domain is su�iciently scientifically established
and mature to be safely applied to guiding real-
world policies. The di�erence between applying policies to the
real world and making experimental interven-
tions in a policy model might be too big to generate any
learning from the latter to inform the former.
Policypilots
3.2 One response is to argue that actual policy implementations
are themselves experimental interventions and
are therefore of the same character as interventions in a policy
model. Boeschen et al. (2017) propose that
we live in “experimental societies" and that implementing
policies is nothing but conducting “real-world ex-
periments". Real-world experiments are “a more or less
legitimate, methodically guided or carelessly adopted
social practice to start something new" (Krohn 2007, p. 344;
own translation). Their outcomes immediately
display “success or failure of a design process" (ibid., p. 347).
3.3 A real-world experiment implements one solution for the
policy design problem. It does not check for other
possible solutions or alternative options, but at best monitors
and responds to what is emerging in real time.

JASSS, 21(1) 14, 2018
10.18564/jasss.3669
Implementing policies as a real-world experiment is therefore
far from ideal and far removed from the idea of
reversibility in the laboratory. In laboratory experiments the
experimental system is isolated from its environ-
ment in such a way that the e�ects of single parameters can be
observed.
3.4 One approach that tries to bridge the gap between the real-
world and laboratory experiments is to conduct
policypilots.
Theuseofpolicypilots(Greenberg&Shroder1997;CabinetO�ice20
03;Martin&Sanderson1999)
as social experiments is fairly widespread. In a policy pilot, a
policy change can be assessed against a coun-
terfactual in a limited context before rolling it out for general
implementation. In this way (a small number of)
di�erent solutions can be tried out and evaluated, and learning
fed back into policy design.

3.5 A dominant method for policy pilots is the Randomised
Control Trial (RCT) (Greenberg & Shroder 1997; Boruch
1997), well-known from medical research, where a carefully
selected treatment group is compared with a con-
trol group that is not administered the treatment under scrutiny.
RCTs can thus present a halfway house be-
tween an idealised laboratory experiment and a real-world
experiment. However, the claim that an RCT is ca-
pableofreproducingalaboratorysituationwhererigoroustestingagai
nstacounterfactual ispossiblehasalso
been contested (Cabinet O�ice 2003, p. 19). It is argued that in
principle there is no possibility of social experi-
mentsduetotherequirementofceterisparibus (i.e.
inthesocialworld, it is impossibletohavetwoexperiments
with everything equal but the one parameter under scrutiny);
that the complex system-environment interac-
tions that are necessary to adequately understand social systems
cannot be reproduced in an RCT; and that
random allocation is impossible in many domains, so that a
‘neutral’ counterfactual cannot be established.
Moreover, itmaybeariskypoliticalstrategyorevenunethical
toadministeracertainbenefit insomepilotcon-
textbutnottothecorrespondingcontrolgroup. This
isevenmorethecase if thepolicywouldputtheselected

recipients at a disadvantage (Cabinet O�ice 2003, p. 17).
3.6
Whileapilotcanbegoodforgatheringevidenceaboutasinglecase,
itmightnotserveasagood‘one-size-fits-
all’ role model for other cases in other contexts. Furthermore, it
cannot say much about why or how the policy
workedordidnotwork,ordecomposethe‘whatworks’questions into,
‘whatworks,where, forwhom,atwhat
costs,andunderwhatconditions’?
Therearealsomorepracticalproblemstoconsider,amongthemtime,s
ta�
resources and budget. There is general agreement that a good
pilot is costly, time-consuming, “administra-
tively cumbersome" and in need of well-trained managing sta�
(Cabinet O�ice 2003, p. 5). There is “a sense of
pessimismanddisappointmentwiththewaypolicypilotsandevaluati
onsarecurrentlyusedandwereusedin
the past (...): poorly designed studies; weak methodologies;
impatient political masters; time pressures and
unrealistic deadlines" (Seminar on Policy Pilots and Evaluation
2013, p. 11).
3.7
Thus,policypilotscannotmeettheclaimtobeahappymediumbetwee

nlaboratoryexperiments,withtheiriso-
lationstrategiescapableofparametervariation,andKrohn’sreal-
worldexperimentsinvolvingcomplexsystem-
environment interactions in real time. This is where
computational policy modelling comes in.
Policymodels forpolicyexperimentation
3.8 Unlike policy pilots, computational policy models are able
to work with ceteris paribus rules, random control,
andnon-contaminatedcounterfactuals(seebelow).
Usingpolicymodels,wecanexplorealternativesolutions,
simplybytryingoutparametervariations inthemodel,
andexperimentwithcontext-specificmodelsandwith
short,mediumandlongtimehorizons.
Furthermore,policymodelsareethicallyandpoliticallyneutraltobui
ld
and run, though the use of their outcomes may not be.
3.9 Unlikereal-
worldandpolicypilots,policymodelsallowtheusertoinvestigatethef
uture. Initiallythemodellers
will seektoreproducethedatabasedescribingthe
initialstateofareal-worldexperimentandthenextrapolate
simulatedstructuresanddynamicsintothefuture.

Atfirstabaselinescenariocanbederived: whatiftherewere
no changes in the future? This is artificial and, for
methodological reasons, boring: nothing much happens but
incremental evolution, no event, no surprise, no intervention;
changes can then be introduced.
3.10 As with real-world experiments, modelling experiments
enable recursive learning by stakeholders. Stakehold-
ers can achieve system competence and practical skills through
interacting with the model to learn ‘by doing’
how to act in complex situations. With the model, it is not only
possible to simulate the real-world experiment
envisagedbutalsototestmultiplescenariosforpotential real-
worldexperimentsviaextensiveparametervari-
ations. The whole solution space can be checked, where future
states are not only accessible but tractable.
3.11 Thisdoesnotimplythatit
ispossibletoobtainexactpredictionsforfuturestatesofcomplexsocia
lsystems(see
the discussion on prediction above). Deciding under uncertainty
has to be informed di�erently:
“Experimenting under conditions of uncertainties of this kind, it
appears, will be one of the most

distinctivecharacteristicsofdecision-makinginfuturesocieties[...],
theyimportandusemethods
JASSS, 21(1) 14, 2018
10.18564/jasss.3669
of investigation and research. Among these are conceptual
modelling of complex situations, com-
puter simulation of possible futures, and – perhaps most
promising – the turning of scenarios into
‘real-world experiments’" (Gross & Krohn 2005, p. 77).
3.12 Regarding the continuum between the extremes of giving
no consideration (e.g. with laboratory experiments)
and full consideration (e.g. real-world experiments) to complex
system-environment interactions, policy mod-
elling experiments indeed sit somewhere in the (happy) middle.
We would argue that, where the costs or risks
associated with a policy change are high, and the context is
complex, it is not only common sense to carry out
policy modelling, but it would be unethical not to.

ExamplesofPolicyModels
4.1 Wehavediscussedtheroleofpolicymodels
inabstractatsomelength, it isnowimportantto illustratetheuse
of policy models using a number of examples of policy
modelling drawn from our own experience. These have
beenselectedtoo�erawiderangeoftypesofmodelandcontextsofapp
lication. Inthespiritofrecordingfailure
as well as success, we mention not only the ultimate outcomes,
but also some of the problems and challenges
encountered along the way. In the next section, we shall draw
out some general lessons from these examples.
Tell-Me
4.2 TheEuropean-fundedTELLMEproject
focusedonhealthcommunicationassociatedwithinfluenzaepidemic
s.
One output was a prototype agent-based model, intended to be
used by health communicators to understand
the potential e�ects of di�erent communication plans under
various influenza epidemic scenarios (Figure 1).
4.3 The basic structure of the model was determined by its
purpose: to compare the potential e�ects of di�er-

entcommunicationplansonprotectivepersonalbehaviourandhence
onthespreadofaninfluenzaepidemic.
This requires two linked models: a behaviour model that
simulates the way in which people respond to com-
munication and make decisions about whether to vaccinate or
adopt other protective behaviour, and an epi-
demic model that simulates the spread of influenza. The key
model entities are: (i) messages, which together
implement the communication plans; (ii) individuals, who
receive communication and make decisions about
whethertoadoptprotectivebehaviour;and(iii)regions,whichholdin
formationaboutthelocalepidemicstate.
The major flow of influence is the e�ect that communication
has on attitude and hence behaviour, which af-
fects epidemic transmission and hence incidence. Incidence
contributes to perceived risk, which influences
behaviourandestablishesafeedbackrelationship(seeBadham&Gilb
ert2015forthedetailedspecification). A
fullerdescriptionofthemodelanddiscussiononitusescanbefoundin
Barbrook-Johnsonetal. (2017). Amore
technicalpaperonanovelmodelcalibrationapproachusingtheTellM
easanexampleispresentedinBadham
et al. (2017).
4.4

Drawingonfindingsfromstakeholderworkshopsandtheresultsofthe
modelitself,themodellingteamsuggest
the TELL ME model can be useful: (i) as a teaching tool, (ii) to
test theory, and (iii) to inform data collection
(Barbrook-Johnson et al. 2017).
HOPES
4.5 Practice theories provide an alternative to the theory of
planned behaviour and the theory of reasoned action
to explore sustainability issues such as energy use, climate
change, food production, water scarcity, etc. The
central argument is that the routine activities (aka practices)
that people carry out in the service of everyday
living(e.g. waysofcooking,eating,
travelling,etc.),o�enwithsomelevelofautomaticitydevelopedover
time,
shouldbethefocusof inquiryandinterventionif thegoal
istotransformenergy-andemissions-intensiveways
of living.
4.6 The Households and Practices in Energy use Scenarios
(HOPES) agent-based model (Narasimhan et al. 2017)
was developed to formalise key features of practice theories and
to use the model to explore the dynamics of

energyuseinhouseholds. Akeytheoretical
featurethatHOPESsoughttoformalise is theperformanceofprac-
tices, enabled by the coming together of appropriate meanings
(mental activities of understanding, knowing
how and desiring, Reckwitz 2002), materials (objects, body and
mind) and skills (competences). For example,
JASSS, 21(1) 14, 2018
10.18564/jasss.3669
Figure 1: A screenshot of the Tell Me model interface. The
interface houses key model parameters related to
individuals’ attitude towards influenza, their consumption of
di�erent media types, the epidemiological pa-
rameters of the strain of influenza, and their social networks.
Key outputs shown include changes in people’s’
attitude, actual behaviour, and the progression of the epidemic.
The world view shows the spread of the epi-
demic (blue = epidemic not yet reached, red = high levels of
infection, green = most people recovered).
a laundry practice could signify a desire for clean clothes

(meaning) realised by using a washing machine (ma-
terial) and knowing how to operate the washing machine (skill);
performance of the practice then results in
energy use.
4.7 HOPES has two types of agents: households and practices.
Elements (meanings, materials and skills) are en-
tities in the model. The model concept is that households choose
di�erent elements to perform practices de-
pendingonthesocio-technicalsettingsuniquetoeachhousehold.
Theperformanceofsomepracticesresult in
energy use while some do not, e.g., using a heater to keep warm
results in energy use whereas using a jumper
or blanket does not incur energy use. Furthermore, the repeated
performance of practices across space and
time causes the enabling elements to adapt (e.g. some elements
are used more popularly than others), which
subsequently a�ects the future performance of practices and
thereby energy use. A rule-based system, devel-
oped based on empirical data collected from 60 UK households,
was included in HOPES to enable households
to choose elements to perform practices. The rule-based
approach allowed organising the complex contex-
tual information and socio-technical insights gathered from the
empirical study in a structured way to choose

the most appropriate actions when faced with incomplete and/or
conflicting decisions. HOPES also includes
sub-models to calculate the energy use resulting from the
performance of practices, e.g. a thermal model of a
house is built in to consider the outdoor temperature, the type
and size of heater, and the thermostat setpoint
to estimate the energy used for thermal comfort practices in
each household.
4.8 The model is used to test di�erent policy and innovation
scenarios to explore the impacts of the performance
of practices on energy use. For example, the implementation of
a time of use tari� demand response scenario
shows that while some demand shi�ing is possible as a
consequence of pricing signals, there is no significant
reductioninenergyuseduringpeakperiodsasmanyhouseholdscanno
tputo�usingenergy(Narasimhanetal.
2017). The overall motivation is that by gaining insight into the
trajectories of unsustainable energy consum-
ing practices (and underlying elements) under di�erent
scenarios, it might be possible to propose alternative
pathways that allow more sustainable practices to take hold.
SWAP

4.9 TheSWAPmodel (Johnson2015b,a) isanagent-basedmodelof
farmers’makingdecisionsaboutadoptingsoil
and water conservation (SWC) practices on their land.
Developed in NetLogo (Wilensky 1999), the main agents
inthemodelarefarmers,whoaremakingdecisionsaboutwhethertopr
acticeSWCornot,andextensionagents
who are government and non-governmental actors who
encourage farmers to adopt. Farmers can also be en-
couraged or discouraged to change their behaviour depending on
what those nearby and in their social net-
worksaredoing.
Theenvironmentisasimplemodelofthesoilquality(Figure2).
Themainoutcomesofinterest
JASSS, 21(1) 14, 2018
10.18564/jasss.3669
are the temporal and spatial patterns of SWC adoption. A full
description can be found in Johnson (2015b) and
Johnson (2015a).
Figure 2: Screenshot of the SWAP model key outputs and

worldview in NetLogo. The outputs show the per-
centagesof farmerspracticingSWCandsimilarly thenumberof
fieldswithSWC. Intheworldview,circlesshow
farmer agents in various decision states, triangles show
‘extension’ agents, and the patches’ colour denotes
the presence of conservation (green or brown) and soil quality
(deeper colour means higher quality). Source:
Adapted from Johnson (2015a).
4.10 The SWAP model was developed: (i) as an ‘interested
amateur’ to be used as a discussion tool to improve the
qualityof
interactionbetweenpolicystakeholders;and(ii)asanexplorationoft
hetheoryonfarmerbehaviour
in the SWC literature.
4.11 The model’s use as an ‘interested amateur’ was explored
with stakeholders in Ethiopia. Using a model as an
interestedamateur isaconcept inspiredbyDennett(2013).
Dennettsuggestsexpertso�entalkpasteachother,
make wrong assumptions about others’ beliefs, and/or do not
wish to look stupid by asking basic questions.
These failings can o�en mean experts err on the side of under-
explaining issues, and thus fail to come to con-
sensus or agreeable outcomes in discussion. For Dennett, an

academic philosopher, the solution is to bring
undergraduate students – interested amateurs – into discussions
to ask the simple questions, and generally
force experts to err on the side of over-explanation.
4.12 The SWAP model was used as an interested amateur with a
di�erent set of experts, policy makers and o�icials
in Ethiopia. This was done because policies designed to
increase adoption of SWC have generally been un-
successful due to poor calibration to farmers’ needs. This is
understood in the literature to be a result of poor
interactionbetweenthevariousstakeholdersworkingonSWC.When
used,participantsrecognisedthevalueof
the model and it was successful in aiding discussion. However,
participants described an inability to innovate
in their work, and viewed stakeholders ‘lower-down’ the policy
spectrum as being in more need of discussion
tools. A full description of this use of the model can be found in
Johnson (2015a).
INFSO-SKIN
4.13 The European Commission was expecting to spend
arounde77 billion on research and development through
its Horizon 2020 programme between 2014 and 2020. It is the

successor to the previous, rather smaller pro-
gramme,calledFramework7.
WhenHorizon2020wasbeingdesigned,
theCommissionwantedtounderstand
how the rules for Framework 7 could be adapted for Horizon
2020 to optimise it for current policy goals, such
as increasing the involvement of small and medium enterprises
(SMEs).
4.14 An agent-based model, INFSO-SKIN, was built to evaluate
possible funding policies. The model was set up to
reproducethefundingrules,
thefundedorganisationsandprojects,andtheresultingnetworkstruct
uresofthe
Framework 7 programme. This model, extrapolated into the
future without any policy changes, was then used
asabenchmarkfor furtherexperiments.
Againstthisbaselinescenario,severalpolicychangesthatwereunder
consideration for the design of the Horizon 2020 programme
were then tested, to understand the e�ect of a
rangeofpolicyoptions:
changestothethematicscopeoftheprogramme;thefunding
instruments; theoverall
JASSS, 21(1) 14, 2018

10.18564/jasss.3669
amount of programme funding; and increasing SME
participation (Ahrweiler et al. 2015). The results of these
simulations ultimately informed the design of Horizon 2020.
SilentSpreadandExodis-FMD
4.15 Following the 2001 outbreak of Foot and Mouth Disease
(FMD), the UK Department of the Environment, Food
and Rural A�airs (Defra) imposed a 20-day standstill period
prohibiting any livestock movements o�-farm fol-
lowingthearrivalofananimal. The20-
dayrulecausedsignificantdi�iculties for farmers.
TheLessonsLearned
Inquiry, which reported in July 2002, recommended that the 20-
day standstill remain in place pending a de-
tailed cost-benefit analysis (CBA) of the standstill regime.
4.16 Defra commissioned the CBA in September 2002 and a
report was required in early 2003 in order to inform
changes to the movement regime prior to the spring movements

season. This timescale was challenging due
totheshorttimescalesandlimiteddataavailabletoinformthecostrisk
benefitmodellingrequired. Atopdown
model was therefore developed that captured only the essential
elements of the decision, combining them in
an influence diagram representation of the decision to be made.
As wide a range of experts as possible were
involved in model development, helping inform the structure of
the model, its parameterisation, validation
and interpretation of the results. An Agile approach was
adopted with detail added to the model in a series of
development cycles guided by a steering group.
4.17
Theresultant‘SilentSpread’modelshowedthatfactorssuchastimeto
detectionofdisease,aremuchmoreim-
portantthanlengthofstandstill
indeterminingthesizeofanoutbreak(Risk

Risks Revealed: Understanding Models' Role in Dutch Flood Management

Recommended

Recommended

More Related Content

Similar to Risks Revealed: Understanding Models' Role in Dutch Flood Management

Similar to Risks Revealed: Understanding Models' Role in Dutch Flood Management (20)

More from madlynplamondon

More from madlynplamondon (20)

Recently uploaded

Recently uploaded (20)

Risks Revealed: Understanding Models' Role in Dutch Flood Management