These slides introduce the concept of evidence to counterbalance too much talk about 'data'. It focuses on the socially constructed nature of Evidence, and how it needs to be in the right form at the right time and the right place to have impact on decision making. Evidence is expensive, and relies on legitimising agents. Intro digital world we have many new ways of creating evidence - from the original data sources to the final communication form. What tools and practices are needed to make the creation of evidence more effective and more easily accessible? The slides outlines how a Living Lab and crowdsourcing could be used create new forms of evidence for science and policy

1. Digital
Age
Evidence
and
the
Living
Lab
James
Stewart
Science
Technology
and
Innova<on
Studies
University
of
Edinburgh
j.k.stewart@ed.ac.uk
@jamesks

2. EVIDENCE

9. Common
forms
of
evidence
• Polls
• RCTs
• Official
sta<s<cs
• User
research
• Administra<ve
data
• Prospec<ve
studies
• Detailed
case
study
• Computer
model
• Compe<tor
informa<on
• Expert
knowledge
• Sales
figures
• Military
Intelligence
• Technical
tests
• Guilty
face
and
s<cky
fingers
• Blush

10. OED
Defini<on
1.
The
available
body
of
facts
or
informa<on
indica<ng
whether
a
belief
or
proposi<on
is
true
or
valid.
2.
Informa<on
drawn
from
personal
tes<mony,
a
document,
or
a
material
object,
used
to
establish
facts
in
a
legal
inves<ga<on
or
admissible
as
tes<mony
in
a
law
court.

11. Examples
of
evidence
• An
object
• A
statement
• An
observa<on
• First
hand
accounts
–
text,
video,
recording
• Stories
and
Narra<ves
• Quan<ta<ve
data
• Comparison
• Accepted
knowledge
• Logical
argument
• Analogies
• Theory-­‐backed
proof
• A
model
• Money
• Visualisa<on
• Interac<ve
simula<on
or
model
• Indicators
of
something
in
the
world

12. Evidence
• Resource
for
decision
making
– In
law,
everyday
life,
design,
business,
policy
• Punctual
use
• Con<nual
use
• To
reduce
uncertainty,
• To
shape
opinion
•
Legi7mise
a
decision,
or
lack
of
decision
• Handle
contested
issues

13. A
social
process
and
prac<ce
Legi<mate
evidence
• Experts
and
exper<se
• Method
• Tradi<on
• Scien<sts,
business
people,
poli<cians,
policy
makers,
mangers,
individuals
• (Sociology
of
Science
Knowledge)
Evidence
is
provided
at
the
right
7me
and
in
the
right
place
and
in
a
form
that
can
be
used
by
those
making
decisions,
with
legima7ng
support

14. Norma<ve,
empiricist
approach
• It
is
wrong
always,
everywhere,
and
for
anyone
to
believe
anything
upon
insufficient
evidence
• W.
K.
Clifford
(1879)
Evidence-­‐based
• Natural
Philosophy=Science
• Policy
• Management
• Personal
decision
making
• Medicine
• etc

15. Make
Evident
Obvious
Incontestable

16. How
to
make
something
evident
• Visualisa<on
– Comparison,
Trend,
• Narra<ve
– Chain
of
hypotheses
• Emo<ve
– Witness,
visualisa<on
• Theory
– Scien<fically
proven
hypotheses
• Sta<s<cal
tests
• Experts
and
other
respected
agents
make
evidence
legi<mate
e.g.
Scien<st,
Accountant,
BBC,
Government
Minister

17. Failures
of
evidence
processes
• Costs
–
evidence
is
expensive
• Exper<se
scarce
• Legi<mising
agents
and
processes
can
fail
or
lose
power
• Misuse
–
evidence
is
used
without
cri<cal
examina<on

18. Evidence
in
the
Digital
Age
• New
Visualisa<ons
(interac<ve,
scalable
etc)
• Data
collec<on
at
scale
• ‘Big
Data’
methods
• Easily
accessed
administra<ve
data
• Open
Data
• System
logs
• Computa<onal
models
• Distributed
collec<on
and
analysis
(crowd)

19. Digital
‘crowd’
methods
• Crowdsourced
labour
– Tools
for
distributed
analysis
of
data
as
part
of
human
compu<ng
service
– Data
collec<on
– Reliability,
mo<va<on,
engagement
• Cloud
exper<se
– Access
to
a
global
pool
or
market
of
exper<se
• Ci<zen
Science
– Data
collec<on,
– Analysis
e.g.
classifica<on
– Visualiza<on
and
contextualisa<on

21. ‘Ci<zen’
ini<ated
and
governed
evidence
crea<on
Tools
and
approaches
to
enable
people
to
collect
data,
classify
and
test
the
data,
and
‘make
evident’
in
the
right
form
and
at
the
right
<me
and
place,
with
sufficient
legi<macy
to
influence
decision-­‐making
processes.

22. Why?
• Ocen
there
is
no
or
very
poort
exis<ng
evidence
on
a
topic–
due
to
cost,
lack
of
interest
• Lack
of
trust
in
exis<ng
evidence
–
failure
of
exisi<ng
legi<misa<on
processes
• Evidence
crea<on
is
also
an
engagement
process
–
of
gedng
people
interested
in
a
topic
• It
is
hard
to
bring
legi<mate
evidence
of
the
right
form
to
the
right
place
at
the
right
<me
(hence
lawyers,
scien<sts,
lobbyists
etc).

23. THE
LIVING
LAB

24. What
is
a
Lab
A
Place
to:
Observe
Test
Conduct
Experiments
–
controlled
comparisons,
within
or
against
Theory
or
Laws
Experiment
-­‐
trying
out
new
things
Looking
for
Truth
Crea<ng
and
tes<ng
novelty
Produce
Evidence
and
legi<mising
tools

25. The
Lab
Lab
-­‐
an
infrastructure
for
reuse:
Cupboard
full
of
equipment,
lab
technicians,
reputa<on,
funding
etc

26. From
Lab
to
Living
Lab
Lab
–
highly
controlled
–
but
‘unreal’
‘HCI
living
room’
–pseudo-­‐real
life
Web
experiments
–
in
use,
naturalis<c,
but
limited
to
the
web.
Lived
and
Living
• Allows
par<cipants
to
through
appropria<on
process
of
novel
ideas
and
prac<ces,
as
people
live
in
‘real’
circumstances.
• ‘In
the
wild’
– Digital
city
experiments
–
e.g.
giving
everyone
a
PC
and
classes,
see
what
happens
– Par<cipa<ve
design
prac<ces
• More
generally
a
living,
working
place
for
Genera<ng,
Tes<ng
and
Evalua<ng
interven<ons

27. The
Living
Lab
A
city-­‐scale
lab
infrastructure
for
tes<ng
and
innova<on
ICTs
for
evidence
produc7on,
not
as
interven7ons
• Open
to
different
actors
• Provides
Legi<macy
• Offers
Technical,
methodological,
legal
and
policy
support

28. Edinburgh
Living
Lab
Stair
Treads

29. The
Edinburgh
LL
team
• James
Stewart,
Social
and
Poli<cal
Studies
• Ewan
Klein,
School
of
Informa<cs,
UoE
• Arno
Verhoeven,
School
of
Design,
UoE
– Chris
Speed,
Social
Informa<cs
Edinburgh
University
Crowdsourcing
and
Ci<zen
Science
Network
Many
others…..

30. Edinburgh
Living
Lab
Aims
• Develop
new
means
of
crea7ng
evidence
• Open
to
different
stakeholders
• Design,
development
and
tes<ng
of
interven<ons
• Engaging
stakeholders,
ac<vists,
ci<zens
and
students
Who
• Council
and
University
• Neighbourhood
partnerships
• Third
sector
• Students
–
learning
by
developing,
developing
with
data,
crea<on
of
evidence
with
impact.

31. Pilot:
AcCve
Travel
in
Inverleith

32. Example:
Air
Quality
• Liile
understood,
contested,
poor
data.
• Liile
poli<cal
will
• City
data
collec<on
to
check
regulatory
compliance,
failure
triggering
ac<on.
• Contested
:
Policy
a
poli<cal
balancing
act.
• Ci<zen
engagement
with
Data
and
Pollu<on
models
– Prof
Steve
Yearley

33. Air
Inverleith
Implemen<ng
and
legi<mising
Ci<zen-­‐sourced
air
quality

34. Traffic
movement
–
cycle
movement
Evidence
to
increase
cycling
– Surveys
– Point
counters
– No
actual
journey
data
Require
a
diversity
of
evidence
and
novel
ideas
to
s<mulate
policy
ac<on,
raise
public
and
business
awareness.
Data
collec<on
becomes
awareness
raising
issue.
But
needs
robustness
Break
down
received
wisdom.

35. Paths
by
Beta

36. Brains
on
Bikes
hip://www.bbc.co.uk/news/

37. Opportuni<es
for
the
Living
Lab
• Improve
poor
evidence
currently
available
• Ci<zen
science
approach
–
data
collec<on
and
produc<on
of
evidence
increases
awareness
in
community
and
decision
makers
• Counter-­‐evidence
• Low
cost
of
widespread
data
collec<on,
and
of
analysis

38. Challenges
• Low
par<cipa<on
• ‘weak’
method
v.
exis<ng
evidence
• Problem
of
Legi<misa<on
of
evidence
• Costs
of
city
scale
Lab
• Risk
of
Capture
by
stakeholders

39. Engagement
• How
to
get
people
to
take
part?
– Ac<vists
– Organisa<ons
with
resources
– Organisa<ons
with
power
– Ci<zens
• Expecta<on
that
something
will
be
done!
• Ease
of
par<cipa<on
• Engagement
mechanisms
–
from
Peer-­‐to-­‐peer,
games,
mass
media,
• etc

40. Tools
and
Methods
for
the
Living
Lab
• Quality
Data
Collec<on
• Context
and
Method
• Visualisa<on
• Analysis
• Mo<va<on/Engagement
• Legi<macy
• Right
place
and
right
<me
• Scale
• Re-­‐use
• Sustainability

41. • Your
ideas,
input?
• Please
join
us.
• J.k.stewart@ed.ac.uk
• hip://www.edinburghlivinglab.org
• hip://www.iss<.ed.ac.uk
• hip://www.s<s.ed.ac.uk