This document discusses lessons learned from using prosperity indicators in policymaking and proposes how a platform called Policy Compass could contribute. It covers: 1. A brief history of indicator use in policy from the 1940s onward and concerns about GDP limitations. 2. Attempts to measure prosperity holistically using environmental and social factors beyond just economic metrics. 3. A proposed framework in Policy Compass for defining community-generated indicators, visualizing trends, annotating with events, and constructing causal models to simulate policy impacts. 4. The potential for Policy Compass to experiment with indicators using open data and crowd-source indicator definition, models and deliberation to provide more nuanced and locally relevant measures of societal welfare

1. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
Lessons Learnt from the Use of Prosperity Indicators in
Policy Making: Towards Community-Generated
Indicators
Ourania Markaki,
Panagiotis Kokkinakos, Sotirios Koussouris, Costas Koutras,
John Psarras
National Technical University of Athens
Greece

2. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
Prosperity Indicators and Policy Making
• With the advent of evidence-based policy making and the growing demand for
government accountability, there is a heavy exploitation of indicators in the
public policy arena.
• ‘40s: Evaluation of the US economy in terms of the ‘Monthly Economic
Indicators’
• ‘60s: ‘social indicators’ movement, an explosion of indicators for social
change
• The idea of using social indicators passed gradually to the large international
organizations, e.g. the UN, OECD, etc.
• early ‘90s: ‘community indicators’ movement’, motivated by the global
questions on environmental matters
• Today: the Web 2.0 and the Open Data Movement open a new arena of
experimentation with social indicators.
An indicator is regularly conceived as a sort of ‘statistical measure’ that can
adequately capture crucial aspects of a (social) phenomenon that should be
monitored, in particular when a specific policy measure is enforced to affect it

3. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
On the definition of Prosperity
• Prosperity is frequently used as a synonym to ‘welfare’, ‘well-being’ or
‘quality of life’ (=the product of the interplay among social, health, economic
and environmental conditions affecting human and social development).
• Well-being (or prosperity) does not necessarily equal ‘economic growth’.
• Infinite growth, and thereby endlessly increasing production and
consumption, is impossible on a finite planet.
• The GDP is one of the most successful indicators in economics
(+) Captures succinctly the value of all goods in the economy
(+) Known to the majority of citizens and the totality of people in economics and political science
(-) Does not capture the correct prices for some goods (e.g. state provided healthcare)
(-) Does not reflect the technological quality improvements
(-) Fails to reflect a lot of things about everyday life
“…GDP measures everything, ...except that which makes life worthwhile”
(R. Kennedy, 1968)

4. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
The beyond GDP Discussion
• Environmental and social factors have to be considered as well in measuring
prosperity to ensure sustainable development.
• Attempts to construct prosperity / sustainable development indicators:
– The environmental indicators of the Pressure-State-Response (framework (OECD,
1993).
– Genuine Saving Indicator – A nation’s wealth is determined as the combination of
produced assets, natural capital and human resources (World Bank, 1995, 1997).
– 134 indicators capturing social, economic, environmental and institutional
dimensions of sustainable development (UN Conference on Environment and
Development, 1996).
– Human and ecosystem well-being are equally important for achieving sustainable
development (Barometer of Sustainability, 2001).
– The Ecological Footprint measures humanity’s demand on biologically productive
land and water required to produce the resources needed and absorb carbon dioxide
emissions (Global Footprint Network, 2003).
– The Commission on the Measurement of Economic Performance and Social
Progress provided legitimacy to the criticism raised about the GDP.

5. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
Trends in Indicator Design
• Constructing a single, composite indicator to capture a quality-of-life
dimension.
• Using multiple, separate, indicators for social problems, e.g. crime rate,
poverty level, air pollution, unemployment rate, etc.
• Preparing all-inclusive indicator reports, produced in a collaborative fashion
and intended for wide distribution. Decision-makers and policy-analysts may
consult these in an iterative fashion.
• The trend of re-inventing government:
– dialogue in the design/ use of measures of performance and customer satisfaction
with government and their interpretation in a complex, changing context.
– use of indicators to facilitate the work of many players to make better choices, solve
problems, and be better able to respond to context and change.
• The use of local indicators, parallel to international, national and regional ones
seems to be gaining ground.

6. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
A Methodological Framework of Indicator Development
[Coombes, Wong, 1994]
Step 1: Conceptual consolidation
Clarifying the concept to be represented by the analysis
Step 2: Analytical Structuring
Providing an analytical framework within which indicators will be collated and analysed
Step 3: Identification of Indicators
Translation of key factors identified in step 2 into specific measurable indicators
Step 4: Synthesis of indicator values
Synthesizing the identified indicators into composite index/indices or into analytical summary

7. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
Concerns in Indicator Design
• A well-defined and useful indicator should comprise (OECD, 2014):
– Policy relevance: address issues that are of actual/potential public concern
relevant to policymaking.
– Analytical soundness: being based on the best available science
– Measurability: reflect reality on a timely and accurate basis, and be
measurable at a reasonable cost
• Institutionalization: setup routine procedures and practices to ensure the
continuing existence of an indicator and to legitimize the method and concept of
the measure.
• Avoiding bias: indicators should be produced by professional statistical
agencies that have a strong awareness of policy issues, without having
responsibility for them.
• Maintain the sensitive balance between (global) standardization and local
democracy (identifying the relevant measure to the specific local
circumstances).
• Address the fundamental question of integrating prosperity indicators in the
policy lifecycle (causality dimension).

8. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
A Quick Recap
Citizens formulate their own judgement on the status and progress of
prosperity in an environment, characterised by:
•Controversy around the definition of prosperity
•Diversity on the scope and scale of application
•Indicator design methodological issues
•Legitimacy concerns
•A multitude of indicator development initiatives and approaches of
different interest or even conflicting nature

9. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
How can Policy Compass contribute?
Open new possibilities towards employing quantitative techniques to
circumscribe social phenomena and evaluate the results of planned or enforced
policy measures at local or regional level.
•Explore the limits of social computing with quantitative indicators for policy
design and assessment, given the unprecedented access to the open data
sources available.
•Enhance the experimentation with various kinds of social indicators, ranging
from well-known and widely established metrics, such as the GDP and its variants,
to the composite indicators which can suitably apply to the regional or municipality
level.
•Experiment on the cross-fertilisation of today’s ICT capabilities, with the ideas and
intuition of the social (and political/economic) motivation of describing societal
welfare with well-defined, representative metrics.
•Import the cause/effect component of policy analysis, directly into the
indicator analysis process through the ‘injection’ of causal analysis tools in the
Policy Compass platform and methodology.

10. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
Policy Compass Ingredients
 I. Open Public Data
 II. Prosperity Indicators
 III. Fuzzy Cognitive Maps
(FCMs)
 IV. Argumentation Technology
 V. Deliberation Platforms and
Social Media
Key Platform Functionalities
• Prosperity indicators Construction
• Metrics Visualization
• Trend lines Annotation
• Causal Models Development
• Policy Models Simulation
• Sharing/Debating Metrics/Models
• Structured Surveys
How can Policy Compass contribute?

11. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
The Policy Compass Prosperity Indicator
Framework

12. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
Search for
relevant open
data sets on the
World Wide Web
Import data
sets to the
platform
Search for indicators/data sets,
already uploaded by other users
Select the
indicators/data sets
that best capture the
concepts that reflect
the users’ perception
of prosperity
Visualize indicators over
time or for selected
geographical regions
Compare
indicator
trend lines Search the platform database for
recorded historical events
Search and import information
on historical events
Annotate trend lines with
events to indicate time
points where graphs
show abnormalities
Indicators Definition
Calculate the
indicators’ values,
based on open data
sets
Define the relative weights of
the selected indicators/data
sets and synthesize them
into a composite indicator
Need to
evaluate
political
actions
Need to
evaluate
political
actions

13. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
Causal Models Construction
Seek for existing policy models, correlating
policies with their effects
Edit or construct a new policy model by
graphically creating concept variables and
relating them through chains of cause and
effect to formulate causal networks, and
incorporating fuzzy degrees of causality to
model the strength of the causal
relationships identified.
Simulate the
policy model to
confirm or reject a
causal theory
Indicators
Definition

14. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
Deliberation & Argumentation
Push developed
metrics, policy
models and
visualization
snapshots to the
platform
deliberation space
Share developed metrics, policy models and visualization
snapshots in Web 2.0 channels
Initiate a new
discussion or
participate in an
existing
deliberation
topic
Produce the
discussion
argumentation
map
Obtain feedback for initiating another
usage scenario …
Indicators
Definition
Causal
Models
Construction

15. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
Practical Usage Insights
• Where: Focused Project Workshops
• Who: Researchers, students, e-Government experts, public sector
representatives
• Demo Scenario:
– Development of a composite indicator to assess the quality of tertiary education
• Public expenditure on Education
• Ratio of academic staff involved against students
• Ratio of tertiary education graduates against new entrants
– Effect of education-related policies (lifelong learning, vocational programs, etc.) on
unemployment
• Participant Feedback:
– Open data as a means of holding governments and politicians accountable
– Credibility of data from unofficial sources
– Great value in creating own prosperity indicators and causal policy models
– Strong analytical skills required
– Argumentation and deliberation as the main entry point for citizens

16. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
Conclusions and Lessons Learnt
• There exists a growing concern on the limits of GDP-like measures for
measuring societal welfare. GDP is very successful, but it is impossible to
derive deep results on many aspects of everyday life with simple aggregate
indicators alone.
• It makes perfect sense to attempt constructing aggregate, headline or
composite indicators, to measure important social phenomena at the regional
/local level.
• There do not exist ‘objective’ or ‘neutral’ indicators of any kind. The community
does not pursue objective indicators, it rather attempts to construct useful ones.
• A broad experimentation on the calculation and exploitation of indicators, given
the multitude of data available, is definitely interesting scientifically and will
provide useful "social" feedback.

17. Session 3a, 25 November 2015 eChallenges e-2015 Copyright 2015 Policy Compass
Ourania Markaki, omarkaki@epu.ntua.gr
Thank you!