Stephanie Piper, Peck Sangiambut
● Current knowledge on evaluation of Open
● A framework to use for evaluation
● Problems with evaluation
Reasons for OD
Open Government, Government 2.0, E-Government,
“A piece of data or content is open if anyone is free to
use, reuse, and redistribute it - subject only, at most, to
the requirements to attribute and/or share-alike”
Technically open and legally open data
What do we hope for open data?
● Increased transparency, accountability
● Citizen empowerment
● Business empowerment
● Increased government - citizen interaction
Open Data Handbook. http://opendatahandbook.org/en/why-open-data/
What current metrics are out there?
● Economic valuations of data
○ a value placed on software (apps for democracy)
○ methods from economics: willingness to pay, contingent valuation (stated
● Evaluating public sector information (PSI)
○ current income from data
○ turnover rates
○ calculation of market size by estimating re-users
○ indices for supply-side evaluation of transparency, accessibility
○ still survey based
● Evaluation of public dialogue/awareness
○ questionnaire surveys
● Partners - cost avoidance
Dekkers, Makx, Polman, Femke, Velde, Robbin te, & Vries, Marc de. (2006). Measuring European Public Sector Information Resources
(MEPSIR). European Commission: Retrieved from http://ec.europa.
Pira International Ltd. (2000). Commercial Exploitation of Europe's Public Sector Information. Retrieved from http://epsiplatform.
Office for Public Management (2012). Evaluation of Public Dialogue on open data: Report to Research Councils UK. Retrieved from http:
Current problems with evaluation
● Why bother evaluating?
● How do we actually evaluate an open data project?
○ what is success?
● Metrics - not enough
○ Downloads and visitor counts not enough
● Anecdotes - not enough
○ People like numbers
● Estimates from the likes of Apps for Democracy - not
● Definition of open data is a hindrance
Value (and value-added)
● Why we need a framework
● A value framework
○ 4 types:
■ Operational and Technical
■ Political and Social
Janssen, M., Charalabidis, Y., & Zuiderwijk, A. (2012). Benefits, Adoption Barriers and Myths of Open Data and Open Government. Information
Systems Management, 29(4), 258–268.
Carpenter, J. and Watts, P. (2013) Assessing the Value of OS OpenData to the Economy of Great Britain – Synopsis. Retrieved from https: