Presentation of Marcus Dapp / Fortiss at CeDEM16 confrence

1. Institute for Public Information Management
fortiss GmbH
An-Institut Technische Universität München
Measuring the promise of Open Data:
Development of the Impact Monitoring Framework
CeDEM16, Krems, 2016-05-18
Marcus M. Dapp, dapp@fortiss.org, @digisus

2. Institute for Public Information Management
Berlin, 2016-04-18CeDEM16, KRems2
Prof. Dr. Helmut Krcmar
Lehrstuhl für Wirtschaftsinformatik
Technische Universität München
Scientific Director fortiss
Dr. Marcus M. Dapp
fortiss – An-Institut der TU München
Guerickestraße 25
80805 München
dapp@fortiss.org | @digisus | 089 3603522 19
• eGovernment MONITOR (DACH)
• Open Data Potential für Deutschland

3. www.dataportals.org

4. • Mostly qualitative
• Anecdotal evidence
• Case studies
• Some ranking schemes
• Open Data Barometer – expert opinion
• Open Data Census – expert and non-expert („crowd“) opinion
• Open Government Partnership – national action plans
• Gap identified
• Ubaldi (OECD): 96% of governments (N=16) have NOT “adopted a
methodology to measure return on investment on open government.”
Measuring Open Data Impact (I)

5. • … is hard
• Easy-to-measure metrics (data downloads) do not tell anything about impact
and impactful activities (e.g. startups) are not easy to measure.
• Hard to trace links between cause and effect
• Long time lag between cause and effect
• Complex relationships between causes and effects
• … is needed
• To support the effective and efficient use of resources
• To focus on high-impact Open Data activities
• To increase the success of new Open Data initiatives
Measuring Open Data Impact (II)

6. • While ROI approach measures only profit, SROI includes the
economic, socio-economic, and social value of an enterprise.
(introduced to measure social value of philanthropic investments)
• “Theory of change”: resources (input), transformed into controllable
results (output), can foster indirect activities (outcome) and lead to
value-creating consequences (impact). Impact map and indicators
used to capture these causal relationships across all stakeholders.
• IDEA: Impact Monitoring Framework combines the SROI
technique with open data impact literature to suggest a new
approach.
Concept: Social Return on Investment

7. A „Theory of Change“ for Open (Gov) Data
INPUT all resources such as money, people, equipment, and facilities used by an organization to publish
open data, plus all pre-existing native data. “native data” because it is a natural part of the
organization and available e.g. in proprietary formats and software tools not yet ready to be published
as “raw open data”.
OUTPUT direct and tangible deliverables produced by the organization; mainly the setup and operation of an
open data portal. Quality of portals differs widely today: quality of metadata, completeness of
datasets, platform’s accessibility and visibility, usability and comprehensibility, timeliness, value and
usefulness as well as granularity.
OUTCOME all direct and indirect consequences of certain output actions by the organization; all activities by
the re-users of open data: engineers, entrepreneurs, citizens, journalists, scientists, artists or
administrators re-using the available open data in some form. Activities include hackathons,
visualizations, web and mobile applications, new open-data-based business models, data journalism,
research projects etc. Uptake of open data cannot be controlled by the organization releasing the
data, it can only support and encourage data re-use.
IMPACT outcome adjusted for the effects that would have occurred without publishing data by the
organization. Only results caused by releasing open data are counted. In practice, it is not easy to
measure the impact of specific output. Guidelines suggest using comparison groups or benchmarks.
value
chain

8. Data categories
(G8 Charta)

9. Impact Monitoring Framework (examples)

10. Discussion
• Model critique
• SROI is very costly (teim, data) to be meaningful
• We need to test the framework
• We are looking for „open data projects/initiatives“ as cases for empirical
testing
• Do re-use activity patterns and data category correlate?
• If yes: Coming up with generic activity types most suited for a data category
would help governments and re-users to be more effective.
• Complete the framework by adding monetization
• SROI ratio = Social impact (monetized) / Investment (monetized), e.g. 1:3