SlideShare a Scribd company logo
1 of 44
Download to read offline
Earth Science Information Partners (ESIP) Workshop July 2017
Neil Beagrie (Charles Beagrie Ltd)
Measuring the Value and
(Economic)
Impact of Research Data Services
Illustration by Jørgen Stamp digitalbevaring.dk CC BY 2.5 Denmark
About us
• Charles Beagrie Ltd – independent consultancy in
digital archive, library, science and research
sectors.
• Neil Beagrie consultant specialising in cost-benefit
analysis, research data curation and digital
preservation, e.g. KRDS projects.
• Prof John Houghton economist specialising in
applying economic methods to information
technology policy, science and technology policy,
and Open Access.
Presentation Overview
• Previous work(note focussing on cost-benefit)
• ESDS Impact Study
• ADS/BADC/EBI Impact Studies
• CESSDA Cost-Benefit Advocacy Toolkit
• Key Challenges and Lessons
Informal – happy to take questions as we go
Why measure value and
(economic) impact?
• Advocacy: proving (to non-researchers) that current services
or future services are a worthwhile investment;
• Complementary to other approaches such as data/article
citation analysis;
• Can show more of the impact on research, teaching &
learning, etc.
Note however:
• It is challenging to do;
• Methods well-established but impact data hard to obtain and
to use.
Previous Work
Big Science and Innovation Study for BIS July 2013
• Desk review of c. 100 studies internationally;
• 3 studies highlighted to BIS as being particularly good
examples of ‘good practice’ in the measurement of
economic impacts:
• Berkeley Lab 2010
• Human Genome Project 2011
• Economic and Social Data Service (ESDS) 2012
The ESDS Study
Best Practice from ESDS study
• Applies range of
methods;
• Includes counter-
factual;
• Data collection:
users & depositors,
research & teaching;
• Data weighting -
survey value
responses weighted
to reflect the overall
pattern of use from
weblogs;
• Case studies/ KRDS
benefits illustrate
benefits and impact
pathways.
ESDS Study:
Returns on Investment (ROI)
ROI - service Benefit/cost ratio of
net economic value to
ESDS operational costs
ROI - research data creation
scenarios -Increase in returns
from data re-use
facilitated by ESDS (counterfactual)
5.4 to 1
Up to 10 to 1
ESDS Study:
Researcher Efficiency Gains
Impact of using ESDS data and services on research efficiency
(after Beagrie et al 2012, p77, Figure 15)
Economic and Social Research Council © 2012 CC-BY licensed
The Value and Impact Studies
Economic & Social Data Service (ESDS)
Archaeology Data Service
British Atmospheric Data Centre
European Bioinformatics Institute
Value + Impact Analyses
of Data Services
John Houghton + Neil Beagrie 4 joint studies to date.
5th in progress. Methods applied to:
Approaches and Methods
• We combined quantitative and qualitative approaches.
• All studies used:
• Desk-based analysis of existing literature and reports;
• Existing management and internal data collected by the data
services; and
• Original data collection in the form of online surveys, together with
semi-structured interviews.
Indicative Scale (currently £1.00 = $1.31)
The Value and Impact of the
British Atmospheric Data Centre
Key FindingsThe economic analysis shows that:
• increase in research efficiency estimated as at least £10 million per
annum.
• Increase in ROI from the data re-use facilitated by BADC estimated
as between £11 million and £34 million over thirty years (net present
value) from one-year’s investment.
The qualitative analysis in the user survey showed that:
• 81% of the academic respondents reported that BADC was very or
extremely important for their academic research, and
• 53% of respondents reported that it would have a major or severe
impact on their work if they could not access BADC data and
services.
In the depositor survey:
• 52% suggested that being unable to deposit data with BADC would
have a major or severe impact on them.
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
CESSDA SaW Toolkit
ESDS Study: Context
• UK Data Archive one of the largest European
social science data archives: perhaps the largest
• UK Data Archive established for 40 years: has
built up collections and users over time
• Only economic impact study for any social
science data archive to date
• How to extend work for other social science
archives?
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
CESSDA-SaW Task 4.6
Capturing and communicating the value and
economic impact of social science data services.
Develop a benefit/cost advocacy programme and
supporting tools; assembling an evidence base to support
the negotiation with ministries and funding organisations;
support advocacy with other core stakeholders such as
data creators and data users.
Toolkit published April 2017.
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
CESSDA-SaW
User Requirements Survey
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
Survey Q2
THE DATA SERVICE'S CURRENT STAFFING IS
APPROXIMATELY:
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
Toolkit Components
• Factsheets
• ROI, Benefits, Costs
• Worksheets
• Benefits Summary for a Data Archive
• Archive Development Canvas
• Case studies
• ADP, FSD, LiDA, UKDS
• Selected External Tools
• CCeX, KRDS, CDMA, ESDS Impact, ADP surveys , etc
• User Guide
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
The Factsheets
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
ROI Factsheet (1)
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
ROI Factsheet (2)
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
Counter-factuals – “Costs of Inaction”
“Ideally, economic impact assessments should estimate the
counterfactual – i.e. what would occur in the absence of the
facility…However, counterfactuals are rarely addressed in the
[c.100] studies reviewed due to lack of data. We found two
exceptions that address this issue partially. One is the
evaluation of the economic impacts of ESDS (2012) which
partially explores the counterfactual through a users’
survey…Another exception is a review of economic impacts of
large-scale science facilities in the UK (SQW, 2008) …
however, this estimation is not done rigorously and relies
mostly on the estimation of the local benefits.”
Big Science and Innovation - Report to BIS - Technopolis
2013
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
ROI Factsheet (3)
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
Benefits
Factsheet (1)
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
Benefits
Factsheet (2)
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
Benefits Factsheet (3)
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
Costs
Factsheet (1)
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
Costs
Factsheet (2)
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
Bringing it all together:
The Archive Development Canvas
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
The Archive Development Canvas (detailed)
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
The Archive Development Canvas (mapped)
Key Challenges and Lessons
EBI Report: User population
• How ‘actual direct users’ were estimated from web stats:
• Unique host log of 10.8 million (Annual figure for 2014) adjusted
to reflect key considerations:
• People behind unique IPs in web stats usually over-estimated by a
factor of 7 (Formitchev, 2010)
• EMBL-EBI accessed by organisational IPs (survey: 4185 responses
from 3622 unique IPs): Factor in 1.16 per unique IP
• Respondents accessed 9 services on average – 10.8 million figure
captured across EMBL-EBI resources
• Total user population estimate: 198,000 unique, actual
direct users
Online Surveys
(BADC)
• Two independent online surveys were conducted by CB
Ltd;
• Survey questionnaires developed iteratively - small group
of BADC customers helped test pilot versions;
• Given the nature of subject and diverse customer base
substantial effort was needed to design questionnaires;
• Significant effort spent to reduce the likely burden on
questionnaire recipients.
Online surveys (BADC)
Significant time spent on:
• wording of the invitations to participate
• advance notice of the surveys via websites, news feeds, social
media and email shots
Five £50 Amazon vouchers offered in a draw for participants
Surveys had high response rates and completion rates
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
Toolkit challenges & lessons
• diverse scale / type of organisations
• complexity of topic / local resources
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
Effort Grading Levels
hours days months
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement number 674939.
Design criteria
• Easy for overloaded individuals/ smaller
services
• Short documents
• Good Infographics
• Synthesis
• Making existing tools easier to use/tailored to
(social science) data services
Conclusions
• Value and (economic) impact analysis doable but
challenging;
• Difficulties and critical points for economic analysis include:
• Limited knowledge of real user populations;
• Survey sample and survey design;
• Complexity of mature data services and their resources;
• "Costs of inaction" data - counter-factuals for data
services.
• CESSDA SaW toolkit approach adaptable to other disciplines
and can tailor for/utilise sector specific data and needs.
Questions?

More Related Content

What's hot

Cross border procurement of e-infrastructure services
Cross border procurement of e-infrastructure services Cross border procurement of e-infrastructure services
Cross border procurement of e-infrastructure services Helix Nebula The Science Cloud
 
Opening Comments - Financial incentives to promote local ownership and invest...
Opening Comments - Financial incentives to promote local ownership and invest...Opening Comments - Financial incentives to promote local ownership and invest...
Opening Comments - Financial incentives to promote local ownership and invest...Environmental Protection Agency, Ireland
 
EPA H2020 SC5 Info Day: Research and Innovation Needs and potential Impacts:...
 EPA H2020 SC5 Info Day: Research and Innovation Needs and potential Impacts:... EPA H2020 SC5 Info Day: Research and Innovation Needs and potential Impacts:...
EPA H2020 SC5 Info Day: Research and Innovation Needs and potential Impacts:...Environmental Protection Agency, Ireland
 
Partnership as the driver of R&D and innovation
Partnership as the driver of R&D and innovationPartnership as the driver of R&D and innovation
Partnership as the driver of R&D and innovationSteve Rogers
 
Cost and Value analysis of digital data archiving
Cost and Value analysis of digital data archivingCost and Value analysis of digital data archiving
Cost and Value analysis of digital data archiving Anna Palaiologk
 
Demystifying European Funding: From FP7 to Horizon 2020
Demystifying European Funding: From FP7 to Horizon 2020Demystifying European Funding: From FP7 to Horizon 2020
Demystifying European Funding: From FP7 to Horizon 2020David Young
 
Coote Marine SDI benefits quantification - blue economy 20150414
Coote   Marine SDI benefits quantification - blue economy 20150414Coote   Marine SDI benefits quantification - blue economy 20150414
Coote Marine SDI benefits quantification - blue economy 20150414Andy Coote
 
2021 SEAI RD&D Funding Call Webinar
2021 SEAI RD&D Funding Call Webinar 2021 SEAI RD&D Funding Call Webinar
2021 SEAI RD&D Funding Call Webinar SustainableEnergyAut
 
hackAIR - Open technology to map the air we breathe
hackAIR  - Open technology to map the air we breathehackAIR  - Open technology to map the air we breathe
hackAIR - Open technology to map the air we breathehackAIR
 
Building the Energy Efficiency Financing Market
Building the Energy Efficiency Financing MarketBuilding the Energy Efficiency Financing Market
Building the Energy Efficiency Financing MarketSteven Fawkes
 
What’s in it for us? CERIF and Research Information Management
What’s in it for us? CERIF and Research Information ManagementWhat’s in it for us? CERIF and Research Information Management
What’s in it for us? CERIF and Research Information ManagementJISC infoNet
 
Open Data Ireland Public Meeting
Open Data Ireland Public MeetingOpen Data Ireland Public Meeting
Open Data Ireland Public MeetingDerilinx
 
Session 3 - OECD - Andrew Park
Session 3 - OECD - Andrew ParkSession 3 - OECD - Andrew Park
Session 3 - OECD - Andrew ParkOECD Governance
 
HORIZON 2020
HORIZON 2020HORIZON 2020
HORIZON 2020Maghrenov
 
EPA H2020 SC5 Info Day: Irish Successes – Experiences - Tips for applicants ...
 EPA H2020 SC5 Info Day: Irish Successes – Experiences - Tips for applicants ... EPA H2020 SC5 Info Day: Irish Successes – Experiences - Tips for applicants ...
EPA H2020 SC5 Info Day: Irish Successes – Experiences - Tips for applicants ...Environmental Protection Agency, Ireland
 
Supporting transition towards the Paris Agreement using TIMES: insights from ...
Supporting transition towards the Paris Agreement using TIMES: insights from ...Supporting transition towards the Paris Agreement using TIMES: insights from ...
Supporting transition towards the Paris Agreement using TIMES: insights from ...IEA-ETSAP
 
Progress & outlook cce network
Progress & outlook cce network Progress & outlook cce network
Progress & outlook cce network ccef2f
 

What's hot (20)

Cross border procurement of e-infrastructure services
Cross border procurement of e-infrastructure services Cross border procurement of e-infrastructure services
Cross border procurement of e-infrastructure services
 
Opening Comments - Financial incentives to promote local ownership and invest...
Opening Comments - Financial incentives to promote local ownership and invest...Opening Comments - Financial incentives to promote local ownership and invest...
Opening Comments - Financial incentives to promote local ownership and invest...
 
1 uhlir sharif
1 uhlir sharif1 uhlir sharif
1 uhlir sharif
 
EPA H2020 SC5 Info Day: Research and Innovation Needs and potential Impacts:...
 EPA H2020 SC5 Info Day: Research and Innovation Needs and potential Impacts:... EPA H2020 SC5 Info Day: Research and Innovation Needs and potential Impacts:...
EPA H2020 SC5 Info Day: Research and Innovation Needs and potential Impacts:...
 
Partnership as the driver of R&D and innovation
Partnership as the driver of R&D and innovationPartnership as the driver of R&D and innovation
Partnership as the driver of R&D and innovation
 
Cost and Value analysis of digital data archiving
Cost and Value analysis of digital data archivingCost and Value analysis of digital data archiving
Cost and Value analysis of digital data archiving
 
Demystifying European Funding: From FP7 to Horizon 2020
Demystifying European Funding: From FP7 to Horizon 2020Demystifying European Funding: From FP7 to Horizon 2020
Demystifying European Funding: From FP7 to Horizon 2020
 
Coote Marine SDI benefits quantification - blue economy 20150414
Coote   Marine SDI benefits quantification - blue economy 20150414Coote   Marine SDI benefits quantification - blue economy 20150414
Coote Marine SDI benefits quantification - blue economy 20150414
 
2021 SEAI RD&D Funding Call Webinar
2021 SEAI RD&D Funding Call Webinar 2021 SEAI RD&D Funding Call Webinar
2021 SEAI RD&D Funding Call Webinar
 
6 craglia
6 craglia6 craglia
6 craglia
 
hackAIR - Open technology to map the air we breathe
hackAIR  - Open technology to map the air we breathehackAIR  - Open technology to map the air we breathe
hackAIR - Open technology to map the air we breathe
 
Building the Energy Efficiency Financing Market
Building the Energy Efficiency Financing MarketBuilding the Energy Efficiency Financing Market
Building the Energy Efficiency Financing Market
 
What’s in it for us? CERIF and Research Information Management
What’s in it for us? CERIF and Research Information ManagementWhat’s in it for us? CERIF and Research Information Management
What’s in it for us? CERIF and Research Information Management
 
Open Data Ireland Public Meeting
Open Data Ireland Public MeetingOpen Data Ireland Public Meeting
Open Data Ireland Public Meeting
 
Session 3 - OECD - Andrew Park
Session 3 - OECD - Andrew ParkSession 3 - OECD - Andrew Park
Session 3 - OECD - Andrew Park
 
HORIZON 2020
HORIZON 2020HORIZON 2020
HORIZON 2020
 
EPA H2020 SC5 Info Day: Irish Successes – Experiences - Tips for applicants ...
 EPA H2020 SC5 Info Day: Irish Successes – Experiences - Tips for applicants ... EPA H2020 SC5 Info Day: Irish Successes – Experiences - Tips for applicants ...
EPA H2020 SC5 Info Day: Irish Successes – Experiences - Tips for applicants ...
 
9th IWERN Meeting Presentation
9th IWERN Meeting Presentation9th IWERN Meeting Presentation
9th IWERN Meeting Presentation
 
Supporting transition towards the Paris Agreement using TIMES: insights from ...
Supporting transition towards the Paris Agreement using TIMES: insights from ...Supporting transition towards the Paris Agreement using TIMES: insights from ...
Supporting transition towards the Paris Agreement using TIMES: insights from ...
 
Progress & outlook cce network
Progress & outlook cce network Progress & outlook cce network
Progress & outlook cce network
 

Similar to Measuring the Value and Impact of Research Data Services

Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...
Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...
Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...Sitra / Hyvinvointi
 
Updated presentation on OA in H2020 and ERA (JF Dechamp, UNESCO workshop, Ber...
Updated presentation on OA in H2020 and ERA (JF Dechamp, UNESCO workshop, Ber...Updated presentation on OA in H2020 and ERA (JF Dechamp, UNESCO workshop, Ber...
Updated presentation on OA in H2020 and ERA (JF Dechamp, UNESCO workshop, Ber...Jean-François Dechamp
 
Ia4 si caps concertation presentation
Ia4 si caps concertation presentationIa4 si caps concertation presentation
Ia4 si caps concertation presentationCAPS2020
 
Horizon 2020 Introduction, Peter Walters, FP7UK National Contact Point Team
Horizon 2020 Introduction, Peter Walters, FP7UK National Contact Point Team Horizon 2020 Introduction, Peter Walters, FP7UK National Contact Point Team
Horizon 2020 Introduction, Peter Walters, FP7UK National Contact Point Team Invest Northern Ireland
 
HORIZON 2020 Overview - Robert-Jan Smits - Presentation - Israel 3.2.2014
HORIZON 2020 Overview - Robert-Jan Smits - Presentation - Israel 3.2.2014HORIZON 2020 Overview - Robert-Jan Smits - Presentation - Israel 3.2.2014
HORIZON 2020 Overview - Robert-Jan Smits - Presentation - Israel 3.2.2014ISERD Israel
 
Health and Wellbeing Living Lab Symposium Presentations
Health and Wellbeing Living Lab Symposium PresentationsHealth and Wellbeing Living Lab Symposium Presentations
Health and Wellbeing Living Lab Symposium PresentationsVITALISEProject
 
4 siambekou h2020_mare_5-12-2014_
4 siambekou h2020_mare_5-12-2014_4 siambekou h2020_mare_5-12-2014_
4 siambekou h2020_mare_5-12-2014_MARE EuroMed
 
Christiana Siambekou - National Documentation Centre (EKT)
Christiana Siambekou - National Documentation Centre (EKT)Christiana Siambekou - National Documentation Centre (EKT)
Christiana Siambekou - National Documentation Centre (EKT)WWW.ERFC.GR
 
The New EU Framework Programme for Research and Innovation
The New EU Framework Programme for Research and InnovationThe New EU Framework Programme for Research and Innovation
The New EU Framework Programme for Research and Innovationmathgear
 
Enterprise Europe Network | C-Energy Input to SG IEE | Piotr Swiatek
Enterprise Europe Network | C-Energy Input to SG IEE | Piotr SwiatekEnterprise Europe Network | C-Energy Input to SG IEE | Piotr Swiatek
Enterprise Europe Network | C-Energy Input to SG IEE | Piotr SwiatekInvest Northern Ireland
 
horizon2020-presentation
horizon2020-presentationhorizon2020-presentation
horizon2020-presentationEmilio Mulet
 
H2020. Criterios de evaluación y consejos prácticos para la elaboración de pr...
H2020. Criterios de evaluación y consejos prácticos para la elaboración de pr...H2020. Criterios de evaluación y consejos prácticos para la elaboración de pr...
H2020. Criterios de evaluación y consejos prácticos para la elaboración de pr...CTAEX
 
Anaxo1wwwwwwwww
Anaxo1wwwwwwwwwAnaxo1wwwwwwwww
Anaxo1wwwwwwwwwamautajara
 
Horizon Europe Clean Transport Webinar - Cluster 5 Destination 5 | Slides
Horizon Europe Clean Transport Webinar - Cluster 5 Destination 5 | SlidesHorizon Europe Clean Transport Webinar - Cluster 5 Destination 5 | Slides
Horizon Europe Clean Transport Webinar - Cluster 5 Destination 5 | SlidesKTN
 
NordForsk Open Access Reykjavik 14-15/8-2014: H2020
NordForsk Open Access Reykjavik 14-15/8-2014: H2020NordForsk Open Access Reykjavik 14-15/8-2014: H2020
NordForsk Open Access Reykjavik 14-15/8-2014: H2020NordForsk
 
Opportunities in Horizon 2020 for ICT, Dr Robert Bunn, Invest NI
Opportunities in Horizon 2020 for ICT, Dr Robert Bunn, Invest NI Opportunities in Horizon 2020 for ICT, Dr Robert Bunn, Invest NI
Opportunities in Horizon 2020 for ICT, Dr Robert Bunn, Invest NI Invest Northern Ireland
 
Exploiting the Added Value of Climate Services - From Climate Service Concept...
Exploiting the Added Value of Climate Services - From Climate Service Concept...Exploiting the Added Value of Climate Services - From Climate Service Concept...
Exploiting the Added Value of Climate Services - From Climate Service Concept...Environmental Protection Agency, Ireland
 
Open Science at the University of Edinburgh
Open Science at the University of EdinburghOpen Science at the University of Edinburgh
Open Science at the University of EdinburghUniversity of Edinburgh
 

Similar to Measuring the Value and Impact of Research Data Services (20)

Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...
Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...
Gaia-X and how to accelerate growth – pathway to EU funding webinar 10 March ...
 
Updated presentation on OA in H2020 and ERA (JF Dechamp, UNESCO workshop, Ber...
Updated presentation on OA in H2020 and ERA (JF Dechamp, UNESCO workshop, Ber...Updated presentation on OA in H2020 and ERA (JF Dechamp, UNESCO workshop, Ber...
Updated presentation on OA in H2020 and ERA (JF Dechamp, UNESCO workshop, Ber...
 
Ia4 si caps concertation presentation
Ia4 si caps concertation presentationIa4 si caps concertation presentation
Ia4 si caps concertation presentation
 
Horizon 2020 Introduction, Peter Walters, FP7UK National Contact Point Team
Horizon 2020 Introduction, Peter Walters, FP7UK National Contact Point Team Horizon 2020 Introduction, Peter Walters, FP7UK National Contact Point Team
Horizon 2020 Introduction, Peter Walters, FP7UK National Contact Point Team
 
HORIZON 2020 Overview - Robert-Jan Smits - Presentation - Israel 3.2.2014
HORIZON 2020 Overview - Robert-Jan Smits - Presentation - Israel 3.2.2014HORIZON 2020 Overview - Robert-Jan Smits - Presentation - Israel 3.2.2014
HORIZON 2020 Overview - Robert-Jan Smits - Presentation - Israel 3.2.2014
 
10.richard burger
10.richard burger10.richard burger
10.richard burger
 
Kutsch, Werner: RINGO highlight
Kutsch, Werner: RINGO highlightKutsch, Werner: RINGO highlight
Kutsch, Werner: RINGO highlight
 
Health and Wellbeing Living Lab Symposium Presentations
Health and Wellbeing Living Lab Symposium PresentationsHealth and Wellbeing Living Lab Symposium Presentations
Health and Wellbeing Living Lab Symposium Presentations
 
4 siambekou h2020_mare_5-12-2014_
4 siambekou h2020_mare_5-12-2014_4 siambekou h2020_mare_5-12-2014_
4 siambekou h2020_mare_5-12-2014_
 
Christiana Siambekou - National Documentation Centre (EKT)
Christiana Siambekou - National Documentation Centre (EKT)Christiana Siambekou - National Documentation Centre (EKT)
Christiana Siambekou - National Documentation Centre (EKT)
 
The New EU Framework Programme for Research and Innovation
The New EU Framework Programme for Research and InnovationThe New EU Framework Programme for Research and Innovation
The New EU Framework Programme for Research and Innovation
 
Enterprise Europe Network | C-Energy Input to SG IEE | Piotr Swiatek
Enterprise Europe Network | C-Energy Input to SG IEE | Piotr SwiatekEnterprise Europe Network | C-Energy Input to SG IEE | Piotr Swiatek
Enterprise Europe Network | C-Energy Input to SG IEE | Piotr Swiatek
 
horizon2020-presentation
horizon2020-presentationhorizon2020-presentation
horizon2020-presentation
 
H2020. Criterios de evaluación y consejos prácticos para la elaboración de pr...
H2020. Criterios de evaluación y consejos prácticos para la elaboración de pr...H2020. Criterios de evaluación y consejos prácticos para la elaboración de pr...
H2020. Criterios de evaluación y consejos prácticos para la elaboración de pr...
 
Anaxo1wwwwwwwww
Anaxo1wwwwwwwwwAnaxo1wwwwwwwww
Anaxo1wwwwwwwww
 
Horizon Europe Clean Transport Webinar - Cluster 5 Destination 5 | Slides
Horizon Europe Clean Transport Webinar - Cluster 5 Destination 5 | SlidesHorizon Europe Clean Transport Webinar - Cluster 5 Destination 5 | Slides
Horizon Europe Clean Transport Webinar - Cluster 5 Destination 5 | Slides
 
NordForsk Open Access Reykjavik 14-15/8-2014: H2020
NordForsk Open Access Reykjavik 14-15/8-2014: H2020NordForsk Open Access Reykjavik 14-15/8-2014: H2020
NordForsk Open Access Reykjavik 14-15/8-2014: H2020
 
Opportunities in Horizon 2020 for ICT, Dr Robert Bunn, Invest NI
Opportunities in Horizon 2020 for ICT, Dr Robert Bunn, Invest NI Opportunities in Horizon 2020 for ICT, Dr Robert Bunn, Invest NI
Opportunities in Horizon 2020 for ICT, Dr Robert Bunn, Invest NI
 
Exploiting the Added Value of Climate Services - From Climate Service Concept...
Exploiting the Added Value of Climate Services - From Climate Service Concept...Exploiting the Added Value of Climate Services - From Climate Service Concept...
Exploiting the Added Value of Climate Services - From Climate Service Concept...
 
Open Science at the University of Edinburgh
Open Science at the University of EdinburghOpen Science at the University of Edinburgh
Open Science at the University of Edinburgh
 

More from Neil Beagrie

Datanomics: the value of research data.
 Datanomics: the value of research data.  Datanomics: the value of research data.
Datanomics: the value of research data. Neil Beagrie
 
20yrs: 2013 Screening the Future
20yrs: 2013 Screening the Future20yrs: 2013 Screening the Future
20yrs: 2013 Screening the FutureNeil Beagrie
 
20yrs: 2007 Brussels Digital Preservation: Setting the Course for a Decade of...
20yrs: 2007 Brussels Digital Preservation: Setting the Course for a Decade of...20yrs: 2007 Brussels Digital Preservation: Setting the Course for a Decade of...
20yrs: 2007 Brussels Digital Preservation: Setting the Course for a Decade of...Neil Beagrie
 
20yrs: 2005_warwick3
20yrs: 2005_warwick3 20yrs: 2005_warwick3
20yrs: 2005_warwick3 Neil Beagrie
 
20yrs: 2004 jisc cni-brighton
20yrs: 2004 jisc cni-brighton20yrs: 2004 jisc cni-brighton
20yrs: 2004 jisc cni-brightonNeil Beagrie
 
20yrs: 2004 iPRES Beijing e-journals
20yrs: 2004 iPRES Beijing e-journals20yrs: 2004 iPRES Beijing e-journals
20yrs: 2004 iPRES Beijing e-journalsNeil Beagrie
 
20yrs: 2001 Preservation Management of Digital Materials [the Digital Preserv...
20yrs: 2001 Preservation Management of Digital Materials [the Digital Preserv...20yrs: 2001 Preservation Management of Digital Materials [the Digital Preserv...
20yrs: 2001 Preservation Management of Digital Materials [the Digital Preserv...Neil Beagrie
 
20yrs:1998 Society of Archivists Conference
20yrs:1998 Society of Archivists Conference20yrs:1998 Society of Archivists Conference
20yrs:1998 Society of Archivists ConferenceNeil Beagrie
 

More from Neil Beagrie (9)

Datanomics: the value of research data.
 Datanomics: the value of research data.  Datanomics: the value of research data.
Datanomics: the value of research data.
 
20yrs: 2013 Screening the Future
20yrs: 2013 Screening the Future20yrs: 2013 Screening the Future
20yrs: 2013 Screening the Future
 
20yrs: 2010 KRDS
20yrs: 2010 KRDS20yrs: 2010 KRDS
20yrs: 2010 KRDS
 
20yrs: 2007 Brussels Digital Preservation: Setting the Course for a Decade of...
20yrs: 2007 Brussels Digital Preservation: Setting the Course for a Decade of...20yrs: 2007 Brussels Digital Preservation: Setting the Course for a Decade of...
20yrs: 2007 Brussels Digital Preservation: Setting the Course for a Decade of...
 
20yrs: 2005_warwick3
20yrs: 2005_warwick3 20yrs: 2005_warwick3
20yrs: 2005_warwick3
 
20yrs: 2004 jisc cni-brighton
20yrs: 2004 jisc cni-brighton20yrs: 2004 jisc cni-brighton
20yrs: 2004 jisc cni-brighton
 
20yrs: 2004 iPRES Beijing e-journals
20yrs: 2004 iPRES Beijing e-journals20yrs: 2004 iPRES Beijing e-journals
20yrs: 2004 iPRES Beijing e-journals
 
20yrs: 2001 Preservation Management of Digital Materials [the Digital Preserv...
20yrs: 2001 Preservation Management of Digital Materials [the Digital Preserv...20yrs: 2001 Preservation Management of Digital Materials [the Digital Preserv...
20yrs: 2001 Preservation Management of Digital Materials [the Digital Preserv...
 
20yrs:1998 Society of Archivists Conference
20yrs:1998 Society of Archivists Conference20yrs:1998 Society of Archivists Conference
20yrs:1998 Society of Archivists Conference
 

Recently uploaded

Food_safety_Management_pptx.pptx in microbiology
Food_safety_Management_pptx.pptx in microbiologyFood_safety_Management_pptx.pptx in microbiology
Food_safety_Management_pptx.pptx in microbiologyHemantThakare8
 
Combining Asynchronous Task Parallelism and Intel SGX for Secure Deep Learning
Combining Asynchronous Task Parallelism and Intel SGX for Secure Deep LearningCombining Asynchronous Task Parallelism and Intel SGX for Secure Deep Learning
Combining Asynchronous Task Parallelism and Intel SGX for Secure Deep Learningvschiavoni
 
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...Sérgio Sacani
 
AICTE activity on Water Conservation spreading awareness
AICTE activity on Water Conservation spreading awarenessAICTE activity on Water Conservation spreading awareness
AICTE activity on Water Conservation spreading awareness1hk20is002
 
Pests of Sunflower_Binomics_Identification_Dr.UPR
Pests of Sunflower_Binomics_Identification_Dr.UPRPests of Sunflower_Binomics_Identification_Dr.UPR
Pests of Sunflower_Binomics_Identification_Dr.UPRPirithiRaju
 
Total Legal: A “Joint” Journey into the Chemistry of Cannabinoids
Total Legal: A “Joint” Journey into the Chemistry of CannabinoidsTotal Legal: A “Joint” Journey into the Chemistry of Cannabinoids
Total Legal: A “Joint” Journey into the Chemistry of CannabinoidsMarkus Roggen
 
Anatomy of Duodenum- detailed ppt presentation for science students
Anatomy of Duodenum- detailed ppt presentation for science studentsAnatomy of Duodenum- detailed ppt presentation for science students
Anatomy of Duodenum- detailed ppt presentation for science studentsgargipatil3210
 
Understanding Nutrition, 16th Edition pdf
Understanding Nutrition, 16th Edition pdfUnderstanding Nutrition, 16th Edition pdf
Understanding Nutrition, 16th Edition pdfHabibouKarbo
 
EGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer Zahana
EGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer ZahanaEGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer Zahana
EGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer ZahanaDr.Mahmoud Abbas
 
Loudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptxLoudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptxpriyankatabhane
 
Timeless Cosmology: Towards a Geometric Origin of Cosmological Correlations
Timeless Cosmology: Towards a Geometric Origin of Cosmological CorrelationsTimeless Cosmology: Towards a Geometric Origin of Cosmological Correlations
Timeless Cosmology: Towards a Geometric Origin of Cosmological CorrelationsDanielBaumann11
 
Think Science: What Are Eclipses (101), by Craig Bobchin
Think Science: What Are Eclipses (101), by Craig BobchinThink Science: What Are Eclipses (101), by Craig Bobchin
Think Science: What Are Eclipses (101), by Craig BobchinNathan Cone
 
DETECTION OF MUTATION BY CLB METHOD.pptx
DETECTION OF MUTATION BY CLB METHOD.pptxDETECTION OF MUTATION BY CLB METHOD.pptx
DETECTION OF MUTATION BY CLB METHOD.pptx201bo007
 
CDS Fundamentals of digital communication system UNIT 1 AND 2.pdf
CDS Fundamentals of digital communication system UNIT 1 AND 2.pdfCDS Fundamentals of digital communication system UNIT 1 AND 2.pdf
CDS Fundamentals of digital communication system UNIT 1 AND 2.pdfshubhangisonawane6
 
Role of Gibberellins, mode of action and external applications.pptx
Role of Gibberellins, mode of action and external applications.pptxRole of Gibberellins, mode of action and external applications.pptx
Role of Gibberellins, mode of action and external applications.pptxjana861314
 
DNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptxDNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptxGiDMOh
 
Production technology of Brinjal -Solanum melongena
Production technology of Brinjal -Solanum melongenaProduction technology of Brinjal -Solanum melongena
Production technology of Brinjal -Solanum melongenajana861314
 
BACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
BACTERIAL SECRETION SYSTEM by Dr. Chayanika DasBACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
BACTERIAL SECRETION SYSTEM by Dr. Chayanika DasChayanika Das
 

Recently uploaded (20)

Food_safety_Management_pptx.pptx in microbiology
Food_safety_Management_pptx.pptx in microbiologyFood_safety_Management_pptx.pptx in microbiology
Food_safety_Management_pptx.pptx in microbiology
 
Combining Asynchronous Task Parallelism and Intel SGX for Secure Deep Learning
Combining Asynchronous Task Parallelism and Intel SGX for Secure Deep LearningCombining Asynchronous Task Parallelism and Intel SGX for Secure Deep Learning
Combining Asynchronous Task Parallelism and Intel SGX for Secure Deep Learning
 
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
Observation of Gravitational Waves from the Coalescence of a 2.5–4.5 M⊙ Compa...
 
Bioenergetics and the role of ATP to drive the beats of life.
Bioenergetics and the role of ATP to drive the beats of life.Bioenergetics and the role of ATP to drive the beats of life.
Bioenergetics and the role of ATP to drive the beats of life.
 
AICTE activity on Water Conservation spreading awareness
AICTE activity on Water Conservation spreading awarenessAICTE activity on Water Conservation spreading awareness
AICTE activity on Water Conservation spreading awareness
 
Pests of Sunflower_Binomics_Identification_Dr.UPR
Pests of Sunflower_Binomics_Identification_Dr.UPRPests of Sunflower_Binomics_Identification_Dr.UPR
Pests of Sunflower_Binomics_Identification_Dr.UPR
 
Total Legal: A “Joint” Journey into the Chemistry of Cannabinoids
Total Legal: A “Joint” Journey into the Chemistry of CannabinoidsTotal Legal: A “Joint” Journey into the Chemistry of Cannabinoids
Total Legal: A “Joint” Journey into the Chemistry of Cannabinoids
 
Introduction Classification Of Alkaloids
Introduction Classification Of AlkaloidsIntroduction Classification Of Alkaloids
Introduction Classification Of Alkaloids
 
Anatomy of Duodenum- detailed ppt presentation for science students
Anatomy of Duodenum- detailed ppt presentation for science studentsAnatomy of Duodenum- detailed ppt presentation for science students
Anatomy of Duodenum- detailed ppt presentation for science students
 
Understanding Nutrition, 16th Edition pdf
Understanding Nutrition, 16th Edition pdfUnderstanding Nutrition, 16th Edition pdf
Understanding Nutrition, 16th Edition pdf
 
EGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer Zahana
EGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer ZahanaEGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer Zahana
EGYPTIAN IMPRINT IN SPAIN Lecture by Dr Abeer Zahana
 
Loudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptxLoudspeaker- direct radiating type and horn type.pptx
Loudspeaker- direct radiating type and horn type.pptx
 
Timeless Cosmology: Towards a Geometric Origin of Cosmological Correlations
Timeless Cosmology: Towards a Geometric Origin of Cosmological CorrelationsTimeless Cosmology: Towards a Geometric Origin of Cosmological Correlations
Timeless Cosmology: Towards a Geometric Origin of Cosmological Correlations
 
Think Science: What Are Eclipses (101), by Craig Bobchin
Think Science: What Are Eclipses (101), by Craig BobchinThink Science: What Are Eclipses (101), by Craig Bobchin
Think Science: What Are Eclipses (101), by Craig Bobchin
 
DETECTION OF MUTATION BY CLB METHOD.pptx
DETECTION OF MUTATION BY CLB METHOD.pptxDETECTION OF MUTATION BY CLB METHOD.pptx
DETECTION OF MUTATION BY CLB METHOD.pptx
 
CDS Fundamentals of digital communication system UNIT 1 AND 2.pdf
CDS Fundamentals of digital communication system UNIT 1 AND 2.pdfCDS Fundamentals of digital communication system UNIT 1 AND 2.pdf
CDS Fundamentals of digital communication system UNIT 1 AND 2.pdf
 
Role of Gibberellins, mode of action and external applications.pptx
Role of Gibberellins, mode of action and external applications.pptxRole of Gibberellins, mode of action and external applications.pptx
Role of Gibberellins, mode of action and external applications.pptx
 
DNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptxDNA isolation molecular biology practical.pptx
DNA isolation molecular biology practical.pptx
 
Production technology of Brinjal -Solanum melongena
Production technology of Brinjal -Solanum melongenaProduction technology of Brinjal -Solanum melongena
Production technology of Brinjal -Solanum melongena
 
BACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
BACTERIAL SECRETION SYSTEM by Dr. Chayanika DasBACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
BACTERIAL SECRETION SYSTEM by Dr. Chayanika Das
 

Measuring the Value and Impact of Research Data Services

  • 1. Earth Science Information Partners (ESIP) Workshop July 2017 Neil Beagrie (Charles Beagrie Ltd) Measuring the Value and (Economic) Impact of Research Data Services Illustration by Jørgen Stamp digitalbevaring.dk CC BY 2.5 Denmark
  • 2. About us • Charles Beagrie Ltd – independent consultancy in digital archive, library, science and research sectors. • Neil Beagrie consultant specialising in cost-benefit analysis, research data curation and digital preservation, e.g. KRDS projects. • Prof John Houghton economist specialising in applying economic methods to information technology policy, science and technology policy, and Open Access.
  • 3. Presentation Overview • Previous work(note focussing on cost-benefit) • ESDS Impact Study • ADS/BADC/EBI Impact Studies • CESSDA Cost-Benefit Advocacy Toolkit • Key Challenges and Lessons Informal – happy to take questions as we go
  • 4. Why measure value and (economic) impact? • Advocacy: proving (to non-researchers) that current services or future services are a worthwhile investment; • Complementary to other approaches such as data/article citation analysis; • Can show more of the impact on research, teaching & learning, etc. Note however: • It is challenging to do; • Methods well-established but impact data hard to obtain and to use.
  • 5. Previous Work Big Science and Innovation Study for BIS July 2013 • Desk review of c. 100 studies internationally; • 3 studies highlighted to BIS as being particularly good examples of ‘good practice’ in the measurement of economic impacts: • Berkeley Lab 2010 • Human Genome Project 2011 • Economic and Social Data Service (ESDS) 2012
  • 7. Best Practice from ESDS study • Applies range of methods; • Includes counter- factual; • Data collection: users & depositors, research & teaching; • Data weighting - survey value responses weighted to reflect the overall pattern of use from weblogs; • Case studies/ KRDS benefits illustrate benefits and impact pathways.
  • 8. ESDS Study: Returns on Investment (ROI) ROI - service Benefit/cost ratio of net economic value to ESDS operational costs ROI - research data creation scenarios -Increase in returns from data re-use facilitated by ESDS (counterfactual) 5.4 to 1 Up to 10 to 1
  • 9. ESDS Study: Researcher Efficiency Gains Impact of using ESDS data and services on research efficiency (after Beagrie et al 2012, p77, Figure 15) Economic and Social Research Council © 2012 CC-BY licensed
  • 10. The Value and Impact Studies
  • 11. Economic & Social Data Service (ESDS) Archaeology Data Service British Atmospheric Data Centre European Bioinformatics Institute Value + Impact Analyses of Data Services John Houghton + Neil Beagrie 4 joint studies to date. 5th in progress. Methods applied to:
  • 12. Approaches and Methods • We combined quantitative and qualitative approaches. • All studies used: • Desk-based analysis of existing literature and reports; • Existing management and internal data collected by the data services; and • Original data collection in the form of online surveys, together with semi-structured interviews.
  • 13. Indicative Scale (currently £1.00 = $1.31)
  • 14. The Value and Impact of the British Atmospheric Data Centre
  • 15. Key FindingsThe economic analysis shows that: • increase in research efficiency estimated as at least £10 million per annum. • Increase in ROI from the data re-use facilitated by BADC estimated as between £11 million and £34 million over thirty years (net present value) from one-year’s investment. The qualitative analysis in the user survey showed that: • 81% of the academic respondents reported that BADC was very or extremely important for their academic research, and • 53% of respondents reported that it would have a major or severe impact on their work if they could not access BADC data and services. In the depositor survey: • 52% suggested that being unable to deposit data with BADC would have a major or severe impact on them.
  • 16. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. CESSDA SaW Toolkit
  • 17. ESDS Study: Context • UK Data Archive one of the largest European social science data archives: perhaps the largest • UK Data Archive established for 40 years: has built up collections and users over time • Only economic impact study for any social science data archive to date • How to extend work for other social science archives?
  • 18. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. CESSDA-SaW Task 4.6 Capturing and communicating the value and economic impact of social science data services. Develop a benefit/cost advocacy programme and supporting tools; assembling an evidence base to support the negotiation with ministries and funding organisations; support advocacy with other core stakeholders such as data creators and data users. Toolkit published April 2017.
  • 19. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. CESSDA-SaW User Requirements Survey
  • 20. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. Survey Q2 THE DATA SERVICE'S CURRENT STAFFING IS APPROXIMATELY:
  • 21. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. Toolkit Components • Factsheets • ROI, Benefits, Costs • Worksheets • Benefits Summary for a Data Archive • Archive Development Canvas • Case studies • ADP, FSD, LiDA, UKDS • Selected External Tools • CCeX, KRDS, CDMA, ESDS Impact, ADP surveys , etc • User Guide
  • 22. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. The Factsheets
  • 23. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. ROI Factsheet (1)
  • 24. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. ROI Factsheet (2)
  • 25. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. Counter-factuals – “Costs of Inaction” “Ideally, economic impact assessments should estimate the counterfactual – i.e. what would occur in the absence of the facility…However, counterfactuals are rarely addressed in the [c.100] studies reviewed due to lack of data. We found two exceptions that address this issue partially. One is the evaluation of the economic impacts of ESDS (2012) which partially explores the counterfactual through a users’ survey…Another exception is a review of economic impacts of large-scale science facilities in the UK (SQW, 2008) … however, this estimation is not done rigorously and relies mostly on the estimation of the local benefits.” Big Science and Innovation - Report to BIS - Technopolis 2013
  • 26. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. ROI Factsheet (3)
  • 27. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. Benefits Factsheet (1)
  • 28. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. Benefits Factsheet (2)
  • 29. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. Benefits Factsheet (3)
  • 30. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. Costs Factsheet (1)
  • 31. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. Costs Factsheet (2)
  • 32. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. Bringing it all together: The Archive Development Canvas
  • 33. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939.
  • 34. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. The Archive Development Canvas (detailed)
  • 35. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. The Archive Development Canvas (mapped)
  • 37. EBI Report: User population • How ‘actual direct users’ were estimated from web stats: • Unique host log of 10.8 million (Annual figure for 2014) adjusted to reflect key considerations: • People behind unique IPs in web stats usually over-estimated by a factor of 7 (Formitchev, 2010) • EMBL-EBI accessed by organisational IPs (survey: 4185 responses from 3622 unique IPs): Factor in 1.16 per unique IP • Respondents accessed 9 services on average – 10.8 million figure captured across EMBL-EBI resources • Total user population estimate: 198,000 unique, actual direct users
  • 38. Online Surveys (BADC) • Two independent online surveys were conducted by CB Ltd; • Survey questionnaires developed iteratively - small group of BADC customers helped test pilot versions; • Given the nature of subject and diverse customer base substantial effort was needed to design questionnaires; • Significant effort spent to reduce the likely burden on questionnaire recipients.
  • 39. Online surveys (BADC) Significant time spent on: • wording of the invitations to participate • advance notice of the surveys via websites, news feeds, social media and email shots Five £50 Amazon vouchers offered in a draw for participants Surveys had high response rates and completion rates
  • 40. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. Toolkit challenges & lessons • diverse scale / type of organisations • complexity of topic / local resources
  • 41. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. Effort Grading Levels hours days months
  • 42. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 674939. Design criteria • Easy for overloaded individuals/ smaller services • Short documents • Good Infographics • Synthesis • Making existing tools easier to use/tailored to (social science) data services
  • 43. Conclusions • Value and (economic) impact analysis doable but challenging; • Difficulties and critical points for economic analysis include: • Limited knowledge of real user populations; • Survey sample and survey design; • Complexity of mature data services and their resources; • "Costs of inaction" data - counter-factuals for data services. • CESSDA SaW toolkit approach adaptable to other disciplines and can tailor for/utilise sector specific data and needs.