Presentation on developing DMP services at the University of Edinburgh. It outlines progress on the DMP objectives in the Edinburgh RDM roadmap and covers the findings from evaluating DMPonline. The talk was given at a HEIDS meeting in Edinburgh on 22nd April 2013.
1. Data Management Plans
Implementing the Edinburgh RDM roadmap
Sarah Jones
Digital Curation Centre
sarah.jones@glasgow.ac.uk
Twitter: sjDCC
HEIDS meeting, 22nd April 2013, Edinburgh
2. Outline
Significance of data management planning
Progress on the Edinburgh roadmap
- Tailored assistance
- Customised DMPonline
Next steps at Edinburgh
3. Research funders‟ data policies
www.dcc.ac.uk/resources/policy-and-legal/ overview-funders-data-policies
4. What do research funders want?
A brief plan submitted in grant applications, and in the
case of NERC, a more detailed plan once funded
1-3 sides of A4 as attachment or a section in Je-S form
Typically a prose statement covering suggested themes
An outline of data management and sharing
plans, justifying decisions and any limitations
5. Five common themes / questions
1. Description of data to be collected / created
(i.e. content, type, format, volume...)
2. Standards / methodologies for data collection and
management
3. Ethics and Intellectual Property
(highlight any restrictions e.g. embargoes, confidentiality issues)
4. Plans for data sharing and access
(i.e. how, when, to whom)
5. Strategy for long-term preservation
6. Edinburgh policy requirements
“All new research proposals must include research
data management plans or protocols that explicitly
address data
capture, management, integrity, confidentiality, rete
ntion, sharing and publication.”
www.ed.ac.uk/is/research-data-policy
7. Benefits for researchers
Developing DMPs can help researchers to:
Make informed decisions to anticipate and avoid problems
Avoid duplication, data loss and security breaches
Develop procedures early on for consistency
Ensure data are accurate, complete, reliable and secure
Save time and effort to make their lives easier!
8. Benefits for the University
DMPs allow the University to find out:
What volume of research data will be created
How will researchers share data with collaborators
Security requirements for sensitive data
Whether additional support is needed so this can be
costed into grant applications
A description of the data to populate data registers?
9. DCC support via DMPonline
DMPonline is a web-based tool to help researchers write DMPs.
- based on the DCC Checklist for a DMP
- requirements are mapped to DCC questions
Create a plan
based on
relevant
funder /
institutional
templates...
...and then
answer the
questions using
the guidance
provided
10. Outline
Significance of data management planning
Progress on the Edinburgh roadmap
- Tailored assistance
- Customised DMPonline
Next steps at Edinburgh
11. DMP objectives & deliverables
1. Tailored DMP assistance for PIs submitting research
proposals
- Library of successful DMP examples
- DMP „response team‟ if required
2. Customise DMPonline for optimal UoE use
- University branded DMPonline
- Boilerplate text about IS services
12. Tailored assistance: activity
Collect and analyse example DMPs from ERI
Identity requirements for consultancy services
Trial one-to-one DMP consultations
Desire for tailored support seems to be mainly in Arts &
Humanities. Will test this further and collate more
example DMPs as we roll out services by School.
13. Customise DMPonline: activity
Evaluate DMPonline with researchers
Make changes in light of recommendations
Develop UoE customisation of DMPonline
Main activity has been the evaluation of DMPonline.
Major changes have been requested and are underway.
14. Evaluating DMPonline: inputs
We have engaged researchers and research support staff, mostly
at Edinburgh but some feedback came from elsewhere.
Tool analytics (i.e. statistics on DMPonline usage)
Emails and discussions with „ed.ac.uk‟ users
General email / blog comments (mostly from Jisc MRD)
Focus groups - one at Edinburgh and one with Jisc MRD
Guided interviews with researchers after using the tool
Usability testing
15. User feedback
POSITIVE COMMENTS CONCERNS RAISED
Good to have an online tool Difficulty understanding the
tool and concept of mappings
Technically well-coded
Too detailed and in-depth
Liked „sharing‟ feature
Output too long to submit
Demand for customisations
Issues with design & workflows
Has provided an impetus
Desire to feed into plans – what‟s the minimum you can
DMPonline community actually get away with
16. Example interview feedback
Q: Pick five terms that describe your experience:
Hard to Use
Time-consuming
Complex
High Quality
Powerful
to be honest I didn't actually make use of DMP
online - after registering and determining that it
was going to be more complicated than it was
worth, I pursued an alternative means of putting
together a data management plan
17. Main issues
Checklist too long and has irrelevant questions
Mappings cause confusion and misinterpretation
Desire to ask own questions not use DCC ones
Templates confusing – which to pick?
Users want to see funder requirement in full first
Not enough boilerplate text / examples
18. Common pitfalls in software design
www.sandraandwoo.com/2012/11/19/0430-software-engineering-now-with-cats
19. Planned changes
Shorten the DMP Checklist
Overhaul how the Checklist is used in the tool
(n.b. this is a major architectural change so redevelopment will take time)
Reintroduce a wizard to help users select templates
Summarise requirements prior to writing the DMP
Revise registration, sharing & export workflows
Add more boilerplate text and examples as guidance
Make a number of interface / usability improvements
20. Progress on revising DMPonline
Checklist reworked and taxonomy created
Funder templates revised
New process for institutional customisations proposed
UI changes contracted out and underway
Database redesign nearly complete
Development plan in place – delivery of v4 expected in June
Current status, March 2013: www.dcc.ac.uk/blog/dmponline-current-status
DMPonline future plans, Jan 2013: www.dcc.ac.uk/news/future-plans-dmponline
21. Outline
Significance of data management planning
Progress on the Edinburgh roadmap
- Tailored assistance
- Customised DMPonline
Next steps at Edinburgh
22. Still to do
Finish redeveloping DMPonline and test with users
Define what questions need to be asked at Edinburgh
Provide boilerplate text and links to Edinburgh RDM services
Roll out by School, and in doing so:
- Develop disciplinary / local guidance
- Collate example DMPs to share with others
- Check whether tailored support is required. If so, can this be
embedded in School provision?
23. Any questions?
Sarah Jones
Digital Curation Centre
sarah.jones@glasgow.ac.uk
Twitter: sjDCC
Editor's Notes
We have gathered a handful of example plans from ERI, have asked researchers about consultancy when reviewing DMPonline, and have run a one-to-one session with an Arts researcher. The data library has done more of this in the past. We have also worked with Mike Mineter who’s developing support in GeoSciences.
The main effort to date has been in evaluating DMPonline and making changes in light of recommendations.Major architectural change is underway. This is a big task, knocking delivery dates back a few months.
We initially contacted users registered with an Edinburgh email address, asking them about their experience of using the tool. We also ran focus groups, guided interviews. Usability testing was conducted with people who had no prior experience of using the tool.
People liked having an online tool and thought it functioned very well technically. Users particularly liked the ability to share plans to collaboratively write them. There’s been a lot of demand for institutional customisations and the broader community are positive about DCC work in this area and want to feed into future developments.The main issue faced was understanding the tool, particularly the mappings. Users found it difficult to know what the tool would do for them, how to start writing a plan, and felt the mappings sometimes led to questions being misinterpreted. Once researchers started writing plans they found it too detailed and a bit overwhelming. This also meant the output could be too long to submit to funders. There were some teething troubles with certain workflows and UI design too.Users are looking for the quickest, easiest way to write a DMP. They want to be presented with the bare minimum and lots of boilerplate text / examples to help write a plan.
He found the tool hard to use and overly complex, but conversely said it was high quality and powerful. Technically it functioned well, but he had reservations about come of the concepts behind the tool (i.e. the mappings).The respondent expected a more straight forward tool. He ended up calling the Research Office to send him examples of filled-in data plans, which he then reused/tweaked.
Although there are a number of major changes to make, we’ve simply fallen into a number of common pitfalls when designing software.There have been lots of demands for additional features as we’ve developed v1 - v3 and this can obscure the main function and make it hard for users to understand and navigate the tool.We’re going to strip back to basics to make DMPonline easier to use.
The major change is overhauling how the Checklist will be used in the tool. It will no longer be upfront and visible. We will ask funder / institutional questions directly rather than mapping these to DCC equivalents which were asked instead.
A lot of the background work needed has already been completed. The major task is now redeveloping the tool over the coming months.
We need to sit down as a group to determine what should be asked in DMPs at Edinburgh – do you want to know about data volumes, access needs, security etc.? Is there a desire to gather info to prepopulate the data register?We then need to develop custom guidance with IS staff so researchers are aware of available support and use it.When we roll out by School we will develop any additional guidance needed, collate further DMPs and probe more about the need for tailored support. Where there is demand for one-to-one consultations to write DMPs, we will explore whether this can this be met by within the School e.g. by local IT, lab technicians (example of Mike Mineter developing support in GeoSciences)