The document discusses principles for proper data management and reuse from the perspective of the Research Data Alliance (RDA). It notes that RDA has over 2000 members with diverse opinions. There is an ongoing discussion around trends in data practices and principles that most members agree with, such as data needing to be findable, accessible, combinable and interpretable by others. The document outlines some results from RDA working groups, including a common data model using persistent identifiers, a data type registry, a generic application programming interface for persistent identifier records, and a set of best practice policies for typical data management and data processing tasks.

1. Principles for proper Data Management and
Re-Use – an RDA view
Peter Wittenburg
Max Planck Society

2. 2
 does RDA have one view – yes & no
 RDA is basically a bottom-up organization driven by the many
“creative” minds who want to change data practices
 RDA has now about 2000 members – so we have 2000 opinions?
 we have an intensive discussion process since 2012 (ICRI
Conference Copenhagen) and we can see that there are a number
of trends and principles all or most seem to agree with
 still RDA is a very young initiative and needs
much attention and grease
Clarification

3. 3
Why is this all relevant?
 Naoyuki Tsunematsu (JST ):
• Data exchange (and thus the need for proper data
management) difficult to convey in Japanese Science
• parallel trends observed for Japanese Science
• not so often included in collaborations anymore
• not so often represented in the top papers
• enormous decrease in international ranking
• serious worries about counterproductive encapsulation
• this concern seems to be relevant for all of us

4. 4
Trends I – Volume, Complexity
from simple
structures ...
... towards
complex
relationships

5. 5
Trends II - Anonymity
direct exchange between known colleagues
Domain of Repositories

6. 6
Trends III – Re-Usage
Domain of
trusted
Repositories
• Data will be re-used in different contexts
• Data needs to be findable, accessible, combinable and
interpretable for others

7. 7
Data Practices I – Survey
 ~120 Interviews/Interactions
 2 Workshops with Leading Scientists (EU, US)
 too much manual or via ad hoc scripts
 too much in Legacy formats (no PID & MD)
 there are lighthouse projects etc. but ...
 DM and DP not efficient and too expensive
(Biologist for 75% of his time data manager)
 federating data incl. logical information much too expensive
 hardly usage of automated workflows and lack of
reproducibility

8. 8
Data Practices I – Survey
 ~120 Interviews/Interactions
 2 Workshops with Leading Scientists (EU, US)
 too much manual or via ad hoc scripts
 too much in Legacy formats (no PID & MD)
 there are lighthouse projects etc. but ...
 DM and DP not efficient and too expensive
(Biologist for 75% of his time data manager)
 federating data incl. logical information much too expensive
 hardly usage of automated workflows and lack of
reproducibility

9. 9
12 21 26
95 95 96 97
266
676
DIF DwC DC EML FGDC Open
GIS
ISO My Lab none
Metadata standards
Data Practices III - Metadata
slide von Bill Michener, DataONE

10. 10
 lack of proper documentation,
schemas, semantics, relations, etc.
 directory structures, spreadsheets etc.
are ad hoc creations and knowledge
fades away
 etc.
Data Practices II – Data Entropy

11. 11
Community Center
Common Data
Center
Changes needed – EUDAT and others
many excellent projects
are working on
changes: ESFRI
projects, DataNet
projects, e-
Infrastructures, national
projects
RDA needs to build on
experiences and
expertise

12. 12
RDA widely agreed I – time to change
 management of data objects is widely type and discipline
independent
 still every project defines its own strategies leading to huge stack of
software that will not be maintainable

13. 13
RDA widely agreed II –time to change
what
Value Added
Services
Data
Sources
Persistent
Identifiers
Persistent
Reference
Analysis Citation
Apps
Custom
Clients
Plug-Ins
Resolution System Typing
PID
Local Storage Cloud Computed
Data Sets RDBMS Files
Digital Objects
PID record
attributes
bit sequence
(instance)
metadata
attributes
points to instances
describes properties
describes
properties
& context
point to
each other

14. 14
RDA Results I: common data model
• PIDs at the beginning of trust chain
• have a worldwide, independent and robust PID system
worldwide (DONA Handles – DOIs are Handles)!
• metadata are essential in anonymous data world
taken from RDA WG Data
Foundation & Terminology

15. 15
 result: a registry for data types
 you get an unknown file,
pull it on DTR and content is being
visualized
 extended MIME Type concept
 no free lunch: someone needs to
register and define type
 code available begin 2015
 PIT Demo already working with
DTR
RDA Results II: Data Type Registry
Federated Set of
Type Registries
Visualization
Data Processing10100
11010
101…. Data Set
Dissemination
10100
11010
101….
10100
11010
101….
Terms:…
Rights
Agree
Visualization
Processing
Interpretation
3
Domain of
Services
2
1
Human or Machine
Consumers
4
• NIST is already working with
communities on fargoing ideas

16. 16
 result: a generic API and a set of basic attributes
 a PID Record is like a Passport (Number, Photo, Exp-Date, etc.)
 if all PID Service-Provider agree on one API and talk the same language
(registered terms) SW development will become easy
 Test-Installation
in operation
together with
DTR
RDA Results III: PID Information Types
LOC location, path
CKSM checksum
CKSM_T checksum type
RoR owning repository
MD path to MD

17. 17
 due to unforeseen circumstances need until P5
 Practical Policies = executable Workflow Statements
 result at P5: a set of Best Practice PPs for a number of typical DM/DP
tasks (Integrity Check, Replication, etc.)
 currently a large collection of PPs, currently being evaluated
 you could add your policies
RDA Results IV: Practical Policies
replication policy X
replication policy Y
integrity policy A
integrity policy B
integrity policy C
md extraction policy l
md extraction policy k
etc.
Policy Inventory
Repository
selection
implementation
execution
data manager

18. 18
 need to place many RDA WGs & IGs on a common landscape since
finally everything needs to fit together -> Data Fabric
RDA ongoing: Data Fabric

19. 19
1973
Changes take long ...
1990 1993
TCP/IP
Specification
1977
TCP/IP
Stress-test
WWW-Mosaic
available
worldwide
adoption
 many different suggestion & protocols
 first no advantage for TCP/IP
 at the beginning discussion about different email systems
 at the beginning no interest from researchers and also industry
(toi of some freaks)
 required some top-down decisions to enforce unification
20 years!

20. 20
RDA is about global bridge building
20
RDA is about building the social and technical bridges that
enable global open sharing of data.
Researchers, scientists, data practitioners from around the
world are invited to work together to achieve the vision
Funders: NSF, EC, AU Gov, Japan, Brazil, DE?, UK?, ZA?, FI?,
etc.

21. 21
Thanks for your attention.
http://www.rd-alliance.org
http://europe.rd-alliance.org

22. 22
 see Science 2.0 Initiative of EC
 nr. of researchers increases enormously
 there is a pressure in the direction of Grand Challenges
and those topics relevant for societies
 research is increasingly often data intensive
 border-crossing research is a fact (countries, disciplines)
 faster cycles (hypothesis – analysis – publications –
reviews)
Trends IV: research is changing

23. 23
bottom-up
process
top-down
process
uptake
to come
RDA is about global bridge building

24. 24
EUDAT Services
24
EUDAT Box
dropbox-like service
easy sharing
local synching
Semantic Anno
checking , referencing and
annotating
Dynamic Data
immediate handling
Generic Workflow
automating data
processing
B2DROP B2NOTE