Highlighting Fitness-For-Use of Published Biodiversity Data

HIGHLIGHTING FITNESS-FOR-USE OF
PUBLISHED BIODIVERSITY DATA
Javier Otegui, Arturo H. Ariño
University of Navarra

JAVIER OTEGUI & ARTURO H. ARIÑO: HIGHLIGHTING FITNESS-FOR-USE OF PUBLISHEDBIODIVERSITY DATA. TDWG2012, BEIJING, 22-X-2012
PUBLISHING DATA
• Data published in papers
• Data papers published
• Data published

WHAT, WHERE, WHEN
OUR TARGET DATA
Primary Biodiversity Data Record
PBR

WHAT, WHERE, WHEN
PBR
Megaptera novaehollandiae
Adult female, live
Off North Truro, MA, USA
42.101 N, 70.169 W
2010.09.29 21:47 GMT
Arturo H. Ariño
Aboard Dolphin VI
Canon Eos 450D, 200 mm lens
un

GBIF GEOREFERENCED DATA
237.348.923 animal data records by Oct. 2012 (total georeferenced records: 327.048.532)

THE CASE FOR DIRECT DATA PUBLICATION
• Access to massive data increasingly
commonalized: GBIF
• Spectrum of possible uses increasing: new
science, new paradigms
• Data-Intensive Science
– Reliance on good data: Opportunity for discovery
– Reliance on bad data: Risk of “undiscoveries”

WHAT, WHERE, WHEN
PBR
Nautilus pompilus
4 specimens
Off Palau Islands
1921
Legit :unknown
Det.: J.A. Salinas
Collection: JDR at MZNA
un

BARRIERS TO DIRECT DATA PUBLICATION
• Data availability
• Data sharing mechanisms
• Data publication incentives
• Data quality

DATA AVAILABILITY INCREASE
GBIF, October 2012

Ariño, 2010. Biodiv. Informat. 7: 15-26

ESTIMATED DATA IN MISSING COLLECTIONS
BCI
GBIF
GSAP –DNHC
survey
unknown
est. CI
Cexp = 8.37K
Nexp = 2.01G

WHAT, WHERE, WHEN
PBR
Saccharomyces cerevisae
TCP1-beta
Cask in wreck of ARGO
2 km E of Akta Képhalos
Stratum 2000 BC
Legit : Homer S.
Det.: LoScanSQ-X
Collection: Museum of Beer History
(MBH)
un
??

THE TRUST PARADOX
• Papers are generally more trusted than raw/downloadable data
– Papers have gone through peer review
• Published data have common sources:
– Experiments,
– Observations,
– Digitizations
• Raw data in published papers can go unreviewed
– Review focuses on soundness, methods, conclusions
– Data assumed to be true & correct
• Direct publication of data, in fact, should facilitate revision
– Enforcing rules
– Filtering
– Pattern detection

FITNESS-FOR-USE
• FFU defines whether data can be used for a
specific purpose
• Useful compromise for publishing data
• FFU not equal to data quality
Quality Fitness-for-use
Intrinsic to data Depends on intended use
Conceptual Pragmatical
Good quality predicting good FFU Good FFU not predict good quality

FFU ASSESSMENT
• In 2006 AHA started analyzing our own DB for
FFU, creating pattern-detection visualizations
– First reported in TDWG-2006 (St. Louis)
• In 2008 we started to analyze raw & processed
GBIF data (2.4G records by 2012) (JOT’s thesis)
– Building on works by Chapman, Yesson, Wieckzorek,
etc., changing scope and perspective
• Started producing reports in 2009, 2010
• Teamed up with GBIF-Sec, 2011
• Created BIDDSAT, 2012

Latitude Longitude
30.37, -87.1765
48.5584, -123.463
42.3487, -123.78
41.7866, -100.061
-73.9071, 42.7028
38.8749, -104.88
44.3964, -75.6668
42.1927, -89.106
32.693, -79.9606
44.2124, -88.42
41.6637, -81.3782
39.6992, -121.778
46.13, -72.7196
38.7231, -77.0674
27.7349, -82.6479
36.0852, -121.616
39.0901, -77.5203
-83.1662, 43.0622
41.2956, -74.5956
45.5146, -73.8131
42.0755, -122.759
41.1047, -81.4944
42.4792, -89.0333
40.6956, -74.8913
...
-
+
Otegui & Ariño, 2009. Proceedings of the TDWG 2009 Annual Conference, Montpellier, FR

DATA INDEXING
Provider
B
Provider
A
Provider
C
Provider
D
GBIF index
?

DATA QUERYING
Provider
B
Provider
A
Provider
C
Provider
D
GBIF index
?

Análisis detallado de GBIF Detailed assessment of GBIF
Bad data Good data

Otegui , Ariño, Gaiji & Chavan, in press

CONTROL AND FFU TOOLS AT INDEXING
Gaiji et al., 22011-2012 –
EMBARGOED DECEMBER 2012

CORRECTING MECHANISMS: EXAMPLES
• GBIF has implemented many georeferencing correction
algorithms, such as e.g. coordinate/country match
• This removes many bogus data points, for example redressing
reversed lat/long when serving data
• Still, original data need to be corrected: GBIF cannot alter
original data (only tag them)
David Remsen, TDWG-2011. In ViBRANT, http://vbrant.eu/content/gbif-integration

2010.04.28
2011.14.09 ErrCode: 10
10: Fields “month” and “day” probably swapped
FILTER MECHANISMS: EXAMPLE
• Original data unchanged
• Index entry corrected
• Error entry generated in issue log

-
+
Otegui et al., 2012
FILTERS CANNOT GET ALL

Otegui , Ariño, Gaiji & Chavan, in press
FILTERS CANNOT SOLVE ALL

Modified from Otegui et al. In press
All GBIF data
Some date
element wrong
Some date element
missing
All date elements
missing

BIDDSAT
• Tool to detect space-time and other patterns
• Applicable to data publishers sharing data
through GBIF
• Uses tailored visualizations
• http://www.unav.es/unzyec/mzna/biddsat/
• Open source: https://github.com/jotegui/BIDDSAT
• Bioinformatics, DOI: 10.1093/bioinformatics/BTS359

0 100
Percentage of completeness
Numberofcollections
015304560
Source: BIDDSAT
DATA COMPLETNESS

0 100
Percentage of completeness
Numberofcollections
015304560
• Wrong
implementation of
exchange standards
(DwC) – solvable
• Data loss – not
solvable
• Limited room for
improvement
Fuente: BIDDSAT
DATA COMPLETNESS

Data Provider LEONIDAS, Resource SHIELD
GBIF 2008/05 Version

Data Provider LEONIDAS, Resource SHIELD
GBIF 2009/09 Version

1/Jan31/Dec
1/Mar
1/Feb
1/Apr
1/May
1/Jun
1/Jul
1/Aug
1/Sep
1/Oct
1/Nov
1/Dec
Fall
Winter
Spring
Summer
1750 Year 2012
-
+
Cronhorogram. Introduced by
Ariño & Otegui, 2008, TDWG

Source: BIDDSAT

- +
Hebdogram. Iintroduced by Ariño & Otegui, 2008. Proceedings of TDWG

Ariño, Otegui & Robles, 2009
Provider 180
All datasets

2008/05
Data Provider
Codename:
BORODIN

Data Provider Codename: BORODIN
2009/092008/05

Treemap by Google Charts API on authors’ data

Fungi
INDEX TAXONOMY
Gaiji et al. in press

• Patchy data publishing… also in papers
• Opportunistic behavior: “Low-hanging fruit”
• Data can (and will) evolve
• The human factor still counts
PATTERNS OF PATTERNS

0
5
10
15
20
25
0 5 10 15
Clase de distancia
Clasedeimprecisión
0 10000 20000 30000 40000
Chordata
Orthoptera
Lepidoptera
Hymenoptera
Diptera
Coleoptera
Thysanoptera
Collembola
Acari
Polychaeta
Oligochaeta
Nematoda
Georreferenciado
Localidad sin
coordenadas
Sin localidad
PAPER WOES: PBR FROM LITERATURE

Publisher: Swedish
Publisher: German
Publisher: French
Publisher: British
Publisher: Norwegian
A MATTER OF CONVENIENCE
Otegui, Robles & Ariño, 2009. eBiosphere, London, UK.
Publisher: Parisien
Publisher: Spanish

CLASSIFICATION
ACCORDING TO:
Ariño, Otegui & Robles, 2009
PROVIDERSP2K
GBIF RECORDS
SAMPLE
EVOLUTIONARY DATA

T H E E N D
THANK YOU
WITH SPECIAL THANKS TO:
VISHWAS CHAVAN, SAMY GAIJI, ANDREA HAHN, TIM ROBERTSON, AND
THE DIGIT SCIENCE SUBCOMITEE AND THE GSAP-NHC AND CNA TASK GROUPS
THE GBIF SECRETARIAT (COPENHAGUEN) AND
THE SPANISH COORDINATION NODE (GBIF.ES)
ESTRELLA ROBLES AND
THE PEOPLE AT THE DEPARTMENT OF
ZOOLOGY AND ECOLOGY (UNZYEC),
THE UNIVERSITY OF NAVARRA
No bytes were seriously harmed while preparing this PPTX.
(And copies exist of those who actullay were anyway).
This file used 328 watt-hours, offset by forfeiting Cantonese roast duck for far too long.
All images, plots and analyses by the authors except where otherwise noted
PPTX © 2012 A.H. Ariño, University of Navarra
www.unav.es/unzyec
BIDDSAT, WWW.UNAV.ES/UNZYEC/MZNA/BIDDSAT/, WWW.NCBI.NLM.NIH.GOV/PUBMED/22730433. SOON IN A PDF NEARYOU.

Highlighting Fitness-For-Use of Published Biodiversity Data

Recommended

Recommended

More Related Content

Similar to Highlighting Fitness-For-Use of Published Biodiversity Data

Similar to Highlighting Fitness-For-Use of Published Biodiversity Data (20)

More from Javier Otegui

More from Javier Otegui (7)

Recently uploaded

Recently uploaded (20)

Highlighting Fitness-For-Use of Published Biodiversity Data