Presentation delivered at the Agricultural Data Interoperability Interest Group -- Research Data Alliance (RDA) 4th Plenary Meeting -- Amsterdam, September 2014

1. Opening and Integration of CASDD and
Germplasm Data to AGRIS
Prof. Xuefu Zhang & Dr. Guojian Xian
Agricultural Information Institute of CAAS
Research Data Alliance Fourth Plenary Meeting, 22-24 September, 2014, Amsterdam

2. Contents
 Open CASDD as Restful APIs
 Open Germplasm as Restful APIs
 Integration and Extension to AGRIS
 Fruitful Results
2

3. Main Materials
• Chinese Agricultural Sci-tech Documents Database (CASDD)
– 440,113 records
• CGRIS Germplasm Data
• AGROVOC
– agrovoc_2013-12-17_core.rdf
• Chinese Agricultural Thesaurus(CAT)
• KOS Mapping Results:
– AGROVOC_CAT.nt
• AGRIS 2.0
– (Latest version: 20140427)
3

4. About CASDD
• Chinese Agricultural Sci-tech Documents Database
(CASDD), as agricultural bibliographic/abstracts database
in China developed by CAAS, has the largest number of
records and the longest time span of documents.
• Covering over 1000 kinds of agricultural academic journals
and other materials, over 6 million records, in the fields of
agronomy, horticulture, plant protection, soil sciences,
animal husbandry, veterinary, agricultural engineering,
agricultural products processing, agricultural economic,etc.
• It is the most comprehensive, reliable and accessible
information resources of agricultural science and
technology information from research institutions, education
and related departments.
4

5. Refining and Analyzing CASDD
5
Tagging CAT and AGROVOC concepts to CASDD
CASDD
CAT AGROVOC
RDF Core
Mapping
of CAT and
AGROVOC
Solr 4.7
Write&Read
Tagging(URI,Preflabel)
Indexing
CASDD
Index
Virtuoso
Triple
Store
Tagging(URI,Preflabel)
Sparql
query
MMseg4J/
IKAnalyzer
Java
Application
SQE Plugin

6. English Coverage Analysis of CASDD Records
6
Total Records: 440,113
Fields Records Percentage
English Title 289,314 65.74%
English Keywords 286,032 64.99%
English Abstract 286,921 65.19%

7. 7
The CAT Concepts Coverage in CASDD

8. 8
The AGROVOC Concepts Coverage in CASDD
TermCount TermFreq. Record Number Match Ratio
TermFreq>=3
TermCount>=1
400,009 90.89%
TermFreq>=3
TermCount>=2
320,472 72.82%
TermFreq>=3
TermCount>=3
227,481 51.69%
TermFreq>=3
TermCount>=5
83,992 19.08%
TermFreq>=5
TermCount>=3
51,726 11.75%

9. CASDD Restful API (Architecture)
Accessing & Linking
CASDD Restful Web Service (API) Endpoint
CASDD
Database
Tomcat
(Jersey API)
Reading Only
Container
Solr 4.7
(SQE Plugin)
Third Part
Application
Container
AGRIS
agINFRA
Index
CAT + AGROVOC + Mapping

10. CASDD Restful API(Features)
• Aims to provide a light-weight solution to expose the
records of CASDD to the third party applications.
• Providing several ways to access the records, such as
query with keywords, ARN, PublicationDate, AGROVOC
Concept URIs, Chinese Agricultural Thesaurus (CAT)
URIs.
• The results also supporting pagination and sorting.
• The output formats include RDF/XML following the
AGRIS AP standard and plain JSON.
• Authentication and Detail Logging for evaluations
10

11. CASDD Restful API(Samples)
11
BrowsGinetg rreeccoorrddss wwiitthh ApGagRiOnaVtOioCn URI

12. Contents
 Open CASDD as Restful APIs
 Open Germplasm as Restful APIs
 Integration and Extension to AGRIS
 Fruitful Results
12

13. Germplasm Data of CGRIS
• CGRIS germplasm database is a central
repository for all type of plant genetic
resources information in China. At present,
there are over 4000 MB data on 200 kinds of
crops, 410,000 accessions of germplasm
stored in CGRIS.
13

14. The Germplasm Restful API (Architecture)
Redirect to Detail
CGRIS
Germplasm
Database
Tomcat
(Jersey API)
CGRIS
Website
CGRIS Germplasm Restful API
AGROVOC
CAT
Preflabel2URI
Mapping
Reading Only
Accessing & Linking
Container
Third Part
Application

15. The Germplasm Restful API (Features)
• Aims to provide a light-weight solution to expose
the records of CGRIS Germplasm to the third party
applications.
• Providing several ways to access the records, such
as query with scientific name, vernacular name,
catalogNumber, AGROVOC Concept URI, Chinese
Agricultural Thesaurus (CAT) URI.
• The output formats include RDF/XML following the
darwincore-germplasm schema and plain JSON.
• Authentication and Detail Logging for evaluations

16. The Germplasm Restful API (Samples)
Get records with vernacular name
Get reGcoetr drse cwoirtdhs s wciietnht AifGicR nOaVmOeC URI

17. Contents
 Open CASDD as Restful APIs
 Open Germplasm as Restful APIs
 Integration and Extension to AGRIS
 Fruitful Results
17

18. The Extended AGRIS in Chinese
18
The Extended AGRIS in Chinese
QUERY STATISTICS (CASDD) SEARCH RESULT BROWSING
Restful API
SINGLE RECORD MASHUPS
( Germplasm)
AGRIS SERVICES LAYER
Read
TOOLS LAYER
CASDD
Germplasm
DATA LAYER
JAVA APPLICATION
Custom Modules
CASDD New
Page
CASDD AGRIS
CAT
AGROVOC
RDF Core
Mapping of
CAT and
AGROVOC
Chinese
Query
Solr 4.7
SQE Plugin
CASDD Box
Germplasm Other Resources

19. Enhanced Search in Chinese
• Semantic Query Extension
– Solr Query Expander (SQE)2.0
• Integrating and Linking CASDD API
• Integrating and Linking Germplasm API
• Other Improvements:
– User Query Automatic Suggestion
– Update AGRIS AP XML files Indexer to Solr 4.7
– Integrating Bing Cloud Dictionary
19

20. Improved and Updated SQE 2.0
• Totally be compliance with Solr 4.5.
• Work with SKOS files with suffix .rdf (RDF/XML), .n3
(N3),.ttl (TURTLE) and .zip (ZIP)
• Supports load more than one SKOS files at one time
• Supports customized relationship types expansion,
such as PREF, ALT, HIDDEN, BROADER, NARROWER,
BROADERTRANSITIVE, RELATED.
• Excellent performance with the improved version of
IKAnalyzer2012FF (supports English phrase analysis
and tagging based on English dictionary)
20

21. Semantic Expansion Search with SQE2.0
21

22. Integrating and Linking CASDD
• AGRIS Search Results(CASDD Box)
– The box displays the search results of CASDD (first
five records)
– Records include title, author, keywords,
submission date, and abstract.
– get more related records
– get more (detail information)
22

23. Integrating and Linking CASDD
• Detail information(Single Record information)
– Title(ZH/EN), Keywords(ZH/EN), Authors,
Submission Date, Abstract(ZH/EN), CAT keywords,
AGROVOC keywords, Journal, ISSN
• More Related Records
– Display more related records
– Browing records with pagination
23

24. Linking CGRIS Germplasm Resources
• Germplasm Mashup
– get more…(detail information)
First five CGRIS Germplasm records information
• Navigating to CGRIS Website
– CGRIS website
24

25. Contents
 Open CASDD as Restful APIs
 Open Germplasm as Restful APIs
 Integration and Extension to AGRIS
 Fruitful Results
25

26. 26
Linking CASDD Records with Box
http://agris.fao.org/agris-search/searchIndex.do?query=barley&x=-430&y=-58

27. Detail Info of a CASDD Record
27

28. More Related Records From CASDD
28

29. CGRIS Germplasm Mashup
29

30. Thanks for Your Listening!
30