Successfully reported this slideshow.
Development of Data Integration &
Analysis System in Japan
Activities toward interoperability
Seishi Ninomiya
Institute of...
Productivity
Quality/Safety
Farmers’
benefits
Food
loss/waste
Minimum
emission
Limited
resources/water
/land/energy
Ecosys...
What we need for data-centric science in agriculture
• Utilization of legacy data
o Yield data, variety data, quality data...
DIAS
Data Integration & Analysis System
© GEO Secretariat
• The Group on Earth Observations is coordinating efforts to build a
Global Earth Observation System of ...
S&D strategy in Japan
RECCA2
S&T Basic plan -5
will start since 2016
Data Integration and Analysis System (DIAS)
• DIAS was launched in 2006 as a Japanese contribution to
GEOSS
• one of five ...
20PB by 2014
Data Storage Core System
User Communication & Management
Search &
Discovery
System
Science
Societal Benefit
Data
Mata Data...
10
DB
DB
0
50
100
150
200
250
300
350
19
92
19
94
19
96
19
98
20
00
20
02
20
04
20
06
20
08
20
10
20
12
20
14
20
16
20
18
...
Upload
Meta Data
Meta Data Meta Data
Data Integration and Analysis System (DA-09-02a)
Quality
Control
Data Provider (Obser...
Ontology development in DIAS
Challenge for Data Management & Fusion
Syntax Interoperability
Proposal of Standard Schema and Interface.
It is not enough...
Semantics Activity
Existing Glossaries
1 WMO Glossary
2 CEOS Missions, Instruments and
Measurements(MIM) Database
3 CEOS S...
Vocabulary Registry to Find Similar Technical Term
Input Keywords, “precipitation”
Similarity score
with the input keyword...
INDEX DB RESULTS
- Scholarly Journals
- Social Data
- Lat / Lng
- Scientific Values
- Keywords
- Date
Scientific Data
[ Re...
MetBroker
Weather data integration
• Heterogeneity among data sources is a big issue in the Internet.
(data structure, access methods, etc.)
• Data brokers p...
MetBroker Since 2000 -Spatial integration of weather data
• MetBroker provides applications consistent access to heterogen...
Crop model + MetBroker = Potential Rice Yield
28
IOT/Sensor data
Giving Interoperability to heterogeneous sensing data
via OGC Standard Web Service, SOS
OGCAPI
GetCapabilities
List of aut...
Sensor Infra. And Multi-Layered Web Service
Sensor Infra
Water
Level/Temp
via NICT
NARO 1km Mesh
Agri Weather (Past
and 2w...
Farm data
33
Conceptual sketch of “CLOP”
APAN 39th in FukuokaAPAN 39th in Fukuoka T. Yoshida, NARO
USD 10 million for 5 years by MAF...
• Current range FIX-pms covers is limited to farm
work and production process management.
34
Cover range of ‘FIX-pms’
APAN...
• Defined based on agroXML.
35
Outline structure of ‘FIX-pms’
APAN 39th in Fukuoka T. Yoshida, NARO
Structure of API mashup : 4 Layers
36
Term, Code Layer
Data Content
Layer
Data Format
(Container)
Layer
API
Layer
agroXML,...
International collaboration
W3C Agriculture CG
W3C Team | Posted on: October 15, 2014
The Agriculture Community Group has been launched:
The initial m...
Future Internet PPP – 300 M€ EU
funding
3
9
40
二宮正士 snino@isas.a.u-tokyo.ac.jp
Thank you very much
Development of Data Integration & Analysis System in Japan
Development of Data Integration & Analysis System in Japan
Development of Data Integration & Analysis System in Japan
Development of Data Integration & Analysis System in Japan
Development of Data Integration & Analysis System in Japan
Development of Data Integration & Analysis System in Japan
Development of Data Integration & Analysis System in Japan
Development of Data Integration & Analysis System in Japan
Upcoming SlideShare
Loading in …5
×

Development of Data Integration & Analysis System in Japan

900 views

Published on

Seishi Ninomiya, Institute of Sustainable Agro-ecosystem Services, The University of Tokyo, at RDA 5th Plenary Meeting, IG Agriculture Data Interoperability Session in San Diego (CA, US) on the 9th of March 2015

Published in: Science
  • Be the first to comment

  • Be the first to like this

Development of Data Integration & Analysis System in Japan

  1. 1. Development of Data Integration & Analysis System in Japan Activities toward interoperability Seishi Ninomiya Institute of Sustainable Agro-ecosystem Services, The University of Tokyo
  2. 2. Productivity Quality/Safety Farmers’ benefits Food loss/waste Minimum emission Limited resources/water /land/energy Ecosystem/ Biodiversity Food distribution Diet transition Climatic change/disaster Toward Sustainable and sufficient food productionToward Sustainable and sufficient food production Security Food Security
  3. 3. What we need for data-centric science in agriculture • Utilization of legacy data o Yield data, variety data, quality data, soil data, market data, ………… o Need to rescue such data • Sensor innovation IOT o To efficiently monitor the facts in fields, market, demands, logistics, processing,…. o To collect knowledge of farmers, tacit knowledge • Data integration and efficient usage/ Interoperability o Common platform for seamless data exchange with standard o Agricultural cloud and database • Tools for analysis/analytics and for supporting decisions o Statistics, data-mining , knowledge extraction, risk managements o Big data-based optimization o Enrichment of commonly usable APIs • Communication innovation o Efficient Knowledge transfer to farmers • Service science Big data + Advancement of Agricultural Science
  4. 4. DIAS Data Integration & Analysis System
  5. 5. © GEO Secretariat • The Group on Earth Observations is coordinating efforts to build a Global Earth Observation System of Systems (GEOSS). • GEO was launched in response to calls for action by the 2002 World Summit on Sustainable Development and by the G8 leading industrialized countries. GEO/GEOSS
  6. 6. S&D strategy in Japan RECCA2 S&T Basic plan -5 will start since 2016
  7. 7. Data Integration and Analysis System (DIAS) • DIAS was launched in 2006 as a Japanese contribution to GEOSS • one of five National Key Technologies defined by the 3rd Basic Program for Science and Technology of Japan. o Total of USD 60 million for 10 years by MEXT from 2006 • The missions are: o to coordinate the cutting-edge information science and technology and the various research fields addressing the earth environment; o to construct data infrastructure that can integrate earth observation data, numerical model outputs, and socio- economic data effectively; o to create knowledge enabling us to solve the sustainable world o to generate socio-economic benefits
  8. 8. 20PB by 2014
  9. 9. Data Storage Core System User Communication & Management Search & Discovery System Science Societal Benefit Data Mata Data Document Data Cleansing System (QC) Processed Data Data Loading System Data Integration & Analysis System Data Archive Data Download System Analysis Systems By User Communication Tool Management Tool Data Provider Meta data Registration Document Registration Meta data Standard Inter- operability Portal User Authentication User Authorized User
  10. 10. 10 DB DB 0 50 100 150 200 250 300 350 19 92 19 94 19 96 19 98 20 00 20 02 20 04 20 06 20 08 20 10 20 12 20 14 20 16 20 18 20 20 A1:高度成長社会 A2:多元化型 B1:持続発展型 B2:地域共存型 DB DB DB DB DB 0 50 100 150 200 250 300 350 199 2 199 4 199 6 199 8 200 0 200 2 200 4 200 6 200 8 201 0 201 2 201 4 201 6 201 8 202 0 A1:高度成長社会 A2:多元化型 B1:持続発展型 B2:地域共存型 DB DB 0 50 100 150 200 250 300 350 199 2 199 4 199 6 199 8 200 0 200 2 200 4 200 6 200 8 201 0 201 2 201 4 201 6 201 8 202 0 A1:高度成長社会 A2:多元化型 B1:持続発展型 B2:地域共存型 DB DB D B Meteorology Ecosystem Agriculture Hydrology Land Use Climatology Health 农业 เกษตรกรร ม 농업 Where is data? How to access? Technical Terms among Different Disciplines Quality? Reliability?
  11. 11. Upload Meta Data Meta Data Meta Data Data Integration and Analysis System (DA-09-02a) Quality Control Data Provider (Observer) User Meta Data Registration •Search with Metadata •Data Download •Document Generation from Meta Data •Data Visualization ・・ Observation Data Meta Data Data Upload +(part of )Meta Data Observation Data Meta Data Data Quality Control Process Meta Data Post-QC Observation Data Input Meta Data Data DownloadSearch IF Document Generator Visualization System DataArchivingDataIntegration Web-based Data Archiving & Integration System Basic Information Observation Point Inf., Contact Person Inf.,……
  12. 12. Ontology development in DIAS
  13. 13. Challenge for Data Management & Fusion Syntax Interoperability Proposal of Standard Schema and Interface. It is not enough for diversified Geo-spatial Information. e.g. legacy data. New Challenge of Data Management and Fusion Registry Visualizing diversified data, Helping data convergency . One stop service for data utilization. Very Large& Heterogeneous Data Management and Fusion Data quality checking, Emergency response for disaster, Automatic processing and fusion, Change detection, etc. Integrating Observation Data and Model Simulation Semantic Interoperability (Ontology ) Semantic Interoperability for geo-spatial data by using data definitions, terminologies, relations, landnames, etc. By Msahiko Nagai, U..Tokyo & AIT, 2015
  14. 14. Semantics Activity Existing Glossaries 1 WMO Glossary 2 CEOS Missions, Instruments and Measurements(MIM) Database 3 CEOS Systems Engineering Office(SEO) 4 GEMET 5 INSPIRE Feature Concept Dictionary 6 SWEET 7 CUAHSI 8 CF Standard Names 9 GCMD 10 Eurovoc Thesaurus 11 International Glossary of Hydrology/UNESCO 12 Marine Metadata Interoperability Close Match We are forcing on the one hand the implementation of semantic interoperability arrangement with ontological information. DIAS Vocabulary Registry SKOS 146 observation parameters with SBA Define and associate with EO Vocabulary and Existing Glossaries By Msahiko Nagai, U..Tokyo & AIT, 2015
  15. 15. Vocabulary Registry to Find Similar Technical Term Input Keywords, “precipitation” Similarity score with the input keywords By Msahiko Nagai, U..Tokyo & AIT, 2015
  16. 16. INDEX DB RESULTS - Scholarly Journals - Social Data - Lat / Lng - Scientific Values - Keywords - Date Scientific Data [ Remotely Sensed Data, Meta - Data ] - Description - Images - Date - Geo - Tags - Videos ] Social Data [Ushahidi, Google News, Social Networks] INDEXED DATABASE User Interface QUERY PARAMETERS - Location - COP / SBA - Date Scholarly Journal Data [ Sci-Verse HUB, Mendeley api ] • Keywords • Abstract / Full Text • Author(s) • Published Date • Scientific Models Used • Input Data / Output Data • Geo - Tags ONTOLOGY DEVELOPMENT / CONCEPT TAGGING EXTRACTION / INDEXING ONTOLOGY UPDATE SEARCH REQUEST SEARCH RESULTS ONTOLOGY Application Service EXTRACTOR SEARCH ENGINE Journals Knowledge based Ontology Development / Update JSP Service 24 By Msahiko Nagai, U..Tokyo & AIT, 2015
  17. 17. MetBroker Weather data integration
  18. 18. • Heterogeneity among data sources is a big issue in the Internet. (data structure, access methods, etc.) • Data brokers provide consistent access to those heterogeneous data sources MetBroker for various weather databases Meta Data Heterogeneous and Autonomous DBs Rice Growth Prediction Farm Management MetBroker Pesticide Prediction Heterogeneity is absorbed by brokers (mediators) B-DB C-DB A-DB
  19. 19. MetBroker Since 2000 -Spatial integration of weather data • MetBroker provides applications consistent access to heterogeneous weather databases and covers 30,000 weather stations of 25 DBs • API MetXML
  20. 20. Crop model + MetBroker = Potential Rice Yield 28
  21. 21. IOT/Sensor data
  22. 22. Giving Interoperability to heterogeneous sensing data via OGC Standard Web Service, SOS OGCAPI GetCapabilities List of authorized SOS stations with its sensors GetObservation Sensor Data with Timestamp 1 2 Simulation System User Interface for Famers Kiyoshi HONDA, R. Chinnachodteeranun, A. Witayangkurn, APAN Meeting 4 Mar 2015
  23. 23. Sensor Infra. And Multi-Layered Web Service Sensor Infra Water Level/Temp via NICT NARO 1km Mesh Agri Weather (Past and 2w Forecast) NIAES Point Agri. Weather Other Sensor Interpolation, Statistics, Visualization Weather Generator Rice Crop Simulation Standard APIStandard API SOS Open API Open API Application Open APIOpen API Application Visualization Analysis Appli. Crop Simulation Developed by single developer Obtain necessary functionalities via Web Service Anyone can access to high-level functionalities Sensor Virtualization, New sensor, sensor transfer will be reflected application automatically Kiyoshi HONDA, R. Chinnachodteeranun, A. Witayangkurn, APAN Meeting 4 Mar 2015
  24. 24. Farm data
  25. 25. 33 Conceptual sketch of “CLOP” APAN 39th in FukuokaAPAN 39th in Fukuoka T. Yoshida, NARO USD 10 million for 5 years by MAFF from 2014
  26. 26. • Current range FIX-pms covers is limited to farm work and production process management. 34 Cover range of ‘FIX-pms’ APAN 39th in Fukuoka T. Yoshida, NARO
  27. 27. • Defined based on agroXML. 35 Outline structure of ‘FIX-pms’ APAN 39th in Fukuoka T. Yoshida, NARO
  28. 28. Structure of API mashup : 4 Layers 36 Term, Code Layer Data Content Layer Data Format (Container) Layer API Layer agroXML, Sensor ML, GML/KML, GPX, … FarmXML(FIX-pms), BIX-pp, GPXX, … Data structure? Data meaning? Data relation? V V V RDF, UML, … SOS, WMTS, WMS, WFS, … MetXML, PDS, … Content list in certain region of interest among certain stakeholders, … Terminology, ontology, … Code system definition, … (Language / Localization) APAN 39th in Fukuoka T. Yoshida, NARO
  29. 29. International collaboration
  30. 30. W3C Agriculture CG W3C Team | Posted on: October 15, 2014 The Agriculture Community Group has been launched: The initial mission of the Agriculture Community Group is to gather and categories existing user scenarios, which use Web APIs and services, in the agriculture industry from around the world, and to serve as a portal which helps both web developers and agricultural stakeholders create smarter devices, Web applications & services, and to provide bird’s eye view map of this domain which enables W3C and other SDOs to find overlaps and gaps of user scenarios and the Open Web Platform. We’ll try to collect facts and knowledge from around the world through crowd- sourcing, while, at the same time, build a scaffold for it by quickly gathering key topics from Japanese agricultural stakeholders. Smart Platform Forum supports this early stages by connecting relevant stakeholders in Japan and organizing face-to-face meetings if needed to proceed faster. https://www.w3.org/community/agri/
  31. 31. Future Internet PPP – 300 M€ EU funding 3 9
  32. 32. 40 二宮正士 snino@isas.a.u-tokyo.ac.jp Thank you very much

×