Dynamic integrations of crop data and corresponding meteorological data based on a standardized data exchange framework
•Download as PPT, PDF•
1 like•247 views
The document summarizes a system for integrating crop data and meteorological data using a standardized data exchange framework. The system uses a metadata database and broker service called MetBroker to provide consistent access to heterogeneous weather databases. Crop data from different sources can be uploaded and integrated into a central database. The system then allows users to query the integrated crop and weather data and analyze relationships to support applications like crop modeling.
![What is Grid? ,[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
What's hot
What's hot (15)
Viewers also liked
Viewers also liked (20)
Similar to Dynamic integrations of crop data and corresponding meteorological data based on a standardized data exchange framework
Similar to Dynamic integrations of crop data and corresponding meteorological data based on a standardized data exchange framework (20)
More from AIMS (Agricultural Information Management Standards)
More from AIMS (Agricultural Information Management Standards) (20)
Recently uploaded
Recently uploaded (20)
Dynamic integrations of crop data and corresponding meteorological data based on a standardized data exchange framework
- 1. Dynamic Integrations of Crop Data and Corresponding Meteorological Data based on A Standardized Data Exchange Framework Seishi Ninomiya, Atsushi Yamakawa, Xinwen Yu National Agricultural Research Center, National Agriculture and Food Research Organization, Japan
- 3. Users need to obtain one by one, knowing how to access each
- 4. e.g. Data Grid provides you A virtually integrated huge database We do not need to know where they are, how to use,…
- 5. Concept of Grid System Case Base Weather Data 2 Farm Management Meta Database The Internet Agterm Dictionary User who needs Decision Field Data Monitoring ． Growth Model2 Data Broker Weather Data 1 ． ． Growth Model1
- 10. Objective 2: Integration of crop data with corresponding meteorological data Crop DB Data Extraction Location & Date Crop Data Corresponding weather data XML/Crop data &weather data Models/Analysis SOAP/XML MetBroker Meteorological DB Integration Service
- 11. Basic structure of application Google Map Client Browser Controller Model Services Web application View Crop DB Crop Data Service MetBroker AMeDAS AMeDAS AMeDAS
- 13. e.g. Crop model clients of MetBroker
- 14. Over 22,000 stations of 25 databases
- 15. Coverage of MetBroker Needed Daily 1995 7 Taiwan Ecological Research Network Taiwan Free Hourly 1993 11 Seoul National University Plant Disease and Epidemiology Lab Korea Free Daily 1997 13 South African Sugar Association network South Africa Needed Hourly 1853 6547 National Climate Database NZ Free Hourly 1996 39 HortPlus Ltd NZ Needed Daily 1919 2 Horticulture Research International UK Free Hourly 1987 33 Planteforsk Crop Research Institute Norwayu Needed 15 min 1987 60 Washington State University Public Agricultural Weather System USA Free 15 min 1996 18 Florida Automated Weather Network USA Free Daily 1997 46 Georgia Automated Environmental Monitoring Network USA Free Daily 1964 60 Long Term Ecological Research Network (ClimDB) USA Free Daily 1996 152 Oregon Integrated Plant Protection Center (NorthWest) USA Free Daily 1994 12000< NOAA/WMO Archive US/WMO Needed 10 min 2002 20< FieldServer Project2 Japan Needed 10 min 2002 3 FieldServer Project1 Japan Free Hourly 1986 3 National Hokkaido Agriculture Research Center Japan Free Hourly 1986 3 Tottori Prefec. Hort. Exp. Station Japan Free Hourly 1986 3 Chiba Prefec. Agric. Exp. Station Japan Free Hourly 2000 8 Hokkaido Memoro/MAMEDAS Japan Free Hourly 1998 14 Kanagawa Prefec. Agriculture & Forestry Met. DB Japan Free Hourly 2001 137 Wakayama Prefec. Rainfall DB Japan Free Hourly 1989 150 National Meteorological Observatory Japan Free Hourly 1976 1479 AMeDAS/MAFFIN Japan ID/Passwd Frq. From ＃ Stations Weather Database Country
- 17. 1
- 18. Database Broker Service Data Brokage DB A Database Driver DB B DB C DB D Meta Database Where, How to use Data contents Data Request Search Standardized Data Data Summarization Ex) Daily mean from hourly data Data acquisition Data request translated to DB C Data Standardization Data Secondary Processing Client
- 21. A part of Standard Vocabulary OWL <owl:Class rdf:ID="DailyMaxAirTemperature"> <rdfs:subClassOf rdf:resource="#MaxAirTemperature"/> <rdfs:subClassOf> <owl:Restriction> <owl:allValuesFrom> <owl:Class rdf:about="#DailyMaximum"/> </owl:allValuesFrom> <owl:onProperty> <owl:ObjectProperty rdf:about="#summaryKind"/> </owl:onProperty> </owl:Restriction> </rdfs:subClassOf> </owl:Class> <owl:Class rdf:about="#DailyMaximum"> <rdfs:subClassOf rdf:resource="#Maximum"/> <rdfs:subClassOf> <owl:Restriction> <owl:allValuesFrom rdf:resource="#Daily"/> <owl:onProperty> <owl:ObjectProperty rdf:about="#duration"/> </owl:onProperty> </owl:Restriction> </rdfs:subClassOf> </owl:Class> Sample file: http:// www.agmodel.org/MetBroker.owl “” DailyMaxAirTemperature” is a subclass of “MaxAirTemperature” “” DailyMaxAirTemperature” is translated as daily maximum data
- 22. Sample of Item Definition OWL of a DB <met:DailyMaxAirTemperature rdf:ID="ame_day.temp_max"> <met:summaryKind rdf:resource= "http://www.agmodel.org/MetBroker.owl#DailyMaximumOfSampleEvery10Minutes"/> </met:DailyMaxAirTemperature> <met:HourlySampleAirTemperature rdf:ID="ame_time.temperature"> <met:summaryKind rdf:resource= "http://www.agmodel.org/MetBroker.owl#SampleOnTheHour"/> </met:HourlySampleAirTemperature> A sample file is available on http:// www.agmodel.org/Aclima.owl Local item name “ ame_day.temp_max” is translated as daily maximum data based on every 10 minute data
- 28. System components Java Runtime Environment 1.5.04 PostgreSQL7.4 JRE supported OS JBoss-4.0.3 EJB3.0 (DBMS abstraction) Struts1.2 (Web Interface) ・ IE, Firefox, etc. ・ Excel2002, newer
- 30. Crop d ata upload and integration Crop Data Service EJB3 Source XML Crop Data History Data transforming Data validating XSLT style sheet Data Schema a b c Crop Database
- 32. Data upload
- 33. Upload history
- 34. Data Query Web Application Crop Data Service EJB3 Crop db Specifying query conditions then executing data query. Browsing and/or download queried crop data
- 35. The mechanism of data integration Location Table Longitude, latitude CropDataService Data query Location Time duration Retrieved crop data MetBroker Weather Items Data query Weather data Data Integration Other properties … Weather stations
- 36. Integrating crop data and weather data AMeDAS Web Application Crop Data Location Table Crop Data Service EJB3 MetBroker AMeDAS AMeDAS
- 41. Thank you very much http://www.agmodel.org/ http://www.agmodel.org/vocabulary/200602/MetBroker.owl
- 43. Seamless Integration of Field Server with Legacy Databases through MetBroker 気象 DB 気象 DB 気象 DB 気象 DB FieldServerDB アプリケーション アプリケーション アプリケーション MetBroker 気象 DB 気象 DB 気象 DB 気象 DB FieldServerDB アプリケーション アプリケーション アプリケーション MetBroker 気象 DB 気象 DB 気象 DB 気象 DB FieldServerDB アプリケーション アプリケーション アプリケーション MetBroker W DB 気象 DB 気象 DB 気象 DB アプリケーション アプリケーション アプリケーション MetBroker MetBroker Weather DB Weather DB FS Weather DB Client APP Client APP Client APP Weather DB Station Conf. XML Weather Data XML FS Data Archive
- 45. Brokers Provided as Web Services ChizuBroker MetBroker DEMBroker WebService-SOAP/XML Client Client Client WebService-SOAP/XML WebService-SOAP/XML
- 48. Potential for Data Sharing Between DSS Data Needed Decisions (Clients) Topography Soils Crop details Weather Data O O O Irrigation or not O O O O Spray for disease O O O O Land use O O O To dam? O O O Variety selection
- 49. Concept of Agri-Grid System Case Base Weather Data 2 Farm Management Meta Database The Internet Agterm Dictionary User who needs Decision Field Data Monitoring ． Growth Model2 Data Broker Weather Data 1 ． ． Growth Model1
Editor's Notes
- First, I’d like to talk about dead… As we all know, These data are usually stored using… So it’s important to think about
- And there is another issue, that is Different…. Because these crop data are separated, isolated, they are hard to be … We have to face heterogeneity problem, for example, So we must consider how to
- If we can merge and share those crop data, then end users… After data integration,
- So, We want to develop a system that can make multi-location data integrated and sharable over the Internet. a system that can integrate crop data with weather data
- This is the basic structure of our System. The system consists of 3 parts, client, Web application and services. The System adopted a combination of MVC pattern and SOA pattern. MVC means model, View and Controller. Web application used MVC pattern so that the business logic and data manipulation are separated. SOA means service oriented architecture, the idea is to make use of available services instead of developing these function repeatedly. Service part consists of MetBroker, which provides consistent access to Meteorology databases, Crop Data Service, which provides access to crop data, and Google Map, which provides location information. Each service may have its own enterprise data source.
- This figure shows the components used in our system implementation. As we had just mentioned in last slide, the client is just a browser, can be IE, FireFox, etc., and Spreadsheet software can be Excel 2002 or newer. Struts was used to implement the MVC pattern, making the implementation quite easier. EJB 3 was adopted to implement crop data manipulation, such as data querying, storage, updating, etc. We use PostgreSQL as our database server. JBoss is the application server, Web application and EJB server are deployed to JBoss. We use JBoss 4.0.3 in our system. Of course, JRE and OS are indispensable.
- There are 3 steps for processing uploaded data file. First, when client user uploads a data file, the web application will transform source data file to data XML…, meanwhile, another thread will send the source XML document to Crop data Service, and crop data service will then store source XML document to SourceXML table. 2, web application validates the data XML produced on first step using data schema. We defined the data schema very carefully. 3. Data service binds validated data XML to data object, and then maps data objects to data records to CropData table,….
- This is the main menu of our web application. It has a very simple menu structure, a simple and clean user interface There are 3 functional menu items, I will introduce them in detail one by one. Search actually implements data query functionality, integration of crop data and weather data happens here. Upload allows user to upload crop data file. Data merging happens here. Uploading history allows user manage uploaded data
- First I will talk about data upload. Click on upload menu item will enter the upload page. There are 3 file formats for crop data file. We also provide sample files for each file format. User can check the sample file to determine which format should be selected.
- Let’s have a look of upload history function. Click on uploading history will enter upload history page. This page lists uploaded files, user can display the source XML of a selected file. Or delete an uploaded file, this will also delete the corresponding data sets stored in CropData table.
- Now I will talk about Data Query. Client user can specify query conditions using Web application, web application will send the query conditions to CropData service, then crop data service will retrieve data from crop database based on the query conditions. User can browse and download the returned crop data.
- Now let’s talk about Integrating…. In last slide, we described data query. The queried crop data can be integrated with weather data. Crop data service will obtain the latitude and longitude value of the experimental location from Location table. After some simple configurations for weather data retrieving, web application will use MetBroker to obtain requested weather data. Weather stations are determined based on the location information.. Returned weather data then will be integrated with crop data into a spreadsheet file.
- By default, system will select the nearest weather station to the experiment location for weather data retrieval. But in some cases, user may want to choose a specific weather station, we also provide selection function for weather station.
- Some time, experiment location information is unavailable in the location table, but the user who is working at that place may be able to provide this information.So we also provide a small tool to help user register a new location. If user knows the location information, he can directly input them, or he can find the location on google map, and click on the location, the value will automatically fill in. Once a new location is registered, crop data from this location can be integrated with weather data of this area.
- Because farm DSS in different decision areas tend to use the same underlying data, it is important that DSS be integrated so that they can share data. This avoids having to enter the same data into a number of different software packages.
- With web application for data sharing and integration, Once you upload… Crop data … All these operation can be performed very easily, No skill