Conferencia 9 andreas_hueni


Published on

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Conferencia 9 andreas_hueni

  1. 1. Spectral Databases Motivations, State of the Art, Visions Workshop Advances in Spectroradiometry Madrid, 3rd – 4th Dec. 2009 1
  2. 2. Outline•  Vision of the Complete Observing System•  Vision of Spectral Databases•  State of the Art of Spectral Databases•  Architecture and essential Functionality•  The Metadata Model•  Input and Data Storage•  Metadata Editing•  Retrieval, Processing and Output•  Data Exchange•  Administration Tools•  GIS Connectivity•  SPECCHIO Online•  Data Policy•  Software availability 2
  3. 3. Vision of the Complete Observing System Understanding the Earth system and the changes induced by anthropogenic and natural causes National Research Council, Earth Science and Applications GEO, Global earth observation system of systemsfrom Space: National Imperatives for the Next Decade and GEOSS: 10-Year implementation plan referenceBeyond. Washington, DC: The National Academies Press, document. 2200 AG Noordwijk, The Netherlands ESA2007. Publications Division 2005. 3
  4. 4. Vision Statement Vision of the Complete Observing System Enable a healthy public, economy, and planet through an integrated, comprehensive, and sustained Earth observation system.Acquisition of Earthsurface information insupport of Earth SystemScience. Image sources: CENR/IWGEO. 2005. Strategic Plan for the U.S. Integrated Earth Observation System, National Science and Technology Council Committee on Environment and Natural Resources, Washington, DC. 4
  5. 5. The Complete Observing System Archiving and Sensors and Platforms Data Management Spacebased Airborne In situ Processing ProcessorsImage sources: Google 5
  6. 6. Implementing the System of Systems: Data Grids Clearing House Data clearing houses present users with a uniform interface to the heterogeneous storage/service components of the data grid. UsersE. J. Christian, "GEOSS Architecture Principles and theGEOSS Clearinghouse," IEEE Systems Journal, vol. 2,pp. 333 - 337, 2008. 6
  7. 7. Data Assimilation & CAL/VAL in a Data Grid Vision Data grids encompass spacebased, airborne and in situ data, offering the possibility of seamless data integration at different scales of observation and thus facilitating data assimilation & calibration/ validation processes as standard operations. Metadata Metadata Metadata Assimilation & CAL/VAL Processes See also CEOS CAL/VAL activities: 7
  8. 8. Spectral Databases: Components of a Complete Observing System Archiving and Data Management Archiving and Data ManagementImage sources: • GEO, Global earth observation system of systems GEOSS: 10-Year implementation plan reference document. 2200 AG Noordwijk, The Netherlands ESAPublications Division 2005.• 8
  9. 9. Definition: Spectral Databases Spectral databases are systems for the organised storage of spectral signatures accompanied by associated metadata. MetadataA. Hueni, et al. "The spectral database SPECCHIO for improved long term usability and data sharing," Computers & Geosciences, vol. 35(3), pp. 557-565, 2009. 9
  10. 10. MetadataMetadata are the key to long-term usage and datasharing. Metadata 10
  11. 11. Metadata SpaceMetadata define a multi-dimensional space; a primary resource isidentified by its metadata parameters and thus has a defined position inmetadata space. 11
  12. 12. Vision of Spectral Databases Spectral information is at the scientistʼs fingertips and comprehensive metadata and automaticallyderived quality indicators facilitate the assessment of data quality and hence their suitability for new applications. Clearing House Scientist 12
  13. 13. Roadmap to Data Sharing and Long-term Usage Registration as data grid component (e.g. GEOSS) Homogenized metadata for eased inclusion as data grid service Automated quality indicator generation Spectral database setup for the organised storage of spectral data and metadataData Acquisitionof high accuracy,well-documented spectral data 13
  14. 14. State of the Art of Spectral Databases Logo Institute Online Public Active # of # of Reference www spectra* install. RSL, University of ✔ ✔ ✔ 3316 15 A. Hueni, et al. "The spectral Zurich, database SPECCHIO for improved Switzerland long term usability and data sharing," Computers & Geosciences, vol. 35(3), pp. 557-565, 2009. DLR, ✔ ✔ ✔ 2008 1 Becvar, M. et al, (2006 - 2008) DLR Oberpfaffenhofen, Spectral Archive. Online at: http:// Germany, accessed on: 25.11.2009. SSI, Australia ✔ ✔ ✘ A few 1 Ferwerda, J. et al, 2006. A free www.hyperspectra dozen online reference library for hyperspectral reflectance signatures. SPIE Newsroom, 1-2, x5275.xml. SSD, Darwin, ✘ ✘ >> 1000 1 Pfitzner, K., et al, 2008, 29.09. - Australia ✔ 03.10.2008. SSDs Spectral Library Database. In: Proceedings 14th u/ssd/research/ Australasian Remote Sensing and protect/ Photogrammetry Conference, rehabilitation.html Darwin, AU. * Available in the online system on 25.11.2009 14
  15. 15. State of the Art of Spectral Databases: Technology Logo Database Interface Local Import formats Export formats installation possible MySQL Java and PHP ✔ ASD binary, GER, CSV ENVI SLB, ENVI SLB ASCII, XML XML ? Web ✘ ASD binary, ASCII ? MySQL PHP ✘ ASD binary ASCII ASD text ENVI SLB GER, ASCII JCAMP ENVI SLB SQL Server ? ✘ ASD binary ? 15
  16. 16. Critical Design Criteria•  Extendable, generic data model•  Defined value sets for categorical variables•  Native file reading routines for automatic metadata generation•  Intuitive interface for metadata editing with minimal user interaction (multiple updates)•  Data retrieval via flexible metadata searches Success Criteria •  Users can accept changes in their traditional workflow •  Users will only enter their data in a well-documented fashion if the user interfaces are friendly and quick •  Spectral databases must become the ʻone stop shopʼ for spectral signatures within and across institutions. 16
  17. 17. SPECCHIO: RSLʼs Spectral DatabaseA. Hueni, et al. "The spectral database SPECCHIO for improved long term usability and data sharing," Computers & Geosciences, vol. 35(3), pp. 557-565, 2009. 17
  18. 18. Main components of the spectral database SPECCHIO Query Builder Metadata Editor Data Loader Space Network ProcessorInstrumentation Administration File Export Data Report 18
  19. 19. SPECCHIO ArchitectureSetup of the SPECCHIO online system 19
  20. 20. SPECCHIO Metadata Sources 20
  21. 21. Data Input SPECCHIO Java Application SPECCHIO Spectral Database Spectral files &user defined structures File system 21
  22. 22. Storage ofspectrodirectional data 22
  23. 23. Metadata Editing Easy selections via
 Spectral Data Browser 23
  24. 24. Metadata Editing -  Multiple updates 24
  25. 25. Selected Metadata of Spectral Data Spatial position per spectrum. Used for GIS, sun angle
 calculations and spatial 
 searches. Automatic calculation of
 sensor and illumination
 geometry (in case of FIGOS and location/time is known respectively). General sampling information. 25
  26. 26. Retrieval: Spectral Data Browser 26
  27. 27. Retrieval: Metadata Space Restrictions 27
  28. 28. Retrieval: Metadata Space Restrictions 28
  29. 29. Processing: The Space Network Components: •  Nodes: spaces (data sources/sinks) and processing modules •  Edges: directed connections between nodesA. Hueni, M. Kneubuehler, J. Nieke, and K. Itten, "PROCESSING EXTENSION FOR THE SPECTRAL DATABASE SPECCHIO," in EARSeL SIG IS,Tel Aviv, Israel, 2009. 29
  30. 30. Processing: Creating Spaces Query Builder Space Network Processor 30
  31. 31. Processing & Output: The Space Network Example 31
  32. 32. Available Modules in the Space Network Processor •  Radiance to Reflectance Transformation•  Reference Panel Correction Factors•  Correct for Reference Panel Non-Idealness•  Delta (A-B as matrix operation)•  Waveband Filter •  Broadband and Narrowband Filters•  Visualisation: Spectral Line Plot, Spectral Scatter Plot , Gonio Sampling Points Plot, Gonio Hemisphere Explorer, Time Line Plot, Time Line Explorer•  File Export 32
  33. 33. Output: Visualisation of spectrodirectional data Interactive Gonio Hemisphere Explorer in SPECCHIOA. Hueni, S. Rey, D. Schläpfer, J. Schopfer, and M. Kneubuehler, "VISUALISATION, PROCESSING AND STORAGE OF SPECTRODIRECTIONAL DATABASED ON THE SPECTRAL DATABASE SPECCHIO," in IGARSS 09, Cape Town, South Africa, 2009. 33
  34. 34. Output: Spectral Data Report 34
  35. 35. Output: CSV and ENVI SLB 35
  36. 36. Data Exchange between SPECCHIO Instances Central Server @ RSL SPECCHIO Online Server SPECCHIO SPECCHIO SPECCHIO SPECCHIO Edu Prod Prod Test Workstation @ RSL Workstation @3rd party SPECCHIO SPECCHIOA. Hueni, T. Malthus, M. Kneubuehler, and M. Schaepman, "Data Exchange between distributed SpectralDatabases," Computers & Geosciences, submitted. 36
  37. 37. Data Exchange between SPECCHIO Instances Automated export/import of spectral sampling campaigns including the whole metadata context using XML style files. 37
  38. 38. Administration Tools: Instrumentation 38
  39. 39. Administration Tools: Instrumentation 39
  40. 40. GIS Connectivity -  ODBC connection-  Access to all tables-  Spatial mapping Mapping of sampling locations using ArcMap (ESRI) 40
  41. 41. SPECCHIO Online:  Personal accounts via  Upload your own spectra, browse and download all spectra in the online database 41
  42. 42. SPECCHIO OnlineCurrently 86 users world wide* * Stats from 29.11.2009 42
  43. 43. Data Policy -  Data on the online system is freely accessible (read-only for all, full access for owner)-  Import/Export function for SPECCHIO systems allows to easily include our own data
 in the online system (exact copy including full metadata context) 43
  44. 44. Software availability -  Online user account creation-  Java software download -  Full installation package for local installation available: email 44
  45. 45. Get a Leaflet …. 45
  46. 46. Questions?Get your personal SPECCHIO account for the online SPECCHIO system by browsing to: 46
  47. 47. “Data are unstructured facts and figures. When they have been organised or processed, they become information” –
 Organise your spectral data. 47