Your SlideShare is downloading. ×
Foss4G 2009 Scenz Grid
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Foss4G 2009 Scenz Grid


Published on

Landcare Research SCENZ-Grid architecture presentation as presented at FOSS4G

Landcare Research SCENZ-Grid architecture presentation as presented at FOSS4G

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Manaaki Whenua / Landcare Research is New Zealand's foremost environmental research organisation. Manaaki Whenua = Care for the land We do Environmental Landscape modelling
  • Trends Catchment modeling nutrient loss => hydrology model needs 1m dem 3d soil characterization => micron scale data
  • To respond to the challenges posed by our Scientists, Informatics came up with the SCENZ-Grid vision Data and Computational pressure Modelling environment Managed data Two Pillars: Collaboration and Computation
  • Collaboration Open architecture with a content repository at it’s heart, a cross cutting ‘catalog’ wider than just geospatial metadata or a content management system Spatial is ‘just a datatype’ in the database, likewise a dataset is just an object in the repository Standards: Open Archives Initiative - Object Reuse and Exchange SPARQL - S PARQL P rotocol a nd R DF Q uery L anguage Web Services in general
  • MyExperiment Evernote Social Networking DSpace
  • MyExperiment Taverna Kepler
  • Where does it fit in the Open Geo Stack?
  • Which components in the Geo Stack are we using?
  • Extensions to core WPS: WPS-T WPS-G
  • 1 x SGI Altix XE250 6 x SGI Altix XE320 Each XE320 is a pair of servers 1 + 6 x 2 = 13 nodes 2 x Intel quad core Xeons per node = 8 cores per node 13 x 8 = 104 cores
  • Example Focused on Collaboration as much as compute power, the analysis in this example does not necessarily require a cluster
  • What we’re talking about is the ‘regolith’ layer Solution has 2 components: Computation = WPS Algorithm Collaboration = Working on Similarity matrix
  • Q-Map is Geological mapping done by Geologists with their background LRIS / NZFSL is Soil map, has an attribute for bedrock which is mapped by Soil Scientists What we want to know is how the mappings from those 2 communities match LRIS is a pure hierarchical Classification: Order>Group>Subgroup>Class QMAP is a multi-attribute Classification Sedimentary>clastic Sediment>mud>mud but also mudstone>mud We assigned weights to the attributes in the classification > the more exact the description the higher the weight. If you add the weights you get the ‘similarity value’ a number between 0 and 20
  • Transcript

    • 1. SCENZ-Grid The implementation of a Science Collaboration and Computation Environment Niels Hoffmann Landcare Research
    • 2.
      • Manaaki Whenua / Landcare Research is
      • New Zealand's foremost environmental research organisation.
      • Our research focuses on three key areas:
      • Sustaining and restoring biodiversity;
      • Sustaining land environments;
      • Sustaining business and living.
      • Three themes relate to these three areas:
      • Climate change mitigation and adaptation
      • Maori sustainability
      • Invasive species and disease impacts
    • 3. Sustaining biodiversity & restoration Sustaining land environments Sustaining business & living Climate change Maori sustainable futures Weeds, pests and diseases Capability and collaboration Landcare Research Manaaki Whenua Key outcomes Cross-cutting outcomes Underpinning strengths
    • 4.
      • Data and Computational pressure
      • NOW – 25m national data density
      • NEAR FUTURE – sub 5m national data density /
      • (peri-)urban sub 1m LIDAR data density
      • Modelling environment
      • NOW – essentially batch oriented & 2.5D
      • DESIRED – interactive 4D, with real-time visualisation feedback
      • Managed data
      • NOW – preserve the data, memorise the model
      • DESIRED – keep the model for on-demand re-use
    • 5. science : collaboration : environment SCE NZ-Grid proposes that we can Do science research on-line together Share each other’s data – not duplicate it Collaboratively develop & use shared models / workflows Use shared compute resources Connect researchers directly to consumers : policy / managers / educators / public spatial : computation : engine
    • 6. Dublin Core / RDF Repository Web Services Doc Img Geo Name Search Consume Query Tag Comment Create Workflow OAI-ORE SPARQL
    • 7. A personal home page with links to relevant work for organising and retrieving a large variety of digital resources
    • 8. Detailed view of a resource – in this case a modelling workflow, that can be edited by maybe changing the inputs or logic and then re-used
    • 9. Databases Content (WMS, WFS, WCS) Functionality (WPS) Desktop Web
    • 10. PostGIS / SqlServer2008 GeoServer GeoWebCache 52North Geoprocessing ArcGIS / Udig OpenLayers GeoExt
    • 11.
      • Why WPS ?
      • Distributed Architecture
      • Interoperability
      • Modeling approach
      • (as opposed to data centric outcome)
      • (Grid-) Computing
    • 12.
      • Why 52North ?
      • Build upon robust OS Libraries (JTS, Geotools, xmlBeans, Servlet API)
      • Pluggable framework for algorithms
      • Support for raster processing
      • Support for Grid-Computing
    • 13.
      • Currently using Unicore Middleware
      • Planning a migration to Globus Middleware to integrate with BeSTGRID
      Landcare repository Sextante repository
    • 14. WPS Geoserver 104 Intel Xeon cores 2.8GHz each 386GB RAM 2.6TB storage ~1.16 TFLOPS Air cooled Gb Ethernet nterconnects 4.2kW power
    • 15.
      • Spatially query across datasets from multiple organisations.
      • Qmap: Geology from GNS
      • NZFSL: Soil Data from Landcare Research
      • What kind of relationships exist between soil type and bedrock.
      • What is the association between groundwater quality and soil.
      • Investigate relationships between ecology, ground water and soil.
    • 16.
      • Regolith:
      • Layer between Bedrock and Soil
      • Erosion modelling
      • Groundwater flow modelling
      • WPS Algorithm to combine 2 datasets based on a similarity matrix
      • User interface to enable experts to adapt the similarity matrix
    • 17. Establish the ‘similarity’ of the datasets Different origin of attributes Different classification Similarity Matrix quantify where the classifications match Expert to decide similarity based on documentation
    • 18. QMAP WMS NZFSL WMS Lookup WS Portal User WPS User Interface View Portlet WebService WorkFlow Engine WPS Web Services Data: WMS Business Logic: SOAP
    • 19. Lookup WS WPS User
    • 20.  
    • 21.
      • Thank You,
      • Any questions?
      • [email_address]
      • Slide Credits:
      • Chris McDowall, Aaron Hicks
      science : collaboration : environment spatial : computation : engine