Grass cloud
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Grass cloud

on

  • 1,838 views

The presentation of g.cloud module for GRASS GIS at FOSS4G 2011

The presentation of g.cloud module for GRASS GIS at FOSS4G 2011

Statistics

Views

Total Views
1,838
Views on SlideShare
1,838
Embed Views
0

Actions

Likes
1
Downloads
22
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Grass cloud Presentation Transcript

  • 1. Introduction Idea Implementation Conclusions GRASS GIS in the Cloud Luca Delucchi, Markus NetelerFondazione Edmund Mach – GIS & Remote Sensing Platform http://gis.cri.fmach.it FOSS4G 2011 14th September 2011, Denver (USA) Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 2. Introduction Idea GRASS GIS Implementation Our cluster ConclusionsGRASS GIS The first Open Source GIS Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 3. Introduction Idea GRASS GIS Implementation Our cluster ConclusionsGRASS GIS The first Open Source GIS More than 400 modules for every kind of analysis Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 4. Introduction Idea GRASS GIS Implementation Our cluster ConclusionsGRASS GIS The first Open Source GIS More than 400 modules for every kind of analysis Interoperability with other software Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 5. Introduction Idea GRASS GIS Implementation Our cluster ConclusionsOur cluster: infrastructure FEM GIS cluster consists of 300 nodes 610 GB RAM 10 TB disk storage + 15 TB tape backup 10 Gb/s internal bus Scientific Linux 5.6 Grid Engine Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 6. Introduction Idea GRASS GIS Implementation Our cluster ConclusionsOur cluster: massive spatial data analysis MODIS Land Surface Temperature (LST) map gap-filling (13,000 maps) MODIS data analysis (LST, NDVI, Snow products) LiDAR data analysis Solar energy calculations Neteler et al., 2011: Tiger mosquito, IJHG, doi:10.1186/1476-072X-10-49 Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 7. Introduction Idea Implementation ConclusionsIdea to facilitate GRASS GIS usage on a cluster Simplify the work Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 8. Introduction Idea Implementation ConclusionsIdea to facilitate GRASS GIS usage on a cluster Simplify the work Offer the possibility to use Grid Engine by all our colleagues Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 9. Introduction Idea Implementation ConclusionsIdea to facilitate GRASS GIS usage on a cluster Simplify the work Offer the possibility to use Grid Engine by all our colleagues Launch jobs remotely without connecting every time to the cluster Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 10. Introduction Idea Implementation ConclusionsIdea to facilitate GRASS GIS usage on a cluster Simplify the work Offer the possibility to use Grid Engine by all our colleagues Launch jobs remotely without connecting every time to the cluster Offer the possibility to external people to use the cluster without learning Grid Engine Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 11. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: requirements GRASS GIS 7 (current development version) Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 12. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: requirements GRASS GIS 7 (current development version) Grid Engine Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 13. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: requirements GRASS GIS 7 (current development version) Grid Engine Python > 2.4 Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 14. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: how does it work? Required parameters conf = file with username and password or stdin to connect to the cluster Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 15. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: how does it work? Required parameters conf = file with username and password or stdin to connect to the cluster server = hostname of the cluster Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 16. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: how does it work? Required parameters conf = file with username and password or stdin to connect to the cluster server = hostname of the cluster qsub script = file containing qsub script (template) Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 17. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: how does it work? Required parameters conf = file with username and password or stdin to connect to the cluster server = hostname of the cluster qsub script = file containing qsub script (template) grass script = own GRASS GIS script Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 18. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: how does it work? Required parameters conf = file with username and password or stdin to connect to the cluster server = hostname of the cluster qsub script = file containing qsub script (template) grass script = own GRASS GIS script g.cloud conf=/tmp/passwd server=giscluster qsub script=test launch SGE grassjob.sh grass script=test novariables.sh Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 19. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: how does it work? Optional parameters raster = name of raster(s) Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 20. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: how does it work? Optional parameters raster = name of raster(s) vector = name of vector(s) Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 21. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: how does it work? Optional parameters raster = name of raster(s) vector = name of vector(s) variables = string with name and values of variables stated as Python dictionary “’key’:[’value0’,’value1’]” Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 22. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: how does it work? Optional parameters raster = name of raster(s) vector = name of vector(s) variables = string with name and values of variables stated as Python dictionary “’key’:[’value0’,’value1’]” email = user’s email address Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 23. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: how does it work? Optional parameters raster = name of raster(s) vector = name of vector(s) variables = string with name and values of variables stated as Python dictionary “’key’:[’value0’,’value1’]” email = user’s email address path = path to the folder which must be accessible from cluster frontend and blades Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 24. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: more examples North Carolina sample data r.texture example g.cloud conf=/tmp/passwd server=giscluster qsub script=test launch SGE grassjob.sh grass script=test onevariables raster.sh variables=“’TEXT’ : [’asm’,’contrast’,’corr,var’,’idm’,’sa’,’se’,’sv’,’entr’,’dv’,’de’,’moc1’,’moc2’]” raster=lsat7 2002 40 Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 25. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: more examples North Carolina sample data r.texture example g.cloud conf=/tmp/passwd server=giscluster qsub script=test launch SGE grassjob.sh grass script=test onevariables raster.sh variables=“’TEXT’ : [’asm’,’contrast’,’corr,var’,’idm’,’sa’,’se’,’sv’,’entr’,’dv’,’de’,’moc1’,’moc2’]” raster=lsat7 2002 40 r.sun example g.cloud conf=/tmp/passwd server=giscluster qsub script=test launch SGE grassjob.sh grass script=test onevariables sun.sh variables=“’DOY’ : [1,2,3,4,5,6,7,8,9,10]” raster=elevation Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 26. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: future Support for more clustering system (for example Eucalyptus) Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 27. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: future Support for more clustering system (for example Eucalyptus) Auto installation of GRASS and its dependencies on the cluster if not present (GRASS 7 release as stable version) Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 28. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: future Support for more clustering system (for example Eucalyptus) Auto installation of GRASS and its dependencies on the cluster if not present (GRASS 7 release as stable version) Show the status of jobs Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 29. Introduction Requirements Idea How does it work? Implementation Future ConclusionsImplementation: future Support for more clustering system (for example Eucalyptus) Auto installation of GRASS and its dependencies on the cluster if not present (GRASS 7 release as stable version) Show the status of jobs Return outputs to the client Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 30. Introduction Idea Implementation ConclusionsConclusions First implementation of a module to run GRASS on a cluster Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 31. Introduction Idea Implementation ConclusionsConclusions First implementation of a module to run GRASS on a cluster Experimental version Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 32. Introduction Idea Implementation ConclusionsConclusions First implementation of a module to run GRASS on a cluster Experimental version Please test it and report bugs or improvements Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 33. Introduction Idea Implementation ConclusionsConclusions First implementation of a module to run GRASS on a cluster Experimental version Please test it and report bugs or improvements Contact us for more info markus.neteler@iasma.it - luca.delucchi@iasma.it Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver
  • 34. Introduction Idea Implementation ConclusionsConclusions Thanks for your attention gis.cri.fmach.it Luca Delucchi, Markus Neteler FOSS4G 2011, 14th September 2011, Denver