Raster Processing with Scipy.ndimage (Dev Meet Up II)
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


Raster Processing with Scipy.ndimage (Dev Meet Up II)



Presentation by Henry Walshaw. For Esri Australia's Developer Meetup in Melbourne 27/09/12. Taking a look at SciPy for raster processing.

Presentation by Henry Walshaw. For Esri Australia's Developer Meetup in Melbourne 27/09/12. Taking a look at SciPy for raster processing.



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Raster Processing with Scipy.ndimage (Dev Meet Up II) Presentation Transcript

  • 1. Raster processing withscipy.ndimageHenry Walshawhenry@pythoncharmers.com @om_henners N om gis ta .s .c e s s s s s s s s ck exc h a n g
  • 2. Getting set uparcpy for ArcGIS 10 requires numpy 1.3.0This means we’re restricted to scipy 0.7.1, matplotlib 1.0.1and PIL 1.1.7 (which is still the latest version)If you’re not using ArcGIS, or you’re using virtualenv get thelatest versions!All code for this talk can be downloaded from github:https://github.com/om-henners/ndimage_talk.git
  • 3. Why process with scipy?Open Source scientific algorithmsEasy to set up for large concurrent processing on local PCsand in the cloud (see PiCloud)It’s free!* *Well, aside from development cost
  • 4. Getting data in and outWe’ll be using the arcpy.RasterToNumPyArray andarcpy.NumPyArrayToRaster functionsAlternatives include GDAL Python bindings, scipy image readfunctions, and many othersSee getting_data_in.py and getting_data_out.py
  • 5. Calculating median filter Once you’ve got an array it’s one line of code to perform a basic filter functionscipy.ndimage.filters.median_filter(a,size=9)
  • 6. Getting more complexn-dimensional processingWe can supply a size or filtershape, but now we have to beeven more aware of edgeeffectsStill the same one line of code
  • 7. Generic filters in scipyMost Spatial Analyst operations are in ndimageFor everything else there’s generic_filterFlattens the target region and passes through to a callbackfunctionCan be used to handle null data as ndimage can’t handlemasked arrays (yet)
  • 8. Calculating slopeWe can implement a simple slope calculation using genericfilter over a 3x3 footprintWe can use the form as described by the ESRI documentation:slope = e is atan ( sqrt( [dz/dx]**2 + [dz/dy]**2 ) ) where[dz/dx] = ((c + 2f + i) - (a + 2d + g) / (8 * x_cellsize)[dz/dy] = ((g + 2h + i) - (a + 2b + c)) / (8 * y_cellsize))Slower than a standard ndimage filter, but faster thanarcpy!
  • 9. Calculating a variable focalmaximumCalculate the focal maximum for every point in an array, withthe focal annulus defined by a radius from another rasterDon’t use a generic filter (there’s no need)Don’t use ArcGIS Spatial analyst (it’s really slow) Method 100x100 random raster generic filter 4.7 secs arcpy Spatial Analyst 6.7 secs
  • 10. Do use inbuilt functionsIt’s faster to calculate the focalsum usingndimage.maximum_filterfor every possible buffer valuethan to roll your own function100x100 array? 0.009seconds
  • 11. Last words on scipy.ndimageIt’s awesome, and free and threadsafeIt ties into any number of other packages (scikits-learn, scikits-image)You can do just about anything in ndimage that you can inSpatial Analyst If you have time to code it And code it right
  • 12. John Hunter (1968 - 2012)http://numfocus.org/johnhunter/