The document discusses raster processing using the SciPy ndimage library, highlighting its open-source nature and suitability for large-scale concurrent processing. It explains how to set up data input and output using arcpy with numpy, and illustrates operations such as calculating median filters, slopes, and focal maximums. The author emphasizes the efficiency and flexibility of ndimage compared to ArcGIS's spatial analyst tools.

1. Raster processing with
scipy.ndimage
Henry Walshaw
henry@pythoncharmers.com
@om_henners
N
om
gis
ta
.s
.c
e
s
s
s
s
s
s
s
s
ck
exc h a n g

2. Getting set up
arcpy for ArcGIS 10 requires numpy 1.3.0
This means we’re restricted to scipy 0.7.1, matplotlib 1.0.1
and PIL 1.1.7 (which is still the latest version)
If you’re not using ArcGIS, or you’re using virtualenv get the
latest versions!
All code for this talk can be downloaded from github:
https://github.com/om-henners/ndimage_talk.git

3. Why process with scipy?
Open Source scientific algorithms
Easy to set up for large concurrent processing on local PCs
and in the cloud (see PiCloud)
It’s free!*
*Well, aside from development cost

4. Getting data in and out
We’ll be using the arcpy.RasterToNumPyArray and
arcpy.NumPyArrayToRaster functions
Alternatives include GDAL Python bindings, scipy image read
functions, and many others
See getting_data_in.py and getting_data_out.py

5. Calculating median filter
Once you’ve got an array it’s
one line of code to perform a
basic filter function
scipy.ndimage.filters.median_filter(a,
size=9)

6. Getting more complex
n-dimensional processing
We can supply a size or filter
shape, but now we have to be
even more aware of edge
effects
Still the same one line of code

7. Generic filters in scipy
Most Spatial Analyst operations are in ndimage
For everything else there’s generic_filter
Flattens the target region and passes through to a callback
function
Can be used to handle null data as ndimage can’t handle
masked arrays (yet)

8. Calculating slope
We can implement a simple slope calculation using generic
filter over a 3x3 footprint
We can use the form as described by the ESRI documentation:
slope = e is atan ( sqrt( [dz/dx]**2 + [dz/dy]**2 ) ) where
[dz/dx] = ((c + 2f + i) - (a + 2d + g) / (8 * x_cellsize)
[dz/dy] = ((g + 2h + i) - (a + 2b + c)) / (8 * y_cellsize))
Slower than a standard ndimage filter, but faster than
arcpy!

9. Calculating a variable focal
maximum
Calculate the focal maximum for every point in an array, with
the focal annulus defined by a radius from another raster
Don’t use a generic filter (there’s no need)
Don’t use ArcGIS Spatial analyst (it’s really slow)
Method 100x100 random raster
generic filter 4.7 secs
arcpy Spatial Analyst 6.7 secs

10. Do use inbuilt functions
It’s faster to calculate the focal
sum using
ndimage.maximum_filter
for every possible buffer value
than to roll your own function
100x100 array? 0.009
seconds

11. Last words on scipy.ndimage
It’s awesome, and free and threadsafe
It ties into any number of other packages (scikits-learn, scikits-
image)
You can do just about anything in ndimage that you can in
Spatial Analyst
If you have time to code it
And code it right

12. John Hunter (1968 - 2012)
http://numfocus.org/johnhunter/