This document discusses using deep learning models to segment aerial imagery from a mapping contest. It provides an overview of deep learning concepts like artificial neurons, neural networks, convolutional neural networks and fully convolutional networks. It then describes experiments using a U-Net and FCN models with cross validation and ensemble learning to segment the aerial imagery. Results of the deep learning predictions are visualized on an interactive map. Future work discussed includes applying these techniques to other computer vision tasks and improving the infrastructure for distributed training.

1. Deep Learning on Aerial Imagery: What does it look
like on a map?
Rob Emanuele (@lossyrob)

2. What we’ll cover
• Describe the ISPRS Potsdam image
segmentation contest
• Brief overview of some Deep Learning
concepts
• View and analyze results of model
predictions on a map

6. GeoTrellis
• a Scala library for geospatial data types and
operations.
• enables Spark to work with geospatial data
for raster processing at scale
• provides lots of raster processing capability
for web servers and distributed computation

7. DSM generation

8. DEEP LEARNING

9. WHAT IS DEEP
LEARNING?

10. σ
x
1
x
3
y
Inputs (xi) are weighted
by the wis
The neuron “fires” if the
summed activation (x·w)
is above a threshold,
giving y
[In reality, y = σ(x·w),
where σ is some
“thresholding” function]
w1
w2
w3
x
2
Artificial Neurons

11. https://commons.wikimedia.org/wiki/File:Artificial_neural_network.svg
Artificial Neural Network (ANN)

12. Deep Neural Network (DNN)
…

13. Deep Neural Network (DNN)
…
THE DEEP PART

14. Convolutional Neural Network (CNN)
https://commons.wikimedia.org/wiki/File:Typical_cnn.png

15. Fully Convolutional Network (FCN)

16. Source: https://lmb.informatik.uni-freiburg.de/Publications/2015/RFB15a/
U-Net: Convolutional Networks for Biomedical
Image Segmentation

21. EXPERIMENT 1:
U-NET

23. EXPERIMENT 2:
FCN w/ IRRG

25. EXPERIMENT 3:
CROSS VALIDATION LATE
FUSION
ENSEMBLE

32. LET’S EXPLORE THE MAP

33. FUTURE WORK

34. Future Raster Vision Work
• Other CV tasks: Image Recognition, Object
Detection
• Infrastructure/Workflow improvements: utilize
AWS Batch for distributed training over
hyperparameters
• Integration into
• Inference on the fly, on the map or through an
API

35. THANK YOU
Rob Emanuele (@lossyrob)