#IA Track - Technology and Tools The volume of Earth Observation imagery, acquired by satellites and drones is huge (~100 To/day). DeepLearning approach allow to switch from pixels to insights, the only way to be able to really use these initial raw data. RoboSat.pink is a modular and extensible OpenSource Computer Vision framework, bridging cuting-edges research papers to a robust implementation. This presentation will focus on: - Computer Vision 101 reminder - The specificities of geospatial data aka "Spatial is Special" - Mains RoboSat.pink use cases - Current research issues in Computer Vision GeoSpatial

1. RoboSat.pink :
Computer Vision framework for GeoSpatial Imagery
@o_courtin
@OSS_Paris 2019

2. Earth Observation

3. Earth Observation
Public Policy Monitoring
BioSphere Study
Human Development
Intelligence
Economic Intelligence
Emergency & Crisis

4. From Pixels to Insights

5. THE PERCEPTRON
A PROBABILISTIC MODEL FOR INFORMATION STORAGE
AND ORGANIZATION IN THE BRAIN
Rosenblatt 1958
AI is an old Lady
https://en.wikipedia.org/wiki/Perceptron

6. Data
V
E
C
T
O
R
Classification
World2Vec

7. Neurals
Network
Output
Input
Loss Function
Supervised Learning

8. Neurals
Network
Output
Input
Loss Function
Trained
Model Output
Supervised Learning

9. Polynom
Weighted Graph
Lossy Data Compression
Grey Box
A Trained model ?

10. « Everything is related to everything else,
but near things are more related than distant things. »
Waldo Tobler

11. https://github.com/vdumoulin/conv_arithmetic
Convolutions

12. Unet Like Semantic Segmentation
Image Credit: https://divamgupta.com

13. Medical ImageryAutonomous Vehicle GeoSpatial Imagery
Computer Vision
Images Credit : Albumentations

14. Spatial is Special
MultiBands
Data Complexity
Huge Data

15. Spatial is Special
MultiBands
Data Complexity
Huge Data
Interoperability
FOSS4G
Open Data

16. RoboSat.pink
Computer Vision ecosystem for GeoSpatial Imagery

17. RoboSat.pink
Computer Vision ecosystem for GeoSpatial Imagery
DataSet Quality Analysis
Change Detection highlighter
Features extraction

18. RoboSat != RoboSat.pink
https://github.com/mapbox/robosat/issues/184
https://github.com/mapbox/robosat https://github.com/datapink/robosat.pink

19. RoboSat.pink
Raster
Coverage
WMS
XYZ

20. RoboSat.pink
GeoJSON
PostGIS
Raster
Coverage
WMS
OSM PBF
XYZ

21. RoboSat.pink
GeoJSON
PostGIS
Raster
Coverage
WMS
OSM PBF
XYZ
Masks
Prediction

22. RoboSat.pink
GeoJSON
PostGIS
Raster
Coverage
WMS
OSM PBF
XYZ
Masks
Prediction
Masks
Compare
Vector
Prediction
Spotify
differences
areas

23. Compare Predicts against alternate datasets
Pink : Predicted by trained model
Green : Alternate dataset
Grey : Both agree

24. Compare Predicts against alternate datasets
Pink squares : Significant differences

25. Compare Predicts against alternate datasets
Pink squares : Significant differences
Zoom out

26. Command Line Interface

27. Slippy Map

28. Download
WMS
XYZ
Rasterize
GeoJSON
ExtractOSM pbf
Cover
XYZ Image
Tile
Rasters
XYZ Label
Subset
Training
DataSet
Bbox
XYZ dir
Data Preparation
Rasters
PostGIS

29. Image Label Cross Entropy mIoU Lovasz
http://www.cs.toronto.edu/~wenjie/papers/iccv17/mattyus_etal_iccv17.pdf
http://www.cs.umanitoba.ca/~ywang/papers/isvc16.pdf
https://arxiv.org/abs/1705.08790
Surface based Loss

30. https://arxiv.org/abs/1809.06839
https://github.com/albu/albumentations
Data Augmentations

31. https://arxiv.org/abs/1806.00844
Unet like, with ResNet flavor encoder

32. Easy to deploy
pip3 install RoboSat.pink

33. https://github.com/datapink/robosat.pink/blob/master/docs/101.md
101 RoboSat.pink Tutorial

34. https://github.com/datapink/robosat.pink/blob/master/docs/101.md
101 RoboSat.pink Tutorial
In ~2-3h, on a GPU server:
- RoboSat.pink install
- Download data
- Data Preparation
- Training
- Inference
- Compare to OSM
- Vectorize result

35. More than an application, an easy to extent framework
https://github.com/datapink/robosat.pink/blob/master/docs/extensibility_by_design.md

36. Stacks
Proj 4
GEOS GDAL
Rasterio
CUDAcuDNN
PyTorch
NumPy
OpenCV
RoboSat.pink
Pillow
Shapely Osmium
Mercantile
SuperMercado
Albumentations
LeafLet
+ VectorGrid

37. So all you need is :
- Imagery → any file format readable by GDAL
- GPU → NVIDIA > 8Go RAM
- Initial skills → GeoSpatial Data and CLI fluency
- Labels → usualy the key point

38. https://github.com/datapink/robosat.pink#geospatial-opendatasets

39. Accurate Labels
Imagery Lyon OpenData roofprints OSM Building footprints

40. GIGO

41. From OpenData to OpenDataSet
https://github.com/datapink/robosat.pink/blob/master/docs/from_opendata_to_opendataset.md

42. Few performances Metrics
rsp train ~5 Mp/s, -per epoch-
rsp tile ~5 Mp/s
rsp predict ~5 Mp/s
rsp rasterize ~50 Mp/s
rsp vectorize ~50 Mp/s
8 cores CPU, single GPU (either RTX or V100), SSD
16 tiles = 4 Retina Tiles = 1Mp

43. How to scale it, or improve it again ?
rsp train add more GPU,
reduce dataset redundancy,
improve model, loss or optimizer
rsp tile add more CPU
use raster compression
rsp predict export model to ONNX or JIT,
then use an high performance inference solution.

44. RTX ThreadRipper
Cost Effective GPU
WorkStation
Server
Cloud
SuperMicro

45. Why performances matters ?
- Playful and Human Learning
- Time and money saver
- No Planet B

46. Open Source
https://github.com/datapink/robosat.pink

47. Request For Funding
- Increase prediction accuracy :
- on low resolution imagery
- even with few labels
- on network features
- instance segmentation

48. Request For Funding
- Increase prediction accuracy :
- on low resolution imagery
- even with few labels
- on network features
- instance segmentation
- Improve performances
- on Training
- on Inference

49. Request For Funding
- Increase prediction accuracy :
- on low resolution imagery
- even with few labels
- on network features
- instance segmentation
- Improve performances
- on Training
- on Inference
- Add support for :
- MultiClass classification
- Time Series Imagery

50. Why using Deep Learning for GeoSpatial ?
Easy to spotify at scale inconsistencies beetwen two datasets

51. Why using Deep Learning for GeoSpatial ?
Easy to spotify at scale inconsistencies beetwen two datasets
If you provide accurate labels matching an imagery,
infere at scale on similar new imageries

52. Why using Deep Learning for GeoSpatial ?
Easy to spotify at scale inconsistencies beetwen two datasets
If you provide accurate labels matching an imagery,
infere at scale on similar new imageries
But prefer Machine Learning if your classification is trivial

53. Open Source AI4EO

54. Why using RoboSat.pink ?GIS Standards compliancy
Ease Data Preparation
Build-in WebUI
Handle MultiBands Imagery and DataFusion
High Performances
Easy to deploy
Accurate (state of art Computer Vision)
Extensible by design
Open Source
Community

56. DIY Demo
www.datapink.com
Click Here

57. Human Learning
http://www.math.ens.fr/~feydy/Teaching/culture_mathematique.pdf
http://cs231n.stanford.edu/
https://neurovenge.antonomase.fr/NeuronsSpikeBack.pdf
http://www.numerical-tours.com/python/

58. Pattern
Extraction
Computer
Engineering
Data Expertise
Pattern
Extraction
Computer
Engineering
Data Expertise
Maslow & Skills

59. Extract insights from GeoSpatial data with Deep Learning
@data_pink
www.datapink.com

60. RoboSat.pink powered by @data_pink
Take Away
- Industrial AI4EO Imagery framework available
- Performances already OK to use it for region or country
- No need anymore to be a Computer Vision expert to use it
- Plain OpenData can be use to train accurate model
- Funding and Pull Requests can make the difference