Work done as final thesis presentation for the degree of Master of Science in Geospatial Technologies.

1. al393628@uji.es
janak.parajuli1@gmail.com
05/03/2021
www.mastergeotech.info
Dissertation submitted in partial fulfillment of the requirements for
the Degree of Master of Science in Geospatial Technologies
EXTRACTING SURFACE WATER BODIES FROM SENTINEL-2 IMAGERY USING
CONVOLUTIONAL NEURAL NETWORKS
Supervised by:
Prof. Filiberto Pla Bañón, PhD
Prof. Marco Painho, PhD
Rubén Fernández-Beltrán, PhD
Janak Parajuli

2. 1 • Introduction
2 • Pre-Processing
3 • Processing
4 • Post-Processing
5 • Limitations & Recommendation
6 • Conclusion
CONTENTS

3. INTRODUCTION
Monitoring of Surface Water Bodies
Satellite Imagery
Convolutional Neural Networks

4. PROBLEM STATEMENT
Limitations
of
Indices
• Not suitable to global
scale
• Necessity of auxiliary
data
• Complex band
equations
• Suitable threshold
values
Problems
with
larger
depths
in
CNN
• Loss of information
• Vanishing Gradient
• Increase in number of
parameters
• High computational
cost
Applicability
of
state-of-
the-art
CNN
approaches
• Applied only on
standard datasets
• Need for thorough
exploration
• Implementation on
satellite images

5. PROBLEM STATEMENT
Limitations
of
Indices
• Not suitable to global
scale
• Necessity of auxiliary
data
• Complex band
equations
• Suitable threshold
values
Problems
with
larger
depths
in
CNN
• Loss of information
• Vanishing Gradient
• Increase in number of
parameters
• High computational
cost
Applicability
of
state-of-
the-art
CNN
approaches
• Applied only on
standard datasets
• Need for thorough
exploration
• Implementation on
satellite images

6. PROBLEM STATEMENT
Limitations
of
Indices
• Not suitable to global
scale
• Necessity of auxiliary
data
• Complex band
equations
• Suitable threshold
values
Problems
with
larger
depths
in
CNN
• Loss of information
• Vanishing Gradient
• Increase in number of
parameters
• High computational
cost
Applicability
of
state-of-
the-art
CNN
approaches
• Applied only on
standard datasets
• Need for thorough
exploration
• Implementation on
satellite images

7. AIM & OBJECTIVES
Extracting Surface Water Bodies from
Sentinel-2 Imagery using CNNs
•Explore state-of-the-art
approaches in CNN
•Implement & Compare
their performance
•Design and propose a
novel approach in CNN

8. WORKFLOW
1
• Data Download and Pre-Processing
2
• Preparation of Image-Label Tiles
3
• Patch Extraction
• Preparation of Patch-Label Pairs
4
• Selection & Configuration of Neural
Networks
5
• Experimentation
6
• Integration of Models
• Evolution of New Architect
7
• Choice of Indices
8
• Development of EWI
9
• Experimentation
10
• Performance Assessment
Pre-Processing
Processing
Post-Processing

9. STUDY AREA

10. DATASET PREPARATION
Level-2A
Image
Level-1C
Image
Cloud
Free
Atmospheric
Correction
Resampling
Yes
No
Subsetting
Spatial
Spectral
DEM
Project
Resampling
Clipping
River
Dataset
Data
Correction
Project
Rasterize
Clipping
UTM Zone
44N
UTM Zone
45N
Composite
Bands
Patch-Label
Pairs
Used QGIS
Used SNAP CLI
Used ArcGIS Pro
Legend

11. DATASET PREPARATION
Level-2A
Image
Level-1C
Image
Cloud
Free
Atmospheric
Correction
Resampling
Yes
No
Subsetting
Spatial
Spectral
DEM
Project
Resampling
Clipping
River
Dataset
Data
Correction
Project
Rasterize
Clipping
UTM Zone
44N
UTM Zone
45N
Composite
Bands
Patch-Label
Pairs
Used QGIS
Used SNAP CLI
Used ArcGIS Pro
Legend

12. DATASET PREPARATION
Level-2A
Image
Level-1C
Image
Cloud
Free
Atmospheric
Correction
Resampling
Yes
No
Subsetting
Spatial
Spectral
DEM
Project
Resampling
Clipping
River
Dataset
Data
Correction
Project
Rasterize
Clipping
UTM Zone
44N
UTM Zone
45N
Composite
Bands
Patch-Label
Pairs
Used QGIS
Used SNAP CLI
Used ArcGIS Pro
Legend

13. Convolution Max-Pooling Convolution Max-Pooling
Fully
Connected
Output
SELECTION OF NEURAL NETWORKS
Input
CNNWQC proposed by Pu (2019) et al.
Convolution Convolution Convolution Convolution
Fully
Connected
Fully
Connected
Output
Input
Baseline Architect

14. SELECTION OF NEURAL NETWORKS
Convol
ution
Max-
Pooling
Convol
ution
Max-
Pooling
Convol
ution
Convol
ution
Convol
ution
Max-
Pooling
Fully
Conne
cted
Fully
Conne
cted
Output
Input
CNNCWC proposed by Rezaee (2018) et al.
Convolution
Self-Adaptive
Pooling
Convolution
Self-Adaptive
Pooling
Fully
Connected
Output
Input
SAPCNN proposed by Chen (2018) et al.

15. SELECTION OF NEURAL NETWORKS
Convolution
Batch
Normalization
Blocks ReLU
Global
Average
Pooling
Output
Input
DenseNet proposed by Huang (2017) et al.

16. SELECTION OF NEURAL NETWORKS
Convolution
Batch
Normalization
Blocks ReLU
Global
Average
Pooling
Output
Input
Convolutional Block
DENSE BLOCK 1
TRANSITION BLOCK 1
Convolutional Block
DENSE BLOCK 2
TRANSITION BLOCK 2
Convolutional Block
DENSE BLOCK 3
DenseNet proposed by Huang (2017) et al.

17. SELECTION OF NEURAL NETWORKS
AttResNet proposed by Wang (2017) et al.

18. SELECTION OF NEURAL NETWORKS
Trunk Branch Soft-Mask Branch Trunk Branch
ATTENTION BLOCK
AttResNet proposed by Wang (2017) et al.

19. SELECTION OF NEURAL NETWORKS
Convolution
Batch
Normalization
Max-Pooling
Residual
Block 2
Attention
Block 1
Residual
Block 1
Trunk Branch Soft-Mask Branch Trunk Branch
ATTENTION BLOCK
AttResNet proposed by Wang (2017) et al.
RESIDUAL BLOCK

20. PROPOSED ARCHITECT: AttDenseNet
Convolution
Batch
Normalization
Attention
Block
Blocks ReLU
Global
Average
Pooling
Output
Input
A novel CNN architect proposed by
integrating DenseNet and AttResNet

21. PROPOSED ARCHITECT: AttDenseNet
Convolution
Batch
Normalization
Attention
Block
Blocks ReLU
Global
Average
Pooling
Output
Input
ReLU
Convolution
Concatenate
Convolutional Block
DENSE BLOCK 1
Convolution
Average
Pooling
Batch
Normalization
TRANSITION BLOCK 1
ReLU
Convolution
Concatenate
Convolutional Block
DENSE BLOCK 2
Convolution
Average
Pooling
Batch
Normalization
TRANSITION BLOCK 2
ReLU
Convolution
Concatenate
Convolutional Block
DENSE BLOCK 3
A novel CNN architect proposed by
integrating DenseNet and AttResNet

22. PROPOSED ARCHITECT: AttDenseNet
Convolution
Batch
Normalization
Attention
Block
Blocks ReLU
Global
Average
Pooling
Output
Input
ReLU
Convolution
Concatenate
Convolutional Block
DENSE BLOCK 1
Convolution
Average
Pooling
Batch
Normalization
TRANSITION BLOCK 1
ReLU
Convolution
Concatenate
Convolutional Block
DENSE BLOCK 2
Convolution
Average
Pooling
Batch
Normalization
TRANSITION BLOCK 2
ReLU
Convolution
Concatenate
Convolutional Block
DENSE BLOCK 3
A novel CNN architect proposed by
integrating DenseNet and AttResNet
Residual Block
Encoder-
Decoder
Residual Block
Trunk Branch
Soft-Mask Branch
Trunk Branch

23. First Output
Convolution
Batch
Normalization
Attention Block
Feature
Extraction by
TB
Feature
Selection by
SMB
Dense Block
Feature
Propagation
Re-use
Checks
Vanishing
Gradient
Transition
Block
Volume and
Feature Maps
Halved
Computational
Cost
Reduction
Global
Average
Pooling
Remove
parameters to
optimize
Avoids
Overfitting
Output
Softmax
Classifier
Probability
Calculation
MECHANISM of AttDenseNet

24. 1
• Only RGB Channels as Input
2
• Selected S2 Channels as Input
3
• Contribution of DEM on Accuracy
4 • S2 Channels Integrated with DEM
EXPERIMENTS
On Patch Sizes
8, 12, 16 & 20

25. SELECTION OF INDICES
• 𝑁𝐷𝑊𝐼 =
𝐺𝑟𝑒𝑒𝑛−𝑁𝐼𝑅
Grⅇⅇ𝑛+𝑁𝐼𝑅
• 𝑁𝐷𝑉𝐼 =
𝑁𝐼𝑅−𝑅𝑒𝑑
𝑁𝐼𝑅+𝑅𝑒𝑑
• 𝑁𝐷𝑉𝐼_𝑁𝐷𝑊𝐼
• 𝐸𝑊𝐼 =
𝑁𝐷𝑊𝐼−(𝑃𝐶1+𝑃𝐶2)
𝑁𝐷𝑊𝐼+(𝑃𝐶1+𝑃𝐶2)
Yang (2017) et al.

26. SELECTION OF INDICES
• 𝑁𝐷𝑊𝐼 =
𝐺𝑟𝑒𝑒𝑛−𝑁𝐼𝑅
Grⅇⅇ𝑛+𝑁𝐼𝑅
• 𝑁𝐷𝑉𝐼 =
𝑁𝐼𝑅−𝑅𝑒𝑑
𝑁𝐼𝑅+𝑅𝑒𝑑
• 𝑁𝐷𝑉𝐼_𝑁𝐷𝑊𝐼
• 𝐸𝑊𝐼 =
𝑁𝐷𝑊𝐼−(𝑃𝐶1+𝑃𝐶2)
𝑁𝐷𝑊𝐼+(𝑃𝐶1+𝑃𝐶2)
NDWI
Reflectance
Spectral
Graph
PC1
Reflectance
Spectral
Graph
PC2
Reflectance
Spectral
Graph
Input
Derive
Equation
Extract Water
Yang (2017) et al.

27. 79
81
83
85
87
8 12 16 20
Test
Accuracy
Patch Size
TEST ACCURACY VS PATCH SIZE
Baseline
CNNWQC
CNNCWC
SAPCNN
DenseNet
AttResNet
RESULTS & DISCUSSION
62
64
66
68
70
72
74
76
78
80
8 12 16 20
Recall
of
Water
Patch Size
RECALL OF WATER VS PATCH SIZE
Baseline
CNNWQC
CNNCWC
SAPCNN
DenseNet
AttResNet
Quantitative Analysis: All Models
Performance of neural networks for RGB channels as input

28. 84
86
88
90
8 12 16 20
Test
Accuracy
Patch Size
TEST ACCURACY VS PATCH SIZE
Baseline
CNNWQC
CNNCWC
SAPCNN
DenseNet
AttResNet
RESULTS & DISCUSSION
70
72
74
76
78
80
82
84
8 12 16 20
Recall
of
Water
Patch Size
RECALL OF WATER VS PATCH SIZE
Baseline
CNNWQC
CNNCWC
SAPCNN
DenseNet
AttResNet
Quantitative Analysis: All Models
Performance of neural networks for selected S2 channels as input

29. 80
82
84
86
8 12 16 20
Test
Accuracy
Patch Size
TEST ACCURACY VS PATCH SIZE
With DEM
Without DEM
RESULTS & DISCUSSION
64
66
68
70
72
74
76
8 12 16 20
Recall
of
Water
Patch Size
RECALL OF WATER VS PATCH SIZE
With DEM
Without DEM
Quantitative Analysis: Baseline Model
Contribution of DEM with RGB on Baseline model

30. 86
88
90
92
8 12 16 20
Test
Accuracy
Patch Size
TEST ACCURACY VS PATCH SIZE
Baseline
CNNWQC
CNNCWC
SAPCNN
DenseNet
AttResNet
RESULTS & DISCUSSION
76
78
80
82
84
86
8 12 16 20
Recall
of
Water
Patch Size
RECALL OF WATER VS PATCH SIZE
Baseline
CNNWQC
CNNCWC
SAPCNN
DenseNet
AttResNet
Quantitative Analysis: All Models
Performance of neural networks for selected
S2 channels integrated with DEM

31. 88
90
92
8 12 16 20
Test
Accuracy
Patch Size
TEST ACCURACY VS PATCH SIZE
AttDenseNet
DenseNet
RESULTS & DISCUSSION
80
82
84
86
88
8 12 16 20
Recall
of
Water
Patch Size
RECALL OF WATER VS PATCH SIZE
AttDenseNet
DenseNet
Quantitative Analysis: AttDenseNet
Proposed Network (AttDenseNet) Vs DenseNet for
selected S2 channels integrated with DEM as input

32. 72
74
76
78
80
82
84
86
8 12 16 20
Test
Accuracy
Patch Size
TEST ACCURACY VS PATCH SIZE
Baseline
NDWI
NDVI
NDVI_NDWI
RESULTS & DISCUSSION
Quantitative Analysis: Comparison with Indices
Performance comparison of Baseline for only RGB
channels as input with index-based approaches
48
56
64
72
8 12 16 20
Recall
of
Water
Patch Size
RECALL OF WATER VS PATCH SIZE
Baseline
NDWI
NDVI
NDVI_NDWI
*EWI done only for qualitative analysis
due to memory issues

33. DIAGNOSTIC ABILITY: ROC & PR PLOTS
Baseline
CNNWQC SAPCNN
CNNCWC

34. DIAGNOSTIC ABILITY: ROC & PR PLOTS
AttResNet
DenseNet AttDenseNet
ROC & PR Values

35. QUALITATIVE ANALYSIS

36. LIMITATIONS
Hardware
Memory
• Step = 8
• No PCs for EWI
• Required
Reduction Factor
Spatial
Resolution
• Maximum 10m
• High Resolution
preferable
Image-Label
Compatibility
• Image from 2020
• Label from 2015
Model’s
Configuration
• Same for all
models
• No superpixels
for SAPCNN
• AttResNet was
affected much

37. RECOMMENDATION
Further Enhancement of Proposed Network
Introduction of Temporal Component
Original configuration of Models would produce better results

38. CONCLUSION
A novel CNN-based approach proposed for water extraction
problems
Neural Networks better than traditional approaches
State-of-the-art approaches successfully implemented in
satellite imageries

39. F. Pu, C. Ding, Z. Chao, Y. Yu, and X. Xu. “Water-quality classification of inland lakes using landsat8 images by
convolutional neural networks. ”In: Remote Sensing11.14 (2019), p. 1674
F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, and X.Tang. “Residual Attention Network for Image
Classification.” In: (Apr.2017).url: http://arxiv.org/abs/1704.06904
G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger. “Densely Connected Convolutional Networks.” In:2017
IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Vol. 2017-January. IEEE,2017, pp. 2261–
2269.isbn: 978-1-5386-0457-1.doi: 10.1109/CVPR.2017 . 243. arXiv: 1608 . 06993.
J. Yang and X. Du. “An enhanced water index in extracting water bodies from Landsat TM imagery.” In: Annals of
GIS23.3 (2017), pp. 141–148.issn: 19475691.doi: 10.1080/19475683.2017.1340339.url: https://doi.org/10.1080/194
75 683 .2017.1340339
M. Rezaee, M. Mahdianpari, Y. Zhang, and B. Salehi. “Deep convolutional neural network for complex wetland
classification using optical remote sensing imagery.” In: IEEE Journal of Selected Topics in Applied Earth
Observations and Remote Sensing11.9 (2018), pp. 3030–3039
Y. Chen, R. Fan, X. Yang, J. Wang, and A. Latif. “Extraction of urban water bodies from high-resolution remote-
sensing imagery using deep
REFERENCES

40. THANK
YOU!!!

41. EXTRACTING SURFACE WATER BODIES FROM SENTINEL-2 IMAGERY USING
CONVOLUTIONAL NEURAL NETWORKS
Janak Parajuli
al393628@uji.es
janak.parajuli1@gmail.com
05-03-2021
www.mastergeotech.info
Doubts / Comments /Sugestions

42. EXTRA SLIDES

43. NOVELTY & CONTRIBUTION
Novel approach for feature extraction in the study area
Implementation of DenseNet & AttResNet to extract water
Proposition of a novel approach in CNN

44. Search &
Encode Labels
Load Single
Label
Rename to
Encoded Labels
Balance Labels
PATCH EXTRACTION
Extract Safe
Positions
Extract
Labels
Stack Labels
Save Labels
Load Image
Tile
Adjust
Radiometry
Extract
Patches
Stack
Patches
Save
Patches
Label
Dataset
Label Balancing
Label Extraction
Patch Extraction
Legend

45. RESULTS & DISCUSSION: Neural Networks
B. Qualitative Analysis: Water Pixels Extraction
Scale: RGB 1:200000 Rest at 1:50000

46. RESULTS & DISCUSSION: Neural Networks
B. Qualitative Analysis: Urban Pixels Suppression
Scale: RGB at 1:100000 Rest at 1:15000

47. RESULTS & DISCUSSION: Indices
B. Qualitative Analysis
Water Pixels Extraction (Scale at 1:50000) Urban Pixels Suppression (Scale at 1:25000)

48. RESULTS & DISCUSSION: AttDenseNet
B. Qualitative Analysis
Scale: RGB at 1:200000 Rest at 1:50000
a) Tile T44RQR b) Tile T44RNS

49. Channels
Models
Patch Size
8 12 16 20
F1-Score Precision F1-Score Precision F1-Score Precision F1-Score Precision
W N W N W N W N W N W N W N W N
RGB
Baseline 69 86 75 83 73 88 79 85 75 88 79 86 76 89 79 87
CNNWQC 72 87 74 85 76 89 79 87 77 89 78 88 77 89 79 87
CNNCWC No convergence 71 87 77 84 73 88 78 85 74 88 77 86
SAPCNN 72 87 75 85 76 88 77 87 77 89 79 88 78 89 79 89
denseNet 73 87 77 86 77 89 78 88 78 89 80 89 78 90 81 88
attResNet 69 85 71 84 NA 73 87 75 86 NA
PRECISION & F1-SCORES OF EXPERIMENTS
Selected
S2
Bands
Baseline 80 90 82 89 81 91 83 90 82 91 84 91 82 91 84 91
CNNWQC 82 91 82 91 83 92 83 91 84 92 84 92 84 92 84 92
CNNCWC 77 90 81 87 77 90 82 87 81 91 83 90 81 91 82 91
SAPCNN 81 91 82 90 82 91 83 91 84 92 85 92 84 92 84 92
denseNet 82 91 83 90 83 92 84 91 85 92 85 92 85 91 86 91
attResNet 79 90 81 89 NA 81 90 81 90 NA

50. Channels
Models
Patch Size
8 12 16 20
F1-Score Precision F1-Score Precision F1-Score Precision F1-Score Precision
W N W N W N W N W N W N W N W N
RGB
with
DEM
Baseline 72 88 80 84 75 89 80 86 78 89 80 88 78 90 81 88
PRECISION & F1-SCORES OF EXPERIMENTS
Selected
S2
Bands
with
DEM
Baseline 81 91 83 90 82 91 84 90 83 92 84 91 84 92 85 92
CNNWQC 83 92 83 91 84 92 84 92 84 92 84 92 85 93 85 93
CNNCWC 79 90 82 89 81 91 83 90 82 91 84 90 83 91 83 91
SAPCNN 82 91 83 91 83 92 84 91 84 92 85 91 85 92 85 93
denseNet 83 91 84 91 84 92 84 92 85 92 85 92 86 93 86 93
attResNet 81 90 82 90 NA 81 91 83 90 NA

51. INITIAL EXPERIMENTS
Patch Size Vs Accuracy & Recall Ratio Factor Vs Accuracy & Recall
Step Vs Accuracy & Recall Learning Rate Vs Accuracy & Recall

52. SPECTRAL GRAPHS for EWI