Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
What to Upload to SlideShare
Loading in …3
×
1 of 20

CARI-2020, Application of LSTM architectures for next frame forecasting in Sentinel-1 images time series

0

Share

Download to read offline

Waytehad Rose Moskolaïa,b , Wahabou Abdoua , Albert Dipandaa , Kolyangb

Related Books

Free with a 30 day trial from Scribd

See all

CARI-2020, Application of LSTM architectures for next frame forecasting in Sentinel-1 images time series

  1. 1. African Conference on Research in Computer Science and Applied Mathematics CARI’2020 – Polytech School of Thiès, Senegal October 2020 Application of LSTM architectures for next frame forecasting in Sentinel-1 images time series Waytehad Rose Moskolaïa,b , Wahabou Abdoua , Albert Dipandaa , Kolyangb a Computer Science Department, University of Burgundy, 21078 DIJON Cedex, France. b Computer Science Department, University of Maroua, P.O. Box 46 MAROUA, Cameroon.
  2. 2. OUTLINE 2 • Introduction • Methodology • Results and Discussions • Conclusion and Perspectives
  3. 3. • Definition :  Part of data mining that allows estimating future trends of events.  Create predictions about unknown future events. Used in several activity sectors: Predictive Analytics 3 Historical Data Predictive Algorithms Model New Data Model Predictions Introduction sales Bank Weather Health Energy Agriculture Earth observation
  4. 4. Technologies used 4 • For models using remote sensing data, classical Machine Learning algorithms are generally used: Random Forest, SVM, Regression, Neural Networks, etc. • But some limits exist  The necessity to first extract the features or linearize data  The use of auxiliary data  performance is often subject to many physical assumptions • Recent works use more efficient technologies to achieve better results: Deep Learning (Jason Brownlee, 2018) Introduction Classical Learning Performance Amount of Training data Deep Learning
  5. 5. Deep Learning architectures 5 • Deep Learning (DL) :  Is a part of artificial intelligence and Machine Learning  Mimics the workings of the human brain  Allows computers to learn by themselves from examples… • Several DL architectures are used for prediction in time series:  RNN: Recurrent Neural Networks, namely the Long Short-Term Memory (LSTM) (Sepp Hochreiter and J. Schmidhuber, 1995)  CNN : Convolutional Neural Network, suitable for images  ConvLSTM : Fusion of CNN architecture and LSTM architecture (SHI Xingjian et al., 2015)  CNN-LSTM : Combination of CNN architecture and LSTM architecture (CNN + LSTM), (Z. Shen et al., 2019)… Introduction
  6. 6. Research question 6 • In general, determining which algorithms are best for a problem is the key to getting the most out of a predictive analytics solution. • Main research question : which architecture is the most suitable for prediction tasks in satellite images time series ? • Proposed approach : the implementation and comparative study of three architectures widely used for prediction (ConvLSTM, Stack- LSTM and CNN-LSTM), in the context of next occurrence prediction in a given satellite images time series. Introduction
  7. 7. Objectives 7 Let 𝑋_𝑡 be a function of ℝ × ℝ of size (W, H) representing an image at time t. Given a sequence of images 𝑋_(𝑡−𝑛), 𝑋_(𝑡−𝑛−1), … 𝑋_𝑡, the objectives of this work are :  The implementation of sequence-to-one models based on Stack-LSTM, ConvLSTM and CNN-LSTM architectures, for the prediction of the image at time t+1  Performance evaluation of each model time Predicted Image at time t+1 Images time series Methodology
  8. 8. Materials 8 • Used data: 158 sentinel-1 images (www.earth-explorer.usg.org), Wildlife Reserve of Togodo, from September 2016 to May 2019 • Development tools:  Virtual GPU, Google Colab (https://colab.research.google.com)  Python (Programming language)  Tensorflow and Keras libraries  Quantum GIS (for image preprocessing) Methodology
  9. 9. Data preparation 9 • Preprocessing : Radiometric and geometric corrections, Normalization, resizing, clipping, transformation to RGB files … • Constitution of training set (about 80%) and test set (20%) • Transformation of the training set into the format (samples, timestep, Wx, Hx, features) X_train Y_train 𝑋1, 𝑋2 , 𝑋3, 𝑋4, 𝑋5 [𝑋6] 𝑋2, 𝑋3 , 𝑋4, 𝑋5, 𝑋6 [𝑋7] 𝑋3, 𝑋4 , 𝑋5, 𝑋6, 𝑋7 [𝑋8] … 𝑋𝑡−5, 𝑋𝑡−4 , … , 𝑋𝑡−1 [𝑋𝑡] Timestep : Number of occurrence in input sequence Samples : Batch size for training step (t – timestep) Features : Number of variable to predict (1) Wx, Hx : Size of images (64X64 and 128X128) Methodology
  10. 10. Structure of models 10 Layering arrangement ConvLSTM layers Stack-LSTM layers CNN-LSTM layers • ConvLSTM2D() • BatchNormalization() • Dropout() • ConvLSTM2D() • BatchNormalization() • Dropout() • ConvLSTM2D() • BatchNormalization() • Dropout() • Dense() • Flatten() • LSTM() • LSTM() • LSTM() • Dense() • Reshape() • Dense() • Conv2D() • MaxPooling2D() • Flatten() • LSTM() • LSTM() • Dense () • Reshape() • Dense() Methodology
  11. 11. Training parameters 11 • Optimization function: Adaptive Moment Optimization (adam) • Loss function : Root Mean-Square-Error (RMSE) and Mean Absolute Error (MAE)) • Training steps : 100 epochs (the number of times that the dataset passes through the neural network) Methodology
  12. 12. Evaluation parameters 12 • Evolution of loss during the training step • Total training time • Values of Structural SIMilarity (SSIM) between the predicted image and the real one Activity Recognition: Generating a textual Methodology
  13. 13. Graphical display 13 Prediction results based on ConvLSTM, Stack-LSTM and CNN- LSTM (respectively (a), (b), (c)). Timestep = 5, with (64×64) images from test set. Results and Discussions
  14. 14. Graphical display 14 Prediction results based on ConvLSTM, Stack-LSTM and CNN-LSTM (respectively (a), (b), (c)). Timestep = 10, with (64×64) images from test set. Results and Discussions
  15. 15. Evolution of training Loss 15 Evolution of training loss (MAE) over epochs depending on timestep (vary from 5 to 10). (a) Left: training loss with ConvLSTM, right: Training loss with CNN-LSTM. (b) Training loss with Stack-LSTM Results and Discussions
  16. 16. Evolution of training loss 16 Evolution of training loss values over epochs. (a) Left: MAE with (128×128) images, Right: MAE with (64×64) images. (b) Left: RMSE with (128×128) images, Right: RMSE with (64×64) images. Results and Discussions
  17. 17. Evaluation criteria 17 • Due to convolutions operations, time processing is significantly higher with ConvLSTM model than CNN-LSTM and Stack-LSTM when the resolution of images increases. Results and Discussions
  18. 18. Conclusion 18 • The use of ConvLSTM architecture for the forecasting tasks from earth observation images time series is not advisable (size of images, length of sequences) • The use of CNN-LSTM architecture is recommended • Predictions with Stack-LSTM models are done pixel by pixel • In all situations it is necessary to choose good parameters to achieve better results (optimization). Conclusion and Perspectives
  19. 19. Perspectives 19 • What next?  Optimize model based on CNN-LSTM architecture to improve accuracy  Use more date  Test on others area  Create a model based on different architectures to achieve better results. Conclusion and Perspectives
  20. 20. 20 THANK YOU FOR YOUR ATTENTION QUESTIONS?

×