Did you like it? Check out our blog to stay up to date: https://getindata.com/blog Satellite data is not well-known although it provides a lot useful information for different sectors like agriculture, industry or military.During the workshops participants will learn how to get the satellite imageries, process them by using Python and PySpark and what are the most common use cases. There will be a knowledge sharing session about issues with satellite data and how these difficulties can be overcome. Author: Albert Lewandowski Linkedin: https://www.linkedin.com/in/albert-lewandowski/ _____ Getindata is a company founded in 2014 by ex-Spotify data engineers. From day one our focus has been on Big Data projects. We bring together a group of best and most experienced experts in Poland, working with cloud and open-source Big Data technologies to help companies build scalable data architectures and implement advanced analytics over large data sets. Our experts have vast production experience in implementing Big Data projects for Polish as well as foreign companies including i.a. Spotify, Play, Truecaller, Kcell, Acast, Allegro, ING, Agora, Synerise, StepStone, iZettle and many others from the pharmaceutical, media, finance and FMCG industries. https://getindata.com​

1. Monitoring environment
based on satellite data with
Python and PySpark
Author: Albert Lewandowski

2. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● Big Data DevOps Engineer in GetInData
● Lead DevOps Engineer in EPC
● CTO in Skyverse
● Smart City Consultant in Almine
Who am I?
Trigger 1

3. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● Introduction to satellite data.
● Overview of available datasets.
● Available tools.
● Use cases in practise.
● Exercise I: calculating NDVI
● Exercise II: detect landslides
● Exercise III:
Agenda
Trigger 1

4. Satellite data

5. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● Satellite data or satellte imagery
● It is information about planets in the space that we get by
man-made satellites in their orbits.
● Satellite data is generated via remote sensing technologies:
passive or active.
What are satellite data?
Trigger 1

6. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● Firstly, analyze problem to solve and define what kind of
information we need.
● Secondly, read about each satellite data source.
● Thirdly, download it from web platform or by using scripts by
calling API.
● Fourthly, have fun in Notebook.
How can we get data?
Trigger 1

7. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Vector vs. Raster data
Trigger 1

8. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● There are multiple available data sources.
● Free data sources - great for learning and basic use cases,
useful for data enrichment process
● Paid data sources - it may be expensive but provides much
better quality of data
Types of satellite data
Trigger 1

9. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Sentinel Data
Trigger 1

10. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● Operating day and night
● Performing C-band synthetic aperture radar imaging
● Global revisit: 6 - 12 days
● Spatial resolution (depends on the mode):
○ 5 m x 20 m for IW mode
○ 20 m x 40 m for EW mode
Sentinel 1
Trigger 1

11. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● High-resolution optical images for land monitoring
● A set of 13 spectral bands: 4 visible (10 m spatial resolution),
6 near infrared (20 m) and 3 shortwave infrared (60 m)
● Sentinel-2A and Sentinel-2B have 5 days revisit time.
Sentinel 2
Trigger 1

12. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● It provides atmospheric measurements, relating to air
quality, climate forcing, ozone and UV radiation
● Spatial resolution is below 8km for wavelengths above
300nm and below 50km for wavelength below 300nm
Sentinel 5P
Trigger 1

13. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● https://modis.gsfc.nasa.gov/data/
● Its detectors measure 36 spectral bands between 0.405 and
14.385 µm, and it acquires data at three spatial resolutions
-- 250m, 500m, and 1,000m.
MODIS
Trigger 1

14. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● Launched in 1972
● Landsat 8: two sensors provide seasonal coverage of the
global landmass at a spatial resolution of 30 meters (visible,
NIR, SWIR); 100 meters (thermal); and 15 meters
(panchromatic).
Landsat
Trigger 1

15. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Landsat
Trigger 1

16. Tools for space journey

17. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● Data gathering
● Data processing
● What do I want to achieve by using satellite data?
Let’s start our journey
Trigger 1

18. Data gathering

19. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● There are multiple available data sources.
● Free data sources - great for learning and basic use cases,
useful for data enrichment process
● Paid data sources - it may be expensive but provides much
better quality of data
Data gathering
Trigger 1

20. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● ESA: Sentinel data: https://scihub.copernicus.eu/
● NASA: Landsat data: https://earthexplorer.usgs.gov/
● SentinelHub
○ https://apps.sentinel-hub.com/sentinel-playground/
○ https://apps.sentinel-hub.com/eo-browser/
Examples of free data platforms
Trigger 1

21. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● (Airbus) OneAtlas: https://oneatlas.airbus.com/
● Planet: https://www.planet.com/
● Iceye: https://www.iceye.com/satellite-data
● Google EarthEngine: https://earthengine.google.com/
Examples of paid data platforms
Trigger 1

22. Data processing

23. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● In production use cases, we need to analyze hundreds of
gigabytes of satellite images.
● Requirements for candidates:
○ Scalability
○ Reliability
○ Support for ML libraries
Big Data requires Big Processing Engine
Trigger 1

24. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
It is developed in coordination with other community projects like
Numpy, Pandas, and Scikit-Learn.
● Official website: https://dask.org/
● Dask uses existing Python APIs and data structures.
● Simple to learn if you know Python.
● Compatibility with multiple Python libraries and support for
scaling up and down
Python with Dask

25. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
At its core, the ODC is a set of Python libraries and PostgreSQL
database that helps you work with geospatial raster data.
● Official website: https://www.opendatacube.org/
● Designed for HPC for analyzing geographical data
● Requires Python 3.5+, PostgreSQL 9.6+ and GDAL library
OpenDataCube

26. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
It is a unified analytics engine for large-scale data processing.
● Official website: https://spark.apache.org/
● Well known with great community.
● Widely used in Big Data platforms.
● Support for multiple programming languages: Java, Scala,
Python, R, .NET
Apache Spark

27. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● Magellan
● GeoMesa
● Apache Sedona
● GeoTrellis
● RasterFrames
Apache Spark Family for Geo Analysis

28. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● RasterFrames builds upon
several other LocationTech
projects
● Support for all Python, Java,
SQL
● Support for vector and raster
data
RasterFrames

29. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● It is a Scala library and
framework that provides APIs
for reading, writing and
operating on geospatial raster
and vector data.
● Live demo:
https://geotrellis.io/
GeoTrellis

30. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Sedona extends Apache Spark /
SparkSQL with a set of
out-of-the-box Spatial Resilient
Distributed Datasets / SpatialSQL.
● Apache Incubating project
Apache Sedona

31. Use cases

32. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● Forestry planning
● Prevention of illegal logging
Forestry

33. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● Infrastructure planning
● Monitoring pipelines
● Infrastructure integrity monitoring
● Monitoring of building activity
Industry

34. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
CORINE Land Cover

35. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● Crop monitoring
● Farms planning
Agriculture

36. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
● Disaster response: monitoring natural catastrophes
● Coastal traffic monitoring
● Oil spills detection
Security

37. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Security

38. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Detect changes in greenery

39. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Detect changes in greenery

40. © Copyright. All rights reserved. Not to be reproduced without prior written consent.
Q&A