A brief overview of CV applications in Geosciences. Dubai'19 DataFest presentation

1. 1
"We know more about the surface of the Moon and
about Mars than we do about [the deep sea floor],
despite the fact that we have yet to extract a gram of
food, a breath of oxygen or a drop of water from those
bodies."
Paul Snelgrove, Oceanographer
Computer vision
in geosciences

2. MSU MARINE RESEARCH
CENTER
Leading marine data collector and
analysis center in Russia
CLIMATE4MEDIA
Estonian climate monitoring
agency
ATLANTNIRO
Leading fishing industry research
center in Russia
GEBCO 2030
Leading bathymetric analysis
organization
NVIDIA CORP
Leading GPU tech corporation
UAE UNIVERSITY POISK
Russian underwater
search&recovery association
AMAZON WEB SERVICES
Cloud provider
Partnerships
18

3. 3
Computer vision application domains

4. Remote sensing (satellite
and UAV) imagery analytics
Automatic object detection, segmentation and
analytics

5. Remote sensing (satellite and UAV) imagery analytics
A combination of remote sensing
data (radar and optics) and UAVs
for ice detection and
segmentation, distinguishing and
characterization of ice formations,
development of conditions
forecast taking into account
hydrometeorological data
Environmental
applications — analytics
of oxygen, chlorophyll,
algae concentrations,
monitoring of bird
colonies, rookeries and
ice leads used for
feeding by mammals.
Litter detection
Shipping applications — optimal
routing of shipping caravans in ice
conditions, monitoring of port
infrastructure, loading and
unloading, analysis of shipping
intensity, volumes of cargo handling
in ports, leaks detection from
ships / platforms
Hydrometeorological
applications — currents,
cyclones, winds
detection and analytics
5

6. Use case example
Optimalroute
DANGER
WARNING
Arctic ice has always been a great danger for shipping
operations up in the North.
However, Nordic countries are increasing number of
shipping routes in the northern seas, mainly for natural
gas transportation purposes.
6
In the same time, ice breakers fleet has limited
capacity, thus making shipping operations costly, as
each convoy route planning takes a lot of time and effort
in order to avoid dangerous zones with high density ice
and ice ridges.
We have developed a demo case for our partner -
Marine Research Center in order to reduce time of ice
situation analysis to provide faster route planning and use
ice breakers fleet more effectively.

7. 7
Results UAV
• We used “one-vs-all” strategy based on
FPN with ResNet18 backbone
• Avg Jaccard - 0.61
• Inference time for 1 image < 1 min on
CPU

8. 8
Results UAV
● Problems with classification of complex areas where all classes are present (southern and
south-eastern margins of stamukha)

9. 9
Results UAV

10. 10
Results UAV

11. 11
Results UAV

12. 12
Results UAV

13. 13
Results UAV

14. 14
Results Satellite
• FPN + ResNet101
• OVA
• Average Jaccard = 0,4

15. 15
Results satellite
For training and testing we used image
S1_EW_HH_sub_20191006T072046
As a testing area we used the part of the
image outlined with red box

16. 16
Results satellite
The following peculiarities of the image were outlined:
Image includes several regions (terms according to WMO sea ice
classification)
1.Area with old ice
2.Transit area between old ice area and open water, mainly with new ice,
ice flows here are lesser in dimensions than image resolution – as a result,
homogenous appearance, ice flows are moved by local eddies
3.Possible thin cover of very young ice with extremely low radar
backscatter
4.Open water with relatively rough surface
5.Water with rough surface
6.Possibly hardly distinguishable mesh of new ice and open water
Thus 6 classes were included into training:
1.Open water
2.Fractures in old ice area
3.Water with low concentration of new ice
4.Old ice
5.New ice
6.Young ice

17. 17
Results satellite
● Multiclass classification icewater on Sentinel-2 images

18. 18
Results satellite

19. Multi-beam and side-scan
sonars imagery analytics
Automatic object detection and classification on
sonar and bathymetry images, analytics
geo-data database creation

20. 20
Multi-beam and side-scan sonars imagery analytics
Inspection of underwater structures,
including detection of sagging, washout
areas, damage, hazardous natural and
man-made objects, leaks, etc.
Underwater geological, geomorphological
and landscape mapping — detection of
bottom sediment types, detection of gas
emanations, detection of microrelief features
and vulnerable habitats
Detection of man-made and
natural objects in offshore
survey, search and rescue,
underwater archeology, marine
litter detection
8

21. Use case example and results
Normally search, rescue & recovery (SRR) operations take
a lot of time and effort as operators have to search
manually the area of investigation, either constantly
looking at sonar screens for 12-14 hours in a row or
trying to identify missing ships on satellite or drones images
9
We aim to use both our knowledge in computer vision and
domain expertise to help SRR teams discover the aftermath
of an accident within smaller time frame and with greater
precision
Our technologies are already used by some of our
partners to find drowning victims in inland waters, both
lakes and rivers, decreasing time-to-discovery from days
to hours

22. 22
Anomalies detection
About 200 images from open sources were
collected as a dataset, additional datasets
were provided by partners.
More than 4000 images were obtained by
using generative algorithms for style
transfer, from satellite images to sonar
images.
The labelling was carried out on both
artificial and real images Real SSS image
Synthetic image

23. 23
Obstacles detection and size assessment
While survey teams analyze the seabed
before laying pipes or cables, or
construction an oil rig they need to asses the
location and the size of obstacles.
Normally, such operations are performed
manually, which requires days or weeks to
proceed.
Computer vision and basic geometry helps
to automate this process and reduce the
needed time to minutes.

24. Seabed classification is essential in biological and
geological survey.
During offshore operations and construction it is
necessary to minimize collateral damage to
endangered habitats which tend to live in different
kinds of bottom sediments.
Normally, seabed analysis is conducted manually,
when operator identifies “with eye and hand” different
seabed types on sonar imagery.
The solution is to implement automated semantic
segmentation
9
Seabed segmentation

25. 25
Mosaic of Sonograms NN classification Interpreter classification
Results
clay silt, marine - gray,
flowing clay, lacustrine - green,
flowing clays, lacustrine-glacial - blue,
fine sands, with interbeds of loam, with inclusions of
gravel and pebbles - red

26. Seismic data analysis
Automatic features detection, reflective horizons
tracing, seismic facies analysis, big data
analytics

27. Tracing of layers
and
seismic-acoustic
facies
Salt
structures,
buried
incisions
detection
Faults
detection
Bottom
simulated
reflectors and
shallow gas
detection
Seismic data analysis
Amplitude
anomalies
detection
14

28. Use case example and results
Several areas of Earth with large accumulations of oil
and gas also have huge deposits of salt below the
surface
But unfortunately, knowing where large salt deposits are
precisely is very difficult. Professional seismic imaging still
requires expert human interpretation of salt bodies. This
leads to very subjective, highly variable renderings.
More alarmingly, it leads to potentially dangerous situations
for oil and gas companies drillers
15
Neural networks based solution will help to get the
detection results in an automated way with more speed
and precision, diminishing manual work and improving
quality of the final results

29. Use case example and results
The basis of the seismic analysis is identification of
horizons, layers and velocities
While layers automated detection is there in the market for
nearly 30 years, the operations are still semi-manual and
the models used don’t render accurate and robust results
15
Neural networks based solution will help to get the
detection results in an automated way with more speed
and precision, diminishing manual work and improving
quality of the final results

30. Beyond Earth
Computer vision for cryo-polygons detection on
Mars

31. An idea
“Just when you thought ol' Curiosity was digging in for the
winter, the little discovery machine came up with a doozy:
It discovered water in Martian soil. NASA scientists just
published five papers in Science detailing the experiments
that led to the discovery. That's right. There's water on
Mars.”
12

32. Use case description
In terrestrial and Martian periglacial environments,
networks of more or less regular polygonally shaped
fracture patterns characterise the wide areas of the
ground. Figure 1 and 2 show the remarkable resemblance
of these polygonal patterns on Earth and on Mars.
Although the formation requires seasonally liquid water,
the Martian polygonal features are interpreted as thermal
contraction polygons or ice-wedge polygons, as well,
owing to their remarkable resemblance to the terrestrial
counterparts
12

33. Data
The PDS Geosciences Node Mars Orbital Data
Explorer (ODE) provides search, display, and download
tools for the PDS science data archives and other data
sets from the Mars Reconnaissance Orbiter (MRO), the
Mars Global Surveyor, and the European Space Agency's
Mars Express missions. ODE also includes selected
non-PDS data contributed by the science community to
support landing site selection.
12

34. Use case solution
So the use case is just to perform a binary
segmentation and identify the areas with high
polygons concentration.
Next steps would include measuring the size
of the polygons, as there is a proven
correlation between size of a polygon and
deepness of permafrost.
12

35. 12

36. 12
Results

37. 12
Results

38. 12
Results

39. Contacts
Pavel Golubev
MaritimeAI
pavel.golubev@maritimeai.net
https://maritimeai.net/
39
Contacts
Pavel Golubev
MARITIME AI
pavel.golubev@maritimeai.net
https://maritimeai.net