This document discusses image search and analysis techniques for remote sensing data. It describes an index management system that takes in data and indexes it using column-based databases. Images are analyzed to extract features that allow for image search based on compression in compressed streams. Queries can be performed on the indexed data to return similar images based on semantic labels and normalized distances from queries. Examples are provided using different remote sensing datasets, including GeoEye, DigitalGlobe, and TerraSAR-X images.

1. Data structures &
representations
mquartulli@vicomtech.org

2. remote sensing data processing architectures
Job Queue
Analysis Workers
Data
Catalogue
Processing Workers
Auto Scaling
Ingestion
Data
Catalogue
Exploitation
Annotations
Catalogue
User Application Servers
Load
Balancer
User
Source Products
Domain
Expert
Configuration
Admin
Domain
Expert
direct data import
Data Processing / Data Intelligence
Servers
13/34

3. Hadoop Cluster computing
[Lisa Vaas 2016]

4. Spark cluster computing
[Hitesh Dharmdasani, “Python and Bigdata - An Introduction to Spark (PySpark)”]

5. spark + mongodb

6. The log data structure
• An append-only ordered sequence
of records.
• In DBs: log shipping protocols to
transmit portions of log to slave
replica databases
• In distributed systems: the State
Machine Replication Principle: If two
identical, deterministic processes
begin in the same state and get the
same inputs in the same order, they
will produce the same output and
end in the same state.
https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying

7. Why database management systems
• An interface between the database and
application programs, ensuring that data
is consistently organized and remains
easily accessible.
• Manages: 1. the data 2. the engine that
allows data to be accessed, locked and
modified 3. the database schema, which
defines the database’s logical structure.

8. Why database management systems
• Provides:
• Data abstraction and independence; Data security; A locking mechanism for concurrent access;
• An efficient handler to balance the needs of multiple applications using the same data;
• The ability to swiftly recover from crashes and errors, including restartability and recoverability;
• Robust data integrity capabilities; Logging and auditing of activity;
• Simple access using a standard application programming interface (API);
• Uniform administration procedures for data.
• Question:
• Does your application need all this? Typically yes if concurrent insert/update accesses, distributed
reads…

9. History
• 1960s: Navigational DBs
• 1970s-1980s: SQL, normalisation and OLTP, transactions
• 1990s: object oriented DBs and OLAP, warehousing
• 2000s: NoSQL and the CAP theorem
• 2010s: newSQL, graph DBs, Big Data

10. Databases: a user’s view
• SQL and ACID: Atomicity, Consistency, Isolation, and Durability
• Variety and NoSQL: MongoDB
• Volume and NoSQL: HBase, Cassandra
• Velocity and NoSQL: MonetDB, KairosDB
• NoSQL: bad at relationships —> Graph DBs: Neo4J
• The CAP theorem: Consistency, Availability, Partition tolerance

11. SQL vs NoSQL
[Lisa Vaas 2016]

12. DB indices
• Objective: sub-linear search (e.g. O(logN), O(1))
• E.g. bitmaps, keys/pointers to records, keys/pointers to blocks,
reverse indices.
• Implementations in terms of (balanced) trees, hashes, B+trees.
• Types:
• non-clustered: logical order only, multiple indices possible
• clustered: physical order too for efficiency,
a single clustered index per table.

13. KD Trees
• Binary space-partitioning trees in D dimensions
• For every non-leaf node: generate a splitting hyperplane that divides
the space into two half-spaces.
• Canonical construction: given all input points
• Cycles through the axes
• Split by the medians with respect to the current axis
source: wikipedia

14. LSH
• Locality-Sensitive Hashing: maximize probability of collision —
similar items end up in same bucket.
• Applications in near-duplicate detection and “fingerprinting”,
similarity nearest neighbor search, hierarchical clustering.
• E.g. by random projection: use a random hyperplane to hash
vectors.

15. Inverted indices
• Forward index: document —> content
• Inverted index: content —> document / location
• Record level, “Word” level
• Allows fast search (increases insertion cost!): queries can be
resolved by jumping to the “word” id in the inverted index.

16. apache arrow
import feather
path = 'my_data.feather'
feather.write_dataframe(df, path)
df = feather.read_dataframe(path)

17. Google Earth Engine
• A Short Intro by Kersten Clauss…
• Question: how would you replicate what’s under the hood?

18. 18
data acquisition & management
GeoEuskadi satellite image services
• linked open data management infrastructure
• public sources: Landsat y Copernicus Sentinel 1-2

19. 19
data processing
map update by
distributed analysis
of 25cm-
regional-scale
ortho-imagery
Lozano Silva, J.; Aginako Bengoa, N.; Quartulli, M.; Olaizola, I.G.; Zulueta, E., "Web-Based
Supervised Thematic Mapping," in Selected Topics in Applied Earth Observations and
Remote Sensing, IEEE Journal of , vol.8, no.5, pp.2165-2176, May 2015

20. 20
data processing – example video

21. image search: iqcbm
Index management system
Data input
Column-based DB
User interface
Analysis & processing
Image
analysis
• with DLR IMF BW
• search by compression
in compressed streams
• dynamic taxonomies
• corel 10k

22. iqcbm: geoeye
semantic label and these labels are stored into the database as part of the patch
information.
In the following, the results of different queries using the CBIR-FCD and TerraSAR-X
images are described.
TELEIOS FP7-257662
7. Applicable and Reference Documents
7.1. Applicable Documents
Document Title
Internal
Referenc
e
Katrin Molch et al., (2010). “Naming Convention for Image Patches”. [RI- 1]
N. Ritter and M. Ruth, (1995) “GeoTIFF Format Specification GeoTIFF”,
Revision 1.0, Version 1.8.1.
[RI- 2]
Robert M Haralick, K Shanmugam, Its'hak Dinstein (1973). "Textural Features
for Image Classification". IEEE Transactions on Systems, Man, and
Cybernetics, SMC-3 (6): 610–621.
[RI- 3]
The GLCM Tutorial
http://www.fp.ucalgary.ca/mhallbey/tutorial.htm
[RI-4]
H.G. Feichtinger, Th. Strohmer (1998). "Gabor Analysis and Algorithms",
Birkhäuser.
[RI-5]
B.S. Manjunath, W.Y. Ma (1996). “Texture features for browsing and retrieval
of image data”. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 18 (8): 837–842.
[RI-6]
A. Popescu, I. Gavat, M. Datcu (2008). "Complex SAR image characterization
using space variant spectral analysis". IEEE Radar Conference, 1-4.
[RI-7]
T. Li, M. Ogihara (2006). “Towards Intelligent Music Information Retrieval”.
IEEE Transactions on Multimedia, 8(3), 564-574.
[RI-8]
A. Croisier, D. Esteban, C. Galand (1976). "Perfect channel splitting by use of
interpolation / decimation / tree decomposition techniques". International Conf.
on information Science and Systems, Patras, Greece: 443-446.
[RI-9]
P.P Vaidyananthan (1987). “Quadrature Mirror Filter Banks, M-Band
Extensions and Perfect-Reconstruction Techniques”. IEEE ASSP Magazine, 4
(3): 4-20.
[RI-10]

23. 23
iqcbm: digitalglobe
TELEIOS FP7-257662
7. Applicable and Reference Documents
7.1. Applicable Documents
Document Title
Internal
Referenc
e
Katrin Molch et al., (2010). “Naming Convention for Image Patches”. [RI- 1]
N. Ritter and M. Ruth, (1995) “GeoTIFF Format Specification GeoTIFF”,
Revision 1.0, Version 1.8.1.
[RI- 2]
Robert M Haralick, K Shanmugam, Its'hak Dinstein (1973). "Textural Features
for Image Classification". IEEE Transactions on Systems, Man, and
Cybernetics, SMC-3 (6): 610–621.
[RI- 3]
The GLCM Tutorial
http://www.fp.ucalgary.ca/mhallbey/tutorial.htm
[RI-4]
H.G. Feichtinger, Th. Strohmer (1998). "Gabor Analysis and Algorithms",
Birkhäuser.
[RI-5]
B.S. Manjunath, W.Y. Ma (1996). “Texture features for browsing and retrieval
of image data”. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 18 (8): 837–842.
[RI-6]
A. Popescu, I. Gavat, M. Datcu (2008). "Complex SAR image characterization
using space variant spectral analysis". IEEE Radar Conference, 1-4.
[RI-7]
T. Li, M. Ogihara (2006). “Towards Intelligent Music Information Retrieval”.
IEEE Transactions on Multimedia, 8(3), 564-574.
[RI-8]
A. Croisier, D. Esteban, C. Galand (1976). "Perfect channel splitting by use of
interpolation / decimation / tree decomposition techniques". International Conf.
on information Science and Systems, Patras, Greece: 443-446.
[RI-9]
P.P Vaidyananthan (1987). “Quadrature Mirror Filter Banks, M-Band
Extensions and Perfect-Reconstruction Techniques”. IEEE ASSP Magazine, 4
(3): 4-20.
[RI-10]

24. 24
.76
.69
.79
.90
.95
.99
TELEIOS FP7-257662
7. Applicable and Reference Documents
7.1. Applicable Documents
Document Title
Internal
Referenc
e
Katrin Molch et al., (2010). “Naming Convention for Image Patches”. [RI- 1]
N. Ritter and M. Ruth, (1995) “GeoTIFF Format Specification GeoTIFF”,
Revision 1.0, Version 1.8.1.
[RI- 2]
Robert M Haralick, K Shanmugam, Its'hak Dinstein (1973). "Textural Features
for Image Classification". IEEE Transactions on Systems, Man, and
Cybernetics, SMC-3 (6): 610–621.
[RI- 3]
The GLCM Tutorial
http://www.fp.ucalgary.ca/mhallbey/tutorial.htm
[RI-4]
H.G. Feichtinger, Th. Strohmer (1998). "Gabor Analysis and Algorithms",
Birkhäuser.
[RI-5]
B.S. Manjunath, W.Y. Ma (1996). “Texture features for browsing and retrieval
of image data”. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 18 (8): 837–842.
[RI-6]
A. Popescu, I. Gavat, M. Datcu (2008). "Complex SAR image characterization
using space variant spectral analysis". IEEE Radar Conference, 1-4.
[RI-7]
T. Li, M. Ogihara (2006). “Towards Intelligent Music Information Retrieval”.
IEEE Transactions on Multimedia, 8(3), 564-574.
[RI-8]
A. Croisier, D. Esteban, C. Galand (1976). "Perfect channel splitting by use of
interpolation / decimation / tree decomposition techniques". International Conf.
on information Science and Systems, Patras, Greece: 443-446.
[RI-9]
P.P Vaidyananthan (1987). “Quadrature Mirror Filter Banks, M-Band
Extensions and Perfect-Reconstruction Techniques”. IEEE ASSP Magazine, 4
(3): 4-20.
[RI-10]
normalized distance from query

25. 25
iqcbm: terrasar-x
TELEIOS FP7-257662
7. Applicable and Reference Documents
7.1. Applicable Documents
Document Title
Internal
Referenc
e
Katrin Molch et al., (2010). “Naming Convention for Image Patches”. [RI- 1]
N. Ritter and M. Ruth, (1995) “GeoTIFF Format Specification GeoTIFF”,
Revision 1.0, Version 1.8.1.
[RI- 2]
Robert M Haralick, K Shanmugam, Its'hak Dinstein (1973). "Textural Features
for Image Classification". IEEE Transactions on Systems, Man, and
Cybernetics, SMC-3 (6): 610–621.
[RI- 3]
The GLCM Tutorial
http://www.fp.ucalgary.ca/mhallbey/tutorial.htm
[RI-4]
H.G. Feichtinger, Th. Strohmer (1998). "Gabor Analysis and Algorithms",
Birkhäuser.
[RI-5]
B.S. Manjunath, W.Y. Ma (1996). “Texture features for browsing and retrieval
of image data”. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 18 (8): 837–842.
[RI-6]
A. Popescu, I. Gavat, M. Datcu (2008). "Complex SAR image characterization
using space variant spectral analysis". IEEE Radar Conference, 1-4.
[RI-7]
T. Li, M. Ogihara (2006). “Towards Intelligent Music Information Retrieval”.
IEEE Transactions on Multimedia, 8(3), 564-574.
[RI-8]
A. Croisier, D. Esteban, C. Galand (1976). "Perfect channel splitting by use of
interpolation / decimation / tree decomposition techniques". International Conf.
on information Science and Systems, Patras, Greece: 443-446.
[RI-9]
P.P Vaidyananthan (1987). “Quadrature Mirror Filter Banks, M-Band
Extensions and Perfect-Reconstruction Techniques”. IEEE ASSP Magazine, 4
(3): 4-20.
[RI-10]

26. 26
iqcbm: terrasar-x
TELEIOS FP7-257662
7. Applicable and Reference Documents
7.1. Applicable Documents
Document Title
Internal
Referenc
e
Katrin Molch et al., (2010). “Naming Convention for Image Patches”. [RI- 1]
N. Ritter and M. Ruth, (1995) “GeoTIFF Format Specification GeoTIFF”,
Revision 1.0, Version 1.8.1.
[RI- 2]
Robert M Haralick, K Shanmugam, Its'hak Dinstein (1973). "Textural Features
for Image Classification". IEEE Transactions on Systems, Man, and
Cybernetics, SMC-3 (6): 610–621.
[RI- 3]
The GLCM Tutorial
http://www.fp.ucalgary.ca/mhallbey/tutorial.htm
[RI-4]
H.G. Feichtinger, Th. Strohmer (1998). "Gabor Analysis and Algorithms",
Birkhäuser.
[RI-5]
B.S. Manjunath, W.Y. Ma (1996). “Texture features for browsing and retrieval
of image data”. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 18 (8): 837–842.
[RI-6]
A. Popescu, I. Gavat, M. Datcu (2008). "Complex SAR image characterization
using space variant spectral analysis". IEEE Radar Conference, 1-4.
[RI-7]
T. Li, M. Ogihara (2006). “Towards Intelligent Music Information Retrieval”.
IEEE Transactions on Multimedia, 8(3), 564-574.
[RI-8]
A. Croisier, D. Esteban, C. Galand (1976). "Perfect channel splitting by use of
interpolation / decimation / tree decomposition techniques". International Conf.
on information Science and Systems, Patras, Greece: 443-446.
[RI-9]
P.P Vaidyananthan (1987). “Quadrature Mirror Filter Banks, M-Band
Extensions and Perfect-Reconstruction Techniques”. IEEE ASSP Magazine, 4
(3): 4-20.
[RI-10]

27. 27
experiment: tsx

28. iqcbm: terrasar-x
D3.1 KDD concepts and methods proposal: report & design recommendations 96
The patches were annotated with a semantic label by using the Search Engine based on
SVM tool and user supervision previously presented in section 5.1. The semantic labels
associated to the selected classes were previously described in Table 5.
In the following, we present some examples of retrieving TerraSAR-X structures using
both images. Table 11 displays the query images and the 20 top retrieved images. Some
quality metrics (Precision and Recall) were computed from these results and they are
summarized in Table 12.
Query
images
Retrieved images
Class9
Class6
Class7
Class36
TELEIOS FP7-257662
Class20
Class31
Class28
Class32
Table 11: Results of the queries based on image content using CBIR-FCD as data
mining tool.
Table 12 shows the precision and recall for the classes and the query time in seconds
needed for searching and retrieving the results.
Table 12: Precision and recall of the semantic classes using query based on content
and the query time.
Class Precision
(%)
Recall
(%)
Query time
(sec)
Class1 5,36 5,17 0.32882
Class2 10,71 10,34 0.318238
Class3 5,36 5,17 0.235323
Class4 7,14 6,90 0.107209
TELEIOS FP7-257662
7. Applicable and Reference Documents
7.1. Applicable Documents
Document Title
Internal
Referenc
e
Katrin Molch et al., (2010). “Naming Convention for Image Patches”. [RI- 1]
N. Ritter and M. Ruth, (1995) “GeoTIFF Format Specification GeoTIFF”,
Revision 1.0, Version 1.8.1.
[RI- 2]
Robert M Haralick, K Shanmugam, Its'hak Dinstein (1973). "Textural Features
for Image Classification". IEEE Transactions on Systems, Man, and
Cybernetics, SMC-3 (6): 610–621.
[RI- 3]
The GLCM Tutorial
http://www.fp.ucalgary.ca/mhallbey/tutorial.htm
[RI-4]
H.G. Feichtinger, Th. Strohmer (1998). "Gabor Analysis and Algorithms",
Birkhäuser.
[RI-5]
B.S. Manjunath, W.Y. Ma (1996). “Texture features for browsing and retrieval
of image data”. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 18 (8): 837–842.
[RI-6]
A. Popescu, I. Gavat, M. Datcu (2008). "Complex SAR image characterization
using space variant spectral analysis". IEEE Radar Conference, 1-4.
[RI-7]
T. Li, M. Ogihara (2006). “Towards Intelligent Music Information Retrieval”.
IEEE Transactions on Multimedia, 8(3), 564-574.
[RI-8]
A. Croisier, D. Esteban, C. Galand (1976). "Perfect channel splitting by use of
interpolation / decimation / tree decomposition techniques". International Conf.
on information Science and Systems, Patras, Greece: 443-446.
[RI-9]
P.P Vaidyananthan (1987). “Quadrature Mirror Filter Banks, M-Band
Extensions and Perfect-Reconstruction Techniques”. IEEE ASSP Magazine, 4
(3): 4-20.
[RI-10]
TELEIOS KDD concepts and methods proposal: report & design recommendations
Corneliu Octavian Dumitru, Daniela Espinoza Molina, Shiyong Cui, Jagmal Singh, Marco Quartulli, Mihai Datcu
2011, FP7 TELEIOS Tech Report

29. 29
interactive web-based data retrieval
web-based
interactive
classification
of image content