SlideShare a Scribd company logo
Jérôme Gasperi
WGISS #37

Cocoa Beach, Florida - USA - April 16th, 2014
Semantic search
Semantic search helps users to
find the right data
Semantic search helps users to
find the right data
How to add semantics capabilities to EO products search services ?
Characterize products with relevant information.
Think « users », not « experts »
1
Characterize products with relevant information.
Think « users », not « experts »
1
Decode users natural language queries
2
Use footprint to enrich metadata from exogenous data
1 Enrich products
github.com/jjrom/itag
!
Tag this footprint with continent, country and Land use
!
http://goo.gl/WtbcbR

iTag1 Enrich products
2 Decode queries
RESTo provides semantic search capabilities
It uses a Query Analyzer to translate query into a set of EO OpenSearch parameters
Query string analysis algorithm
is based on simple recognition
of words and patterns
Split query string
into list of unitary words
Extract «key=value» strings e.g. orbitNumber=4
Extract Platforms and
Instruments
Platforms and instruments list are stored
within common dictionary
!https://github.com/jjrom/resto/blob/
master/resto/dictionaries/common.php
Remove excluded words
and non dictionary words
with length < 4 characters
e.g. «area of Mexico in 2012»
Extract patterns and
dates
e.g. «acquired in the last 2 days»
Extract keywords e.g. «urban area in France»
Extract location on
remaining words
e.g. «images acquired in Toulouse»
2 Decode queries
Recognized patterns
<with> "keyword"	
<without> "keyword"	
!
"quantity" <lesser> (than) "numeric" "unit"	
"quantity" <greater> (than) "numeric" "unit"	
"quantity" <equal> (to) "numeric" "unit"	
<lesser> (than) "numeric" "unit" (of) "quantity"	
<greater> (than) "numeric" "unit" (of) "quantity"	
<equal> (to) "numeric" "unit" (of) "quantity"	
"quantity" <between> "numeric" <and> "numeric" ("unit")	
<between> "numeric" <and> "numeric" "unit" (of) "quantity"	
!
<today>	
<yesterday>	
<before> "date"	
<after> "date"	
<between> "date" <and> "date"	
"numeric" "(year|day|month)" <ago>	
<last> "(year|day|month)"	
<last> "numeric" "(year|day|month)"	
"numeric" <last> "(year|day|month)"	
"(year|day|month)" <last>	
<since> "numeric" "(year|day|month)"	
<since> "month" "year"	
<since> "date"	
<since> "numeric" <last> "(year|day|month)"	
<since> <last> "numeric" "(year|day|month)"	
<since> <last> "(year|day|month)"	
<since> "(year|day|month)" <last>	
2 Decode queries
$dictionary = array( 	
'excluded' => array(	
'than',	
'image',

...	
),	
'modifiers' => array(	
'ago' => 'ago',	
'before' => 'before',	
'after' => 'after',	
...	
),	
'units' => array(	
'm' => 'm',	
'meter' => 'm',

'days' => 'days',

...	
),	
'numbers' => array(	
'one' => '1',	
...	
),	
'months' => array(	
'january' => '01',	
...	
),	
'quantities' => array(	
'resolution' => 'resolution',	
...	
),	
'keywords' => array(	
'continent' => array(	
'europe' => 'europe',	
...	
)	
)	
Words are stored within a dictionary
2 Decode queries
« Images of urban area in the US acquired in the last 10 days with less than 5 % of cloud cover »
Example
2 Decode queries
2 Decode queries
« Images of urban area in the US acquired in the last 10 days with less than 5 % of cloud cover »
Example
keyword location date acquisition parameter
2. Each search result has an « human readable url » that can
be indexed by web crawler (i.e. google robots)
1. Search parameters are derived from
Natural Language query
3. Keywords on resources are links to search requests :
they can be indexed by web crawler…and so on
Search (example)
2. Each search result has an « human readable url » that can
be indexed by web crawler (i.e. google robots)
1. Search parameters are derived from
Natural Language query
3. Keywords on resources are links to search requests :
they can be indexed by web crawler…and so on
Search (example)
http://goo.gl/GvMEHj
Issues with keywords approach
Earthquakes in november 2008 in china
Earthquakes in november 2008 in china
Ambiguous since it appears to be
a location in New Zealand
« Linked data is the right way to do Semantic Web »
Tim Berners-Lee
Update RESTo JSON model to follow JSON-LD format
{
"@context": "http://json-ld.org/contexts/person.jsonld",
"@id": "http://dbpedia.org/resource/John_Lennon",
"name": "John Lennon",
"born": "1940-10-09",
"spouse": "http://dbpedia.org/resource/Cynthia_Lennon"
}

More Related Content

Viewers also liked

2016 vendor showcase track: an introduction to spike and the application of p...
2016 vendor showcase track: an introduction to spike and the application of p...2016 vendor showcase track: an introduction to spike and the application of p...
2016 vendor showcase track: an introduction to spike and the application of p...
GIS in the Rockies
 
2016 conservation track: under the hood of an rea: what is within a rapid ec...
2016 conservation track: under the hood of an rea:  what is within a rapid ec...2016 conservation track: under the hood of an rea:  what is within a rapid ec...
2016 conservation track: under the hood of an rea: what is within a rapid ec...
GIS in the Rockies
 
2016 asprs track: gis support for trail planning by jeff orlowski
2016 asprs track: gis support for trail planning by jeff orlowski2016 asprs track: gis support for trail planning by jeff orlowski
2016 asprs track: gis support for trail planning by jeff orlowski
GIS in the Rockies
 
2016 conservation track: evaluating lidar derived synthetic streams as a ...
2016 conservation track: evaluating   lidar derived   synthetic streams as a ...2016 conservation track: evaluating   lidar derived   synthetic streams as a ...
2016 conservation track: evaluating lidar derived synthetic streams as a ...
GIS in the Rockies
 
2016 conservation track: broad scale assessment, planning and management of ...
2016 conservation track:  broad scale assessment, planning and management of ...2016 conservation track:  broad scale assessment, planning and management of ...
2016 conservation track: broad scale assessment, planning and management of ...
GIS in the Rockies
 
2016 foss4 g track facilitators and inhibitors for the integration and use ...
2016 foss4 g track  facilitators and inhibitors  for the integration and use ...2016 foss4 g track  facilitators and inhibitors  for the integration and use ...
2016 foss4 g track facilitators and inhibitors for the integration and use ...
GIS in the Rockies
 
2016 conservation track: geolocation by light: following the migration of le...
2016 conservation track:  geolocation by light: following the migration of le...2016 conservation track:  geolocation by light: following the migration of le...
2016 conservation track: geolocation by light: following the migration of le...
GIS in the Rockies
 
2016 conservation track: strategies and tips for large scale data collection ...
2016 conservation track: strategies and tips for large scale data collection ...2016 conservation track: strategies and tips for large scale data collection ...
2016 conservation track: strategies and tips for large scale data collection ...
GIS in the Rockies
 
2016 conservation track: automated river classification using gis delineated ...
2016 conservation track: automated river classification using gis delineated ...2016 conservation track: automated river classification using gis delineated ...
2016 conservation track: automated river classification using gis delineated ...
GIS in the Rockies
 
2016 conservation track: identifying key wetlands areas in the rio grande na...
2016 conservation track:  identifying key wetlands areas in the rio grande na...2016 conservation track:  identifying key wetlands areas in the rio grande na...
2016 conservation track: identifying key wetlands areas in the rio grande na...
GIS in the Rockies
 
2016 conservation track: ecological and social resilience to climate variabil...
2016 conservation track: ecological and social resilience to climate variabil...2016 conservation track: ecological and social resilience to climate variabil...
2016 conservation track: ecological and social resilience to climate variabil...
GIS in the Rockies
 
2016 conservation track: applications of rapid ecoregional assessments (re as...
2016 conservation track: applications of rapid ecoregional assessments (re as...2016 conservation track: applications of rapid ecoregional assessments (re as...
2016 conservation track: applications of rapid ecoregional assessments (re as...
GIS in the Rockies
 
2016 asprs track: overview and user perspective of usgs 3 dep lidar by john ...
2016 asprs track:  overview and user perspective of usgs 3 dep lidar by john ...2016 asprs track:  overview and user perspective of usgs 3 dep lidar by john ...
2016 asprs track: overview and user perspective of usgs 3 dep lidar by john ...
GIS in the Rockies
 

Viewers also liked (13)

2016 vendor showcase track: an introduction to spike and the application of p...
2016 vendor showcase track: an introduction to spike and the application of p...2016 vendor showcase track: an introduction to spike and the application of p...
2016 vendor showcase track: an introduction to spike and the application of p...
 
2016 conservation track: under the hood of an rea: what is within a rapid ec...
2016 conservation track: under the hood of an rea:  what is within a rapid ec...2016 conservation track: under the hood of an rea:  what is within a rapid ec...
2016 conservation track: under the hood of an rea: what is within a rapid ec...
 
2016 asprs track: gis support for trail planning by jeff orlowski
2016 asprs track: gis support for trail planning by jeff orlowski2016 asprs track: gis support for trail planning by jeff orlowski
2016 asprs track: gis support for trail planning by jeff orlowski
 
2016 conservation track: evaluating lidar derived synthetic streams as a ...
2016 conservation track: evaluating   lidar derived   synthetic streams as a ...2016 conservation track: evaluating   lidar derived   synthetic streams as a ...
2016 conservation track: evaluating lidar derived synthetic streams as a ...
 
2016 conservation track: broad scale assessment, planning and management of ...
2016 conservation track:  broad scale assessment, planning and management of ...2016 conservation track:  broad scale assessment, planning and management of ...
2016 conservation track: broad scale assessment, planning and management of ...
 
2016 foss4 g track facilitators and inhibitors for the integration and use ...
2016 foss4 g track  facilitators and inhibitors  for the integration and use ...2016 foss4 g track  facilitators and inhibitors  for the integration and use ...
2016 foss4 g track facilitators and inhibitors for the integration and use ...
 
2016 conservation track: geolocation by light: following the migration of le...
2016 conservation track:  geolocation by light: following the migration of le...2016 conservation track:  geolocation by light: following the migration of le...
2016 conservation track: geolocation by light: following the migration of le...
 
2016 conservation track: strategies and tips for large scale data collection ...
2016 conservation track: strategies and tips for large scale data collection ...2016 conservation track: strategies and tips for large scale data collection ...
2016 conservation track: strategies and tips for large scale data collection ...
 
2016 conservation track: automated river classification using gis delineated ...
2016 conservation track: automated river classification using gis delineated ...2016 conservation track: automated river classification using gis delineated ...
2016 conservation track: automated river classification using gis delineated ...
 
2016 conservation track: identifying key wetlands areas in the rio grande na...
2016 conservation track:  identifying key wetlands areas in the rio grande na...2016 conservation track:  identifying key wetlands areas in the rio grande na...
2016 conservation track: identifying key wetlands areas in the rio grande na...
 
2016 conservation track: ecological and social resilience to climate variabil...
2016 conservation track: ecological and social resilience to climate variabil...2016 conservation track: ecological and social resilience to climate variabil...
2016 conservation track: ecological and social resilience to climate variabil...
 
2016 conservation track: applications of rapid ecoregional assessments (re as...
2016 conservation track: applications of rapid ecoregional assessments (re as...2016 conservation track: applications of rapid ecoregional assessments (re as...
2016 conservation track: applications of rapid ecoregional assessments (re as...
 
2016 asprs track: overview and user perspective of usgs 3 dep lidar by john ...
2016 asprs track:  overview and user perspective of usgs 3 dep lidar by john ...2016 asprs track:  overview and user perspective of usgs 3 dep lidar by john ...
2016 asprs track: overview and user perspective of usgs 3 dep lidar by john ...
 

Similar to Semantic search for Earth Observation products

Semantic search within Earth Observation products databases based on automati...
Semantic search within Earth Observation products databases based on automati...Semantic search within Earth Observation products databases based on automati...
Semantic search within Earth Observation products databases based on automati...
Gasperi Jerome
 
Machine Learning, Key to Your Classification Challenges
Machine Learning, Key to Your Classification ChallengesMachine Learning, Key to Your Classification Challenges
Machine Learning, Key to Your Classification Challenges
Marc Borowczak
 
DataCamp Cheat Sheets 4 Python Users (2020)
DataCamp Cheat Sheets 4 Python Users (2020)DataCamp Cheat Sheets 4 Python Users (2020)
DataCamp Cheat Sheets 4 Python Users (2020)
EMRE AKCAOGLU
 
Elasticsearch first-steps
Elasticsearch first-stepsElasticsearch first-steps
Elasticsearch first-steps
Matteo Moci
 
DH11: Browsing Highly Interconnected Humanities Databases Through Multi-Resul...
DH11: Browsing Highly Interconnected Humanities Databases Through Multi-Resul...DH11: Browsing Highly Interconnected Humanities Databases Through Multi-Resul...
DH11: Browsing Highly Interconnected Humanities Databases Through Multi-Resul...
Michele Pasin
 
Crab: A Python Framework for Building Recommender Systems
Crab: A Python Framework for Building Recommender Systems Crab: A Python Framework for Building Recommender Systems
Crab: A Python Framework for Building Recommender Systems
Marcel Caraciolo
 
RESTo - restful semantic search tool for geospatial
RESTo - restful semantic search tool for geospatialRESTo - restful semantic search tool for geospatial
RESTo - restful semantic search tool for geospatial
Gasperi Jerome
 
A Rusty introduction to Apache Arrow and how it applies to a time series dat...
A Rusty introduction to Apache Arrow and how it applies to a  time series dat...A Rusty introduction to Apache Arrow and how it applies to a  time series dat...
A Rusty introduction to Apache Arrow and how it applies to a time series dat...
Andrew Lamb
 
Reflected Intelligence - Lucene/Solr as a self-learning data system: Presente...
Reflected Intelligence - Lucene/Solr as a self-learning data system: Presente...Reflected Intelligence - Lucene/Solr as a self-learning data system: Presente...
Reflected Intelligence - Lucene/Solr as a self-learning data system: Presente...
Lucidworks
 
Reflected Intelligence: Lucene/Solr as a self-learning data system
Reflected Intelligence: Lucene/Solr as a self-learning data systemReflected Intelligence: Lucene/Solr as a self-learning data system
Reflected Intelligence: Lucene/Solr as a self-learning data system
Trey Grainger
 
An Introduction to Data Mining with R
An Introduction to Data Mining with RAn Introduction to Data Mining with R
An Introduction to Data Mining with R
Yanchang Zhao
 
Improving Semantic Search Using Query Log Analysis
Improving Semantic Search Using Query Log AnalysisImproving Semantic Search Using Query Log Analysis
Improving Semantic Search Using Query Log Analysis
Stuart Wrigley
 
Machine Learning ebook.pdf
Machine Learning ebook.pdfMachine Learning ebook.pdf
Machine Learning ebook.pdf
HODIT12
 
1_5_AI_edx_ml_51intro_240204_104838machine learning lecture 1
1_5_AI_edx_ml_51intro_240204_104838machine learning lecture 11_5_AI_edx_ml_51intro_240204_104838machine learning lecture 1
1_5_AI_edx_ml_51intro_240204_104838machine learning lecture 1
MostafaHazemMostafaa
 
Simple APIs and innovative documentation
Simple APIs and innovative documentationSimple APIs and innovative documentation
Simple APIs and innovative documentation
PyDataParis
 
Mapreduce Algorithms
Mapreduce AlgorithmsMapreduce Algorithms
Mapreduce Algorithms
Amund Tveit
 
know Machine Learning Basic Concepts.pdf
know Machine Learning Basic Concepts.pdfknow Machine Learning Basic Concepts.pdf
know Machine Learning Basic Concepts.pdf
hemangppatel
 
What's Coming Next in Sencha Frameworks
What's Coming Next in Sencha FrameworksWhat's Coming Next in Sencha Frameworks
What's Coming Next in Sencha Frameworks
Grgur Grisogono
 
Decision Tree.pptx
Decision Tree.pptxDecision Tree.pptx
Decision Tree.pptx
Ramakrishna Reddy Bijjam
 
Evaluating Your Learning to Rank Model: Dos and Don’ts in Offline/Online Eval...
Evaluating Your Learning to Rank Model: Dos and Don’ts in Offline/Online Eval...Evaluating Your Learning to Rank Model: Dos and Don’ts in Offline/Online Eval...
Evaluating Your Learning to Rank Model: Dos and Don’ts in Offline/Online Eval...
Sease
 

Similar to Semantic search for Earth Observation products (20)

Semantic search within Earth Observation products databases based on automati...
Semantic search within Earth Observation products databases based on automati...Semantic search within Earth Observation products databases based on automati...
Semantic search within Earth Observation products databases based on automati...
 
Machine Learning, Key to Your Classification Challenges
Machine Learning, Key to Your Classification ChallengesMachine Learning, Key to Your Classification Challenges
Machine Learning, Key to Your Classification Challenges
 
DataCamp Cheat Sheets 4 Python Users (2020)
DataCamp Cheat Sheets 4 Python Users (2020)DataCamp Cheat Sheets 4 Python Users (2020)
DataCamp Cheat Sheets 4 Python Users (2020)
 
Elasticsearch first-steps
Elasticsearch first-stepsElasticsearch first-steps
Elasticsearch first-steps
 
DH11: Browsing Highly Interconnected Humanities Databases Through Multi-Resul...
DH11: Browsing Highly Interconnected Humanities Databases Through Multi-Resul...DH11: Browsing Highly Interconnected Humanities Databases Through Multi-Resul...
DH11: Browsing Highly Interconnected Humanities Databases Through Multi-Resul...
 
Crab: A Python Framework for Building Recommender Systems
Crab: A Python Framework for Building Recommender Systems Crab: A Python Framework for Building Recommender Systems
Crab: A Python Framework for Building Recommender Systems
 
RESTo - restful semantic search tool for geospatial
RESTo - restful semantic search tool for geospatialRESTo - restful semantic search tool for geospatial
RESTo - restful semantic search tool for geospatial
 
A Rusty introduction to Apache Arrow and how it applies to a time series dat...
A Rusty introduction to Apache Arrow and how it applies to a  time series dat...A Rusty introduction to Apache Arrow and how it applies to a  time series dat...
A Rusty introduction to Apache Arrow and how it applies to a time series dat...
 
Reflected Intelligence - Lucene/Solr as a self-learning data system: Presente...
Reflected Intelligence - Lucene/Solr as a self-learning data system: Presente...Reflected Intelligence - Lucene/Solr as a self-learning data system: Presente...
Reflected Intelligence - Lucene/Solr as a self-learning data system: Presente...
 
Reflected Intelligence: Lucene/Solr as a self-learning data system
Reflected Intelligence: Lucene/Solr as a self-learning data systemReflected Intelligence: Lucene/Solr as a self-learning data system
Reflected Intelligence: Lucene/Solr as a self-learning data system
 
An Introduction to Data Mining with R
An Introduction to Data Mining with RAn Introduction to Data Mining with R
An Introduction to Data Mining with R
 
Improving Semantic Search Using Query Log Analysis
Improving Semantic Search Using Query Log AnalysisImproving Semantic Search Using Query Log Analysis
Improving Semantic Search Using Query Log Analysis
 
Machine Learning ebook.pdf
Machine Learning ebook.pdfMachine Learning ebook.pdf
Machine Learning ebook.pdf
 
1_5_AI_edx_ml_51intro_240204_104838machine learning lecture 1
1_5_AI_edx_ml_51intro_240204_104838machine learning lecture 11_5_AI_edx_ml_51intro_240204_104838machine learning lecture 1
1_5_AI_edx_ml_51intro_240204_104838machine learning lecture 1
 
Simple APIs and innovative documentation
Simple APIs and innovative documentationSimple APIs and innovative documentation
Simple APIs and innovative documentation
 
Mapreduce Algorithms
Mapreduce AlgorithmsMapreduce Algorithms
Mapreduce Algorithms
 
know Machine Learning Basic Concepts.pdf
know Machine Learning Basic Concepts.pdfknow Machine Learning Basic Concepts.pdf
know Machine Learning Basic Concepts.pdf
 
What's Coming Next in Sencha Frameworks
What's Coming Next in Sencha FrameworksWhat's Coming Next in Sencha Frameworks
What's Coming Next in Sencha Frameworks
 
Decision Tree.pptx
Decision Tree.pptxDecision Tree.pptx
Decision Tree.pptx
 
Evaluating Your Learning to Rank Model: Dos and Don’ts in Offline/Online Eval...
Evaluating Your Learning to Rank Model: Dos and Don’ts in Offline/Online Eval...Evaluating Your Learning to Rank Model: Dos and Don’ts in Offline/Online Eval...
Evaluating Your Learning to Rank Model: Dos and Don’ts in Offline/Online Eval...
 

More from Gasperi Jerome

Big data from space - Module Big Data ISAE 2017
Big data from space - Module Big Data ISAE 2017Big data from space - Module Big Data ISAE 2017
Big data from space - Module Big Data ISAE 2017
Gasperi Jerome
 
Le Big Data et les données Copernicus
Le Big Data et les données CopernicusLe Big Data et les données Copernicus
Le Big Data et les données Copernicus
Gasperi Jerome
 
2016.02.18 big data from space toulouse data science
2016.02.18   big data from space    toulouse data science2016.02.18   big data from space    toulouse data science
2016.02.18 big data from space toulouse data science
Gasperi Jerome
 
2015.11.12 big data from space - cusi toulouse
2015.11.12   big data from space - cusi toulouse2015.11.12   big data from space - cusi toulouse
2015.11.12 big data from space - cusi toulouse
Gasperi Jerome
 
Big Data - Accès et traitement des données d’Observation de laTerre
Big Data - Accès et traitement des données d’Observation de laTerreBig Data - Accès et traitement des données d’Observation de laTerre
Big Data - Accès et traitement des données d’Observation de laTerre
Gasperi Jerome
 
2014.09.04 federated ground segments - toulouse
2014.09.04   federated ground segments - toulouse2014.09.04   federated ground segments - toulouse
2014.09.04 federated ground segments - toulouse
Gasperi Jerome
 
Web Processing Service
Web Processing ServiceWeb Processing Service
Web Processing Service
Gasperi Jerome
 
2014.04.22 - HyDre - Hydroweb Distribution Server
2014.04.22 - HyDre - Hydroweb Distribution Server2014.04.22 - HyDre - Hydroweb Distribution Server
2014.04.22 - HyDre - Hydroweb Distribution Server
Gasperi Jerome
 
Single Sign On with OAuth and OpenID
Single Sign On with OAuth and OpenIDSingle Sign On with OAuth and OpenID
Single Sign On with OAuth and OpenID
Gasperi Jerome
 
CNES Data Center
CNES Data CenterCNES Data Center
CNES Data Center
Gasperi Jerome
 
CNES OpenSearch implementations
CNES OpenSearch implementationsCNES OpenSearch implementations
CNES OpenSearch implementations
Gasperi Jerome
 
Web Processing Service
Web Processing ServiceWeb Processing Service
Web Processing Service
Gasperi Jerome
 
Unify Earth Observation products access with OpenSearch
Unify Earth Observation products access with OpenSearchUnify Earth Observation products access with OpenSearch
Unify Earth Observation products access with OpenSearch
Gasperi Jerome
 
CNES activities on semantic search
CNES activities on semantic searchCNES activities on semantic search
CNES activities on semantic search
Gasperi Jerome
 
Traitements de données à la demande - Introduction au Web Processing Service
Traitements de données à la demande - Introduction au Web Processing ServiceTraitements de données à la demande - Introduction au Web Processing Service
Traitements de données à la demande - Introduction au Web Processing Service
Gasperi Jerome
 
Data access and data extraction services within the Land Imagery Portal
Data access and data extraction services within the Land Imagery PortalData access and data extraction services within the Land Imagery Portal
Data access and data extraction services within the Land Imagery Portal
Gasperi Jerome
 
Semantic search applied to Earth Observation products
Semantic search applied to Earth Observation productsSemantic search applied to Earth Observation products
Semantic search applied to Earth Observation products
Gasperi Jerome
 
Accès à l’information satellitaire dans un contexte réactif de catastrophe na...
Accès à l’information satellitaire dans un contexte réactif de catastrophe na...Accès à l’information satellitaire dans un contexte réactif de catastrophe na...
Accès à l’information satellitaire dans un contexte réactif de catastrophe na...
Gasperi Jerome
 
Experimenting a cloud based solution for image processing and data access
Experimenting a cloud based solution for image processing and data accessExperimenting a cloud based solution for image processing and data access
Experimenting a cloud based solution for image processing and data access
Gasperi Jerome
 
Interoperability and value added to earth observation data - 2011.11.24
Interoperability and value added to earth observation data - 2011.11.24Interoperability and value added to earth observation data - 2011.11.24
Interoperability and value added to earth observation data - 2011.11.24
Gasperi Jerome
 

More from Gasperi Jerome (20)

Big data from space - Module Big Data ISAE 2017
Big data from space - Module Big Data ISAE 2017Big data from space - Module Big Data ISAE 2017
Big data from space - Module Big Data ISAE 2017
 
Le Big Data et les données Copernicus
Le Big Data et les données CopernicusLe Big Data et les données Copernicus
Le Big Data et les données Copernicus
 
2016.02.18 big data from space toulouse data science
2016.02.18   big data from space    toulouse data science2016.02.18   big data from space    toulouse data science
2016.02.18 big data from space toulouse data science
 
2015.11.12 big data from space - cusi toulouse
2015.11.12   big data from space - cusi toulouse2015.11.12   big data from space - cusi toulouse
2015.11.12 big data from space - cusi toulouse
 
Big Data - Accès et traitement des données d’Observation de laTerre
Big Data - Accès et traitement des données d’Observation de laTerreBig Data - Accès et traitement des données d’Observation de laTerre
Big Data - Accès et traitement des données d’Observation de laTerre
 
2014.09.04 federated ground segments - toulouse
2014.09.04   federated ground segments - toulouse2014.09.04   federated ground segments - toulouse
2014.09.04 federated ground segments - toulouse
 
Web Processing Service
Web Processing ServiceWeb Processing Service
Web Processing Service
 
2014.04.22 - HyDre - Hydroweb Distribution Server
2014.04.22 - HyDre - Hydroweb Distribution Server2014.04.22 - HyDre - Hydroweb Distribution Server
2014.04.22 - HyDre - Hydroweb Distribution Server
 
Single Sign On with OAuth and OpenID
Single Sign On with OAuth and OpenIDSingle Sign On with OAuth and OpenID
Single Sign On with OAuth and OpenID
 
CNES Data Center
CNES Data CenterCNES Data Center
CNES Data Center
 
CNES OpenSearch implementations
CNES OpenSearch implementationsCNES OpenSearch implementations
CNES OpenSearch implementations
 
Web Processing Service
Web Processing ServiceWeb Processing Service
Web Processing Service
 
Unify Earth Observation products access with OpenSearch
Unify Earth Observation products access with OpenSearchUnify Earth Observation products access with OpenSearch
Unify Earth Observation products access with OpenSearch
 
CNES activities on semantic search
CNES activities on semantic searchCNES activities on semantic search
CNES activities on semantic search
 
Traitements de données à la demande - Introduction au Web Processing Service
Traitements de données à la demande - Introduction au Web Processing ServiceTraitements de données à la demande - Introduction au Web Processing Service
Traitements de données à la demande - Introduction au Web Processing Service
 
Data access and data extraction services within the Land Imagery Portal
Data access and data extraction services within the Land Imagery PortalData access and data extraction services within the Land Imagery Portal
Data access and data extraction services within the Land Imagery Portal
 
Semantic search applied to Earth Observation products
Semantic search applied to Earth Observation productsSemantic search applied to Earth Observation products
Semantic search applied to Earth Observation products
 
Accès à l’information satellitaire dans un contexte réactif de catastrophe na...
Accès à l’information satellitaire dans un contexte réactif de catastrophe na...Accès à l’information satellitaire dans un contexte réactif de catastrophe na...
Accès à l’information satellitaire dans un contexte réactif de catastrophe na...
 
Experimenting a cloud based solution for image processing and data access
Experimenting a cloud based solution for image processing and data accessExperimenting a cloud based solution for image processing and data access
Experimenting a cloud based solution for image processing and data access
 
Interoperability and value added to earth observation data - 2011.11.24
Interoperability and value added to earth observation data - 2011.11.24Interoperability and value added to earth observation data - 2011.11.24
Interoperability and value added to earth observation data - 2011.11.24
 

Recently uploaded

Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
David Brossard
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 

Recently uploaded (20)

Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 

Semantic search for Earth Observation products

  • 1. Jérôme Gasperi WGISS #37
 Cocoa Beach, Florida - USA - April 16th, 2014 Semantic search
  • 2. Semantic search helps users to find the right data
  • 3. Semantic search helps users to find the right data
  • 4. How to add semantics capabilities to EO products search services ?
  • 5.
  • 6. Characterize products with relevant information. Think « users », not « experts » 1
  • 7. Characterize products with relevant information. Think « users », not « experts » 1 Decode users natural language queries 2
  • 8.
  • 9. Use footprint to enrich metadata from exogenous data 1 Enrich products github.com/jjrom/itag
  • 10. ! Tag this footprint with continent, country and Land use ! http://goo.gl/WtbcbR
 iTag1 Enrich products
  • 11.
  • 12. 2 Decode queries RESTo provides semantic search capabilities It uses a Query Analyzer to translate query into a set of EO OpenSearch parameters
  • 13. Query string analysis algorithm is based on simple recognition of words and patterns Split query string into list of unitary words Extract «key=value» strings e.g. orbitNumber=4 Extract Platforms and Instruments Platforms and instruments list are stored within common dictionary !https://github.com/jjrom/resto/blob/ master/resto/dictionaries/common.php Remove excluded words and non dictionary words with length < 4 characters e.g. «area of Mexico in 2012» Extract patterns and dates e.g. «acquired in the last 2 days» Extract keywords e.g. «urban area in France» Extract location on remaining words e.g. «images acquired in Toulouse» 2 Decode queries
  • 14. Recognized patterns <with> "keyword" <without> "keyword" ! "quantity" <lesser> (than) "numeric" "unit" "quantity" <greater> (than) "numeric" "unit" "quantity" <equal> (to) "numeric" "unit" <lesser> (than) "numeric" "unit" (of) "quantity" <greater> (than) "numeric" "unit" (of) "quantity" <equal> (to) "numeric" "unit" (of) "quantity" "quantity" <between> "numeric" <and> "numeric" ("unit") <between> "numeric" <and> "numeric" "unit" (of) "quantity" ! <today> <yesterday> <before> "date" <after> "date" <between> "date" <and> "date" "numeric" "(year|day|month)" <ago> <last> "(year|day|month)" <last> "numeric" "(year|day|month)" "numeric" <last> "(year|day|month)" "(year|day|month)" <last> <since> "numeric" "(year|day|month)" <since> "month" "year" <since> "date" <since> "numeric" <last> "(year|day|month)" <since> <last> "numeric" "(year|day|month)" <since> <last> "(year|day|month)" <since> "(year|day|month)" <last> 2 Decode queries
  • 15. $dictionary = array( 'excluded' => array( 'than', 'image',
 ... ), 'modifiers' => array( 'ago' => 'ago', 'before' => 'before', 'after' => 'after', ... ), 'units' => array( 'm' => 'm', 'meter' => 'm',
 'days' => 'days',
 ... ), 'numbers' => array( 'one' => '1', ... ), 'months' => array( 'january' => '01', ... ), 'quantities' => array( 'resolution' => 'resolution', ... ), 'keywords' => array( 'continent' => array( 'europe' => 'europe', ... ) ) Words are stored within a dictionary 2 Decode queries
  • 16. « Images of urban area in the US acquired in the last 10 days with less than 5 % of cloud cover » Example 2 Decode queries
  • 17. 2 Decode queries « Images of urban area in the US acquired in the last 10 days with less than 5 % of cloud cover » Example keyword location date acquisition parameter
  • 18. 2. Each search result has an « human readable url » that can be indexed by web crawler (i.e. google robots) 1. Search parameters are derived from Natural Language query 3. Keywords on resources are links to search requests : they can be indexed by web crawler…and so on Search (example)
  • 19. 2. Each search result has an « human readable url » that can be indexed by web crawler (i.e. google robots) 1. Search parameters are derived from Natural Language query 3. Keywords on resources are links to search requests : they can be indexed by web crawler…and so on Search (example) http://goo.gl/GvMEHj
  • 21.
  • 22. Earthquakes in november 2008 in china
  • 23. Earthquakes in november 2008 in china Ambiguous since it appears to be a location in New Zealand
  • 24. « Linked data is the right way to do Semantic Web » Tim Berners-Lee
  • 25. Update RESTo JSON model to follow JSON-LD format { "@context": "http://json-ld.org/contexts/person.jsonld", "@id": "http://dbpedia.org/resource/John_Lennon", "name": "John Lennon", "born": "1940-10-09", "spouse": "http://dbpedia.org/resource/Cynthia_Lennon" }