Down and dirty with Elasticsearch
Upcoming SlideShare
Loading in...5
×
 

Down and dirty with Elasticsearch

on

  • 7,330 views

An introduction to Elasticsearch for the programmer. The basic concepts you need to understand to use Elasticsearch as a search engine

An introduction to Elasticsearch for the programmer. The basic concepts you need to understand to use Elasticsearch as a search engine

Statistics

Views

Total Views
7,330
Views on SlideShare
3,958
Embed Views
3,372

Actions

Likes
21
Downloads
225
Comments
0

8 Embeds 3,372

http://www.elasticsearch.org 3337
https://twitter.com 22
http://staging.elasticsearch.org 6
http://www.berlinbuzzwords.de 3
http://s0.wp.com 1
http://alu5.whsixnrfygnyo4te.uk35.e 1
http://translate.googleusercontent.com 1
http://new.elasticsearch.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Down and dirty with Elasticsearch Down and dirty with Elasticsearch Presentation Transcript

  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGetting Down and Dirtywith Elasticsearch@clintongormleyBerlin Buzzwords 2013
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearch
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchreal time,search andanalytics engine
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchreal time,search andanalytics enginedistributed
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchreal time,search andanalytics enginedistributedscalesmassively
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchreal time,search andanalytics enginedistributedscalesmassivelyhighavailability
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchreal time,search andanalytics enginedistributedscalesmassivelyhighavailabilityRESTfulAPI
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchreal time,search andanalytics enginedistributedscalesmassivelyhighavailabilityRESTfulAPIJSONover HTTP
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchreal time,search andanalytics enginedistributedscalesmassivelyhighavailabilityRESTfulAPIJSONover HTTPschemafree
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchreal time,search andanalytics enginedistributedscalesmassivelyhighavailabilityRESTfulAPIJSONover HTTPschemafreemultitenancy
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchreal time,search andanalytics engineopen-sourcedistributedscalesmassivelyhighavailabilityRESTfulAPIJSONover HTTPschemafreemultitenancy
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedElasticsearchreal time,search andanalytics engineopen-sourceLucenebaseddistributedscalesmassivelyhighavailabilityRESTfulAPIJSONover HTTPschemafreemultitenancy
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedCool.
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedCool. Bonsai cool...
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedThis is WHY we use it...
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited> ./bin/elasticsearch> _
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedBut HOW do we use it?
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited> curl -XGET localhost:9200/?pretty
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited> curl -XGET localhost:9200/?prettyverb
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited> curl -XGET localhost:9200/?prettynode
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited> curl -XGET localhost:9200/?prettyHTTP port
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited> curl -XGET localhost:9200/?prettypath
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited> curl -XGET localhost:9200/?prettyquery string
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited> curl -XGET localhost:9200/?pretty
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /{"name" : "Exploding Man","tagline" : "You Know, for Search","ok" : true,"status" : 200,"version" : {"number" : "0.90.1","snapshot_build" : false}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedWhere do we start?
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedWith data
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"tweet": "I think #elasticsearch is AWESOME","nick": "@clintongormley","name": "Clinton Gormley","date": "2013-06-03","rt" : 5,"loc": {! "lat": 13.4,! "lon": 52.5}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedHow to put it into ES?
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPUT /index/type/id
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPUT /index/type/idwhere?
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPUT /myapp/type/id
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPUT /myapp/type/idwhat?
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPUT /myapp/tweet/id
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPUT /myapp/tweet/idwhich?
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPUT /myapp/tweet/1
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPUT /myapp/tweet/1 -d {"tweet": "I think #elasticsearch is AWESOME","nick": "@clintongormley","name": "Clinton Gormley","date": "2013-06-03","rt": 5,"loc": {! "lat": 13.4,! "lon": 52.5}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"_index": "myapp","_type": "tweet","_id": "1","_version": 1,"ok": true}# 201 CREATED
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGet
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /myapp/tweet/1
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"_index": "myapp","_type": "tweet","_id": "1","_version": 1,"exists": true,"_source": { ...OUR TWEET... }}# 200 OK
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedExists?
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedHEAD /myapp/tweet/1
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedHEAD /myapp/tweet/1 # 200 OK
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedHEAD /myapp/tweet/1 # 200 OKHEAD /myapp/tweet/2 # 404 Not Found
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUpdate
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPUT /myapp/tweet/1 -d {"tweet": "I know #elasticsearch is AWESOME","nick": "@clintongormley","name": "Clinton Gormley","date": "2013-06-03","rt": 5,"loc": {! "lat": 13.4,! "lon": 52.5}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited! atomic DELETE & PUT
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"_index": "myapp","_type": "tweet","_id": "1","_version": 2,"ok": true}# 200 OK
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedDelete
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedDELETE /myapp/tweet/1
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"_index": "myapp","_type": "tweet","_id": "1","_version": 3,"ok": true,"found": true}# 200 OK
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedOptimisticconcurrency control
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedOptimisticconcurrency controlwithout locking
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPUT /myapp/tweet/1?version=3 -d {...}# 200 OK
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPUT /myapp/tweet/1?version=2 -d {...}# 409 Conflict
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUpdate in place
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPOST /myapp/tweet/1/_update -d {"script": "ctx._source.count+=1","retry_on_conflict": 3}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPOST /myapp/tweet/1/_update -d {"script": "ctx._source.count+=1","retry_on_conflict": 3}GET ! change ! PUT
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedCheaper in bulk
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedAnydatastoreElasticsearchClientMirror external DB
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedAnydatastoreElasticsearchClientStandalone
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited"Empty" Search
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search{"took" : 2,}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search{"took" : 2,"timed_out" : false,}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search{"took" : 2,"timed_out" : false,"_shards" : {"total" : 10,"successful" : 10,"failed" : 0},}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search{"took" : 2,"timed_out" : false,"_shards" : {"total" : 10,"successful" : 10,"failed" : 0},"hits" : {"total" : 14,"max_score" : 1.0,"hits" : [ { ... }]}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search"hits" : [{"_index" : "de","_type" : "tweet","_id" : "4","_source" : { ... },"_score" : 1.0,},...]
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedMulti-indexMulti-type
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /index/_search
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /index/_searchGET /index1,index2/_search
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /index/_searchGET /index1,index2/_searchGET /ind*/_search
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /index/_searchGET /index1,index2/_searchGET /ind*/_searchGET /index/type/_search
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /index/_searchGET /index1,index2/_searchGET /ind*/_searchGET /index/type/_searchGET /index/type1,type2/_search
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /index/_searchGET /index1,index2/_searchGET /ind*/_searchGET /index/type/_searchGET /index/type1,type2/_searchGET /index/type*/_search
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /index/_searchGET /index1,index2/_searchGET /ind*/_searchGET /index/type/_searchGET /index/type1,type2/_searchGET /index/type*/_searchGET /_all/type*/_search
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPagination
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPaginationsize = num of results
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPaginationsize = num of resultsfrom = results to skip
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search?size=5&from=0GET /_search?size=5&from=5GET /_search?size=5&from=10
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch Lite
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedSearch LiteGET /_search?q=name:john
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited+tweet:foo +name:john +date:>2013-05-01
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited+tweet:foo +name:john +date:>2013-05-01→ percent encoding →
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited+tweet:foo +name:john +date:>2013-05-01?q=%2Btweet%3Afoo+%2Bname%3Ajohn+%2Bdate%3A%3E2013-05-01→ percent encoding →
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search?q=mary
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited→ user named "Mary"→ tweets by "Mary"→ tweet mentioning "@mary"GET /_search?q=mary
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search?q=_all:mary→ user named "Mary"→ tweets by "Mary"→ tweet mentioning "@mary"
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited_all fieldstring values fromall other fields
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search?q=2013! 12 results
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search?q=2013! 12 resultsGET /_search?q=2013-06-03! 12 results!!
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search?q=2013! 12 resultsGET /_search?q=2013-06-03! 12 results!!GET /_search?q=date:2013-06-03! 1 result
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search?q=2013! 12 resultsGET /_search?q=2013-06-03! 12 results!!GET /_search?q=date:2013-06-03! 1 resultGET /_search?q=date:2013! 0 results!!
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibiteddatatype differences?
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedcheck "mapping"(field definitions)
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /myapp/tweet/_mapping
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /myapp/tweet/_mapping{"tweet" : {"properties" : {"tweet" : { "type" : "string" },"name" : { "type" : "string" },"nick" : { "type" : "string" },"date" : { "type" : "date" },"rt" : { "type" : "long" },"loc" : {"type": "object","properties" : {"lat" : { "type" : "double" },"lon" : { "type" : "double" }}}}}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibiteddate = type:date_all = type:string
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedExact value vs Full text
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedExact value vs104.52013-01-01trueFoofooFull text
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedExact value vs104.52013-01-01trueFoofooFull textThe quickbrown foxjumpedover thelazy dog
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedInverted index“The quick brown fox jumped over the lazy dog”“Quick brown foxes leap over lazy dogs in summer”
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedInverted index→ separate words / terms“The quick brown fox jumped over the lazy dog”“Quick brown foxes leap over lazy dogs in summer”
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedInverted index→ separate words / termsThe,quick,brown,fox,jumped,over,the,lazy,dogQuick,brown,foxes,leap,over,lazy,dogs,in,summer
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedInverted indexThe,quick,brown,fox,jumped,over,the,lazy,dogQuick,brown,foxes,leap,over,lazy,dogs,in,summer→ separate words / terms→ sort unique terms
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedInverted index→ separate words / terms→ sort unique termsThe,brown,dog,fox,jumped,lazy,over,quick,theQuick,brown,dogs,foxes,in,lazy,leap,over,summer
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedInverted index→ separate words / terms→ sort unique terms→ list docs containing termsThe,brown,dog,fox,jumped,lazy,over,quick,theQuick,brown,dogs,foxes,in,lazy,leap,over,summer
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedTerm Doc 1 Doc 2QuickThebrowndogdogsfoxfoxesinjumpedlazyleapoverquicksummerthe
  • Term Doc 1 Doc 2QuickThebrowndogdogsfoxfoxesinjumpedlazyleapoverquicksummertheCopyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedq=quick brown
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedTerm Doc 1 Doc 2QuickThebrowndogdogsfoxfoxesinjumpedlazyleapoverquicksummerthe
  • Term Doc 1 Doc 2QuickThebrowndogdogsfoxfoxesinjumpedlazyleapoverquicksummertheCopyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedq=+Quick +foxes
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedTerm Doc 1 Doc 2QuickThebrowndogdogsfoxfoxesinjumpedlazyleapoverquicksummerthe
  • Term Doc 1 Doc 2QuickThebrowndogdogsfoxfoxesinjumpedlazyleapoverquicksummertheCopyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedNo matches!
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedImproving recall
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedTerm Doc 1 Doc 2QuickThebrowndogdogsfoxfoxesinjumpedlazyleapoverquicksummerthe
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedTerm Doc 1 Doc 2browndogdogsfoxfoxesinjumpedlazyleapoverquicksummerthe
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedTerm Doc 1 Doc 2browndogdogsfoxfoxesinjumpedlazyleapoverquicksummerthe
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedTerm Doc 1 Doc 2browndogfoxinjumpedlazyleapoverquicksummerthe
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedTerm Doc 1 Doc 2browndogfoxinjumpedlazyleapoverquicksummerthe
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedTerm Doc 1 Doc 2browndogfoxinjumpleapoverquicksummerthe
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitednormalize terms
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedTerm Doc 1 Doc 2browndogfoxinjumpleapoverquicksummerthe
  • Term Doc 1 Doc 2browndogfoxinjumpleapoverquicksummertheCopyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedq=+Quick +foxes
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedTerm Doc 1 Doc 2browndogfoxinjumpleapoverquicksummerthe
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitednormalize termsin query too!
  • Term Doc 1 Doc 2browndogfoxinjumpleapoverquicksummertheCopyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedq=+Quick +foxes
  • Term Doc 1 Doc 2browndogfoxinjumpleapoverquicksummertheCopyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedq=+quick +foxes
  • Term Doc 1 Doc 2browndogfoxinjumpleapoverquicksummertheCopyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedq=+quick +fox
  • Term Doc 1 Doc 2browndogfoxinjumpleapoverquicksummertheCopyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited"Analysis"
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited"Analysis"tokenization + normalization
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited"Analysers"tokenizer + token filters
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedstandard analyzer"The Quick Brown Fox jumpedover the Lazy Dog!"
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedstandard analyzer→ standard tokenizer"The Quick Brown Fox jumpedover the Lazy Dog!"
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedstandard analyzer→ standard tokenizerThe,Quick,Brown,Fox,jumped,over,the,Lazy,Dog
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedstandard analyzer→ standard tokenizer→ lowercase filterThe,Quick,Brown,Fox,jumped,over,the,Lazy,Dog
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedstandard analyzer→ standard tokenizer→ lowercase filterthe,quick,brown,fox,jumped,over,the,lazy,dog
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedstandard analyzer→ standard tokenizer→ lowercase filter→ stopwords filterthe,quick,brown,fox,jumped,over,the,lazy,dog
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedstandard analyzer→ standard tokenizer→ lowercase filter→ stopwords filter,quick,brown,fox,jumped,over, ,lazy,dog
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedenglish analyzer→ standard tokenizer→ lowercase filterthe,quick,brown,fox,jumped,over,the,lazy,dog
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedenglish analyzer→ standard tokenizer→ lowercase filter→ english stemmerthe,quick,brown,fox,jumped,over,the,lazy,dog
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedenglish analyzer→ standard tokenizer→ lowercase filter→ english stemmerthe,quick,brown,fox,jumped,over,the,lazy,dog
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedenglish analyzer→ standard tokenizer→ lowercase filter→ english stemmerthe,quick,brown,fox,jump,over,the,lazy,dog
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedenglish analyzer→ standard tokenizer→ lowercase filter→ english stemmer→ english stopwordsthe,quick,brown,fox,jump,over,the,lazy,dog
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedenglish analyzer→ standard tokenizer→ lowercase filter→ english stemmer→ english stopwords,quick,brown,fox,jump,over, ,lazy,dog
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibiteddate = type:date_all = type:string
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibiteddate = exact value_all = full text
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibiteddate = 2013-06-03_all = 2013,06,03
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search?q=2013! 12 results
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search?q=2013! 12 resultsGET /_search?q=2013-06-03! 12 results
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search?q=2013! 12 resultsGET /_search?q=2013 OR 06 OR 03! 12 results
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search?q=2013! 12 resultsGET /_search?q=2013-06-03! 12 resultsGET /_search?q=date:2013-06-03! 1 result
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search?q=2013! 12 resultsGET /_search?q=2013-06-03! 12 resultsGET /_search?q=date:2013-06-03! 1 resultGET /_search?q=date:2013! 0 results
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedField mapping
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedStrings: stringDatetimes: dateWhole numbers: byte, short, integer, longFloats: float, doubleBooleans: booleanObjects: objectCore field types
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedStrings: stringDatetimes: dateWhole numbers: byte, short, integer, longFloats: float, doubleBooleans: booleanObjects: objectAlso: multi_field, ip, geo_point, geo_shape,Core field types
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited"foo bar" string"2013-01-01" date10 byte, short, integer, long10.0 float, doubletrue boolean{ foo: "bar" } objectDynamic detection
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited"foo bar" string"2013-01-01" date10 byte, short, integer, long10.0 float, doubletrue boolean{ foo: "bar" } object["foo","bar"] No special mapping. Anyfield can have multi-valsDynamic detection
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedMost important: type
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"tweet" : {"properties" : {"tweet" : { "type" : "string" },"name" : { "type" : "string" },"nick" : { "type" : "string" },"date" : { "type" : "date" },"rt" : { "type" : "long" },"loc" : {"type": "object","properties" : {"lat" : { "type" : "double" },"lon" : { "type" : "double" }}}}}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"tweet" : {"properties" : {"tweet" : { "type" : "string" },"name" : { "type" : "string" },"nick" : { "type" : "string" },"date" : { "type" : "date" },"rt" : { "type" : "long" },"loc" : { "type" : "geo_point" }}}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedFull text vs Exact string
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedFull text: (default){ "type": "string", "index": "analyzed" }
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedFull text: (default){ "type": "string", "index": "analyzed" }Exact string:{ "type": "string", "index": "not_analyzed" }
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedFull text: (default){ "type": "string", "index": "analyzed" }Exact string:{ "type": "string", "index": "not_analyzed" }Not searchable:{ "type": "string", "index": "no" }
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"tweet" : {"properties" : {"tweet" : { "type" : "string" },"name" : { "type" : "string" },"nick" : { "type" : "string" },"date" : { "type" : "date" },"rt" : { "type" : "long" },"loc" : { "type" : "geo_point" }}}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"tweet" : {"properties" : {"tweet" : { "type" : "string" },"name" : { "type" : "string" },"nick" : {"type" : "string","index" : "not_analyzed"},"date" : { "type" : "date" },"rt" : { "type" : "long" },"loc" : { "type" : "geo_point" }}}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedAnalyzer
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"tweet" : {"properties" : {"tweet" : { "type" : "string" },"name" : { "type" : "string" },"nick" : {"type" : "string","index" : "not_analyzed"},"date" : { "type" : "date" },"rt" : { "type" : "long" },"loc" : { "type" : "geo_point" }}}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"tweet" : {"properties" : {"tweet" : {"type" : "string","analyzer" : "english"},"name" : { "type" : "string" },"nick" : {"type" : "string","index" : "not_analyzed"},"date" : { "type" : "date" },"rt" : { "type" : "long" },"loc" : { "type" : "geo_point" }}}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedUpdating mappings
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedCan: add new fields
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedCan: add new fieldsPUT /myapp/tweet/_mapping -d {"tweet": {"properties": {...}}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedCannot: change fields
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedCannot: change fieldsDELETE /myapp
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedCannot: change fieldsPUT /myapp -d {"mappings": {"tweet": {"properties": {...}}}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedFull body search
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search -d {"query": {"match_all": {}},"from": 0,"size": 10}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search -d {"query": {"match_all": {}},"from": 0,"size": 10}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedQuery DSLrich flexible query language
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"match": { "tweet": "search" }}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search -d {{"query": {"match": { "tweet": "search" }}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedFilters vs Queries
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedFilters vsexact matchingQueriesfull text search
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedFilters vsexact matchingbinary yes/noQueriesfull text searchrelevance scoring
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedFilters vsexact matchingbinary yes/nofastQueriesfull text searchrelevance scoringheavier
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedFilters vsexact matchingbinary yes/nofastcacheableQueriesfull text searchrelevance scoringheaviernot cacheable
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedQuery: { "match": { "tweet": "search" }}Filter: { "term": { "nick": "@mary" }}Combine filter & query
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"filtered": {"query": {"match": { "tweet": "search" }},"filter": {"term": { "nick": "@mary" }}}}Combine filter & query
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search -d {"query": {"filtered": {"query": {"match": { "tweet": "search" }},"filter": {"term": { "nick": "@mary" }}}}}Combine filter & query
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search -d {"query": {"filtered": {"query": {"match_all": {}},"filter": {"term": { "nick": "@mary" }}}}}Just a filter
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search -d {"query": {"filtered": {"filter": {"term": { "nick": "@mary" }}}}}Just a filter
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search -d {"query": {"filtered": {"filter": {"term": { "nick": "@mary" }}}},"sort": { "date": "desc" }}Users tweets by date
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_search -d {"query": {"filtered": {"filter": {"range": {"date": {"gte": "2013-05-01","lt": "2013-06-01"}}}}}}Tweets for last month
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_all/tweet/_search -d {"facets": {"top_tweeters": {"terms": {"field": "nick"}}}}Top tweeters
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_all/tweet/_search -d {"facets": {"top_tweeters": {"terms": {"field": "nick"}}},"query": {"match": { "tweet": "elasticsearch" }}}Top tweeters for query
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedGET /_all/tweet/_search -d {"facets": {"tweets_by_month": {"date_histogram": {"field": "date","interval": "month"}}}}Tweets by month
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedAutocomplete{ "match": { "name": "joh" }}"John SmithJohnny DeppLyndon Johnson
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedAutocomplete{ "match": { "name": "joh" }}"John SmithJohnny DeppLyndon JohnsonBut "joh" doesnt exist in the index
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedN-grams == window-on-a-word:Autocomplete
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedN-grams == window-on-a-word:Length 1: j,o,h,n,s,m,i,t,hAutocomplete
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedN-grams == window-on-a-word:Length 1: j,o,h,n,s,m,i,t,hLength 2: jo,oh,hn,sm,mi,it,thAutocomplete
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedN-grams == window-on-a-word:Length 1: j,o,h,n,s,m,i,t,hLength 2: jo,oh,hn,sm,mi,it,thLength 3: joh,ohn,smi,mit,ithLength 4: john,smit,mithAutocomplete
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedN-grams == window-on-a-word:Length 1: j,o,h,n,s,m,i,t,hLength 2: jo,oh,hn,sm,mi,it,thLength 3: joh,ohn,smi,mit,ithLength 4: john,smit,mithAutocompleteGood for partial word matching
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedEdge N-grams == anchored N-grams:Autocomplete
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedEdge N-grams == anchored N-grams:jjojohjohnssmsmismitsmithAutocomplete
  • Edge N-grams == anchored N-grams:jjojohjohnssmsmismitsmithCopyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedAutocompletePerfect forautocomplete
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedEdge N-Gram token filter{"filter": {"autocomplete": {"type": "edge_ngram","min_gram": 1,"max_gram": 20}}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedName field analyzers{"analyzer": {"name": {"type": "standard","stopwords": []},}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedName field analyzers{"analyzer": {"name": {"type": "standard","stopwords": []},"name_autocomplete": {"type": "custom","tokenizer": "standard","filter": ["lowercase","autocomplete"]}}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedName field mapping{"name": {"type": "string"}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedName field mapping{"name": {"type": "string"}}multi_field == one field, multi-purposes
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedName field mapping{"name": {"type": "multi_field","fields": {"name": {},"autocomplete": {}}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedName field mapping{"name": {"type": "multi_field","fields": {"name": {},"autocomplete": {}}}Main field:"name" or "name.name"
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedName field mapping{"name": {"type": "multi_field","fields": {"name": {"type": "string","analyzer": "name"},"autocomplete": {}}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedName field mapping{"name": {"type": "multi_field","fields": {"name": {"type": "string","analyzer": "name"},"autocomplete": {}}}Sub field:"name.autocomplete"
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedName field mapping{"name": {"type": "multi_field","fields": {"name": {"type": "string","analyzer": "name"},"autocomplete": {"type": "string","index_analyzer": "name_autocomplete","search_analyzer": "name"}}}
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedDELETE /myappRecreate the index
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPUT /myapp -d {"settings": {"analysis": {"analyzer": {...},"filter": {...}}},}Recreate the index
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedPUT /myapp -d {"settings": {"analysis": {"analyzer": {...},"filter": {...}}},"mappings": {"tweet": {"properties": {...}}}}Recreate the index
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"match": {"name.autocomplete": "john smi"}}Autocomplete query
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"match": {"name.autocomplete": "john smi"}}Autocomplete queryBetter: favor whole word matches
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"bool": {"must": [{...},{...}],"must_not": [{...},{...}],"should": [{...},{...}]}}Autocomplete queryCombines multiple query clauses
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"bool": {"must": [{...},{...}],"must_not": [{...},{...}],"should": [{...},{...}]}}Autocomplete queryMUST match
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"bool": {"must": [{...},{...}],"must_not": [{...},{...}],"should": [{...},{...}]}}Autocomplete queryMUST NOT match
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"bool": {"must": [{...},{...}],"must_not": [{...},{...}],"should": [{...},{...}]}}Autocomplete query"More relevant" if these match
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"bool": {"must": {"match": {"name.autocomplete": "john smi"}},"should": {"match": {"name": "john smi"}}Autocomplete query
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"custom_score_query": {"query": { "match": { "tweet": "search" }},"script": "_score * (1+log(doc[rt].value))"}}Boost popular tweets
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"filtered": {"query": { "match": { "tweet": "search" }},"filter": {"geo_distance": {"distance": "100km","loc": {"lat": 13.4,"lon": 52.5}}}Filter local tweets
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"custom_filters_score_query": {"query": { "match": { "tweet": "search" }},"filters": []}}Boost local tweets
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited{"custom_filters_score_query": {"query": { "match": { "tweet": "search" }},"filters": [{"boost": 2,"filter": {"geo_distance": {"distance": "100km","loc": { "lat": 13.4, "lon": 52.5 }}}}]}}Boost local tweets
  • Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibitedwww.elasticsearch.org