Karel Minařík
elasticsearch
in 15 minutes
Plug & Play
Installation
$ wget https://download.elasticsearch.org/...
$ tar -xf elasticsearch-0.90.2.tar.gz
$ ./elasticsearch-0.90.2/...
Index a document...
$ curl -X PUT localhost:9200/products/product/1 -d '{
"title" : "Welcome!"
}'
Update a document...
$ curl -X PUT localhost:9200/products/product/1 -d '{
"title" : "Welcome to the Elasticsearch meetup!...
Search for documents....
$ curl -X GET localhost:9200/products/_search?q=welcome
Add a node...
$ ./elasticsearch-0.90.2/bin/elasticsearch -f -D es.node.name=Node2
...[cluster.service] [Node2] detected_ma...
Add another node...
$ ./elasticsearch-0.90.2/bin/elasticsearch -f -D es.node.name=Node3
...[cluster.service] [Node3] detec...
Until you know what to tweak...
Shard & Cluster
A
curl  -­‐XPUT  'http://localhost:9200/a/'  -­‐d  '{
        "settings"  :  {
                "index"  :  {
             ...
1 node 2 nodes 3 nodes
Demo
"index.routing.allocation.exclude.name"      :  "Node1"
"cluster.routing.allocation.exclude.na...
JSON & HTTP
{
    "id"        :  "abc123",
    "title"  :  "A  JSON  Document",
    "body"    :  "A  JSON  document  is  a  ...",
    ...
Documents as JSON
https://wiki.postgresql.org/images/b/b4/Pg-as-nosql-pgday-fosdem-2013.pdf
http:// Lingua Franca of APIs
Also supported: Native Java protocol, Thrift, Memcached
Search & Find
Terms
apple
apple  iphone
Phrases "apple  iphone"
Proximity "apple  safari"~5
Fuzzy apple~0.8
Wildcards
app*
*pp*
Boosting...
curl  -­‐X  GET  localhost:9200/articles/_search  -­‐d  '{
"query" : {
"filtered" : {
"query" : {
"bool" : {
"must" : {
"m...
curl  -­‐X  GET  localhost:9200/articles/_search  -­‐d  '{
"query" : {
"filtered" : {
"query" : {
"bool" : {
"must" : {
"m...
curl  -­‐X  GET  localhost:9200/articles/_search  -­‐d  '{
"query" : {
"filtered" : {
"query" : {
"bool" : {
"must" : {
"m...
curl  -­‐X  GET  localhost:9200/articles/_search  -­‐d  '{
"query" : {
"filtered" : {
"query" : {
"bool" : {
"must" : {
"m...
curl  -­‐X  GET  localhost:9200/articles/_search  -­‐d  '{
"query" : {
"filtered" : {
"query" : {
"bool" : {
"must" : {
"m...
“Find all articles with ‘search’ in their title or body, give
matches in titles higher score”
Full-text Search
“Find all a...
Fetch document field ➝ Pick configured analyzer ➝ Parse
text into tokens ➝ Apply token filters ➝ Store into index
How Sear...
Mapping
curl -X PUT localhost:9200/articles/_mapping -d '{
"article" : {
"properties" : {
"title" : {
"type" : "string",
"...
_analyze?pretty&format=text&text=Žluťoučký+kůň+skákal+přes+potok
The _analyze API
[žluťoučký:0-­‐>9:<ALPHANUM>]
nn2:  n[ků...
Slice & Dice
Query
Facets
Location
Product
Tim
e
OLAP Cube
Dimensions, measures, aggregations
Slice Dice Drill Down / Roll Up
Show me sales numbers for all products across all locations in year 2013
Show me product A...
curl -X POST 'localhost:9200/articles/_search?search_type=count&pretty' -d '{
"facets": {
"tag-cloug": {
"terms" : {
"fiel...
curl -X GET 'localhost:9200/scores/_search/?search_type=count&pretty' -d '{
"facets": {
"scores-per-subject" : {
"terms_st...
Facets
Terms
Terms Stats
Statistical
Range
Histogram
Date Histogram
Filter
Query
Geo Distance
Above
& 
Beyond
Above & Beyond
Bulk operations (For indexing and search operations)
Percolator (“reversed search” — alerts, classification...
thanks!
Elasticsearch in 15 Minutes
Elasticsearch in 15 Minutes
Upcoming SlideShare
Loading in...5
×

Elasticsearch in 15 Minutes

4,170

Published on

Short overview of Elasticsearch features at the Prague user group meetup 27/6/2013

Published in: Technology, News & Politics
1 Comment
5 Likes
Statistics
Notes
No Downloads
Views
Total Views
4,170
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
1
Likes
5
Embeds 0
No embeds

No notes for slide

Transcript of "Elasticsearch in 15 Minutes"

  1. 1. Karel Minařík elasticsearch in 15 minutes
  2. 2. Plug & Play
  3. 3. Installation $ wget https://download.elasticsearch.org/... $ tar -xf elasticsearch-0.90.2.tar.gz $ ./elasticsearch-0.90.2/bin/elasticsearch -f ... [INFO ][node][Ghost Maker] {0.90.2}[5645]: initializing ...
  4. 4. Index a document... $ curl -X PUT localhost:9200/products/product/1 -d '{ "title" : "Welcome!" }'
  5. 5. Update a document... $ curl -X PUT localhost:9200/products/product/1 -d '{ "title" : "Welcome to the Elasticsearch meetup!" }'
  6. 6. Search for documents.... $ curl -X GET localhost:9200/products/_search?q=welcome
  7. 7. Add a node... $ ./elasticsearch-0.90.2/bin/elasticsearch -f -D es.node.name=Node2 ...[cluster.service] [Node2] detected_master [Node1] ...
  8. 8. Add another node... $ ./elasticsearch-0.90.2/bin/elasticsearch -f -D es.node.name=Node3 ...[cluster.service] [Node3] detected_master [Node1] ...
  9. 9. Until you know what to tweak...
  10. 10. Shard & Cluster
  11. 11. A curl  -­‐XPUT  'http://localhost:9200/a/'  -­‐d  '{        "settings"  :  {                "index"  :  {                        "number_of_shards"      :  3,                        "number_of_replicas"  :  1                }        } }' Index is partitioned into 3 primary shards, each is duplicated in 1 replica shard A1 A2 A3 Replicas Primaries A1' A2' A3'
  12. 12. 1 node 2 nodes 3 nodes Demo "index.routing.allocation.exclude.name"      :  "Node1" "cluster.routing.allocation.exclude.name"  :  "Node3" ... http://git.io/elasticat
  13. 13. JSON & HTTP
  14. 14. {    "id"        :  "abc123",    "title"  :  "A  JSON  Document",    "body"    :  "A  JSON  document  is  a  ...",    "published_on"  :  "2013/06/27  10:00:00",    "featured"          :  true,        "tags"    :  ["search",  "json"],    "author"  :  {        "first_name"  :  "Clara",        "last_name"    :  "Rice",        "email"            :  "clara@rice.org"    } } Documents as JSON Data structure with basic types, arrays and deep hierarchies
  15. 15. Documents as JSON https://wiki.postgresql.org/images/b/b4/Pg-as-nosql-pgday-fosdem-2013.pdf
  16. 16. http:// Lingua Franca of APIs Also supported: Native Java protocol, Thrift, Memcached
  17. 17. Search & Find
  18. 18. Terms apple apple  iphone Phrases "apple  iphone" Proximity "apple  safari"~5 Fuzzy apple~0.8 Wildcards app* *pp* Boosting apple^10  safari Range [2011/05/01  TO  2011/05/31] [java  TO  json] Boolean apple  AND  NOT  iphone +apple  -­‐iphone (apple  OR  iphone)  AND  NOT  review Fields title:iphone^15  OR  body:iphone published_on:[2011/05/01  TO  "2011/05/27  10:00:00"] http://lucene.apache.org/java/3_1_0/queryparsersyntax.html $  curl  -­‐X  GET  "http://localhost:9200/_search?q=<YOUR  QUERY>"
  19. 19. curl  -­‐X  GET  localhost:9200/articles/_search  -­‐d  '{ "query" : { "filtered" : { "query" : { "bool" : { "must" : { "match" : { "author.first_name" : { "query" : "claire", "fuzziness" : 0.1 } } }, "must" : { "multi_match" : { "query" : "elasticsearch", "fields" : ["title^10", "body"] } } } }, "filter": { "and" : [ { "terms" : { "tags" : ["search"] } }, { "range" : { "published_on": {"from": "2013"} } }, { "term" : { "featured" : true } } ] } } } }' JSON-based Query DSL
  20. 20. curl  -­‐X  GET  localhost:9200/articles/_search  -­‐d  '{ "query" : { "filtered" : { "query" : { "bool" : { "must" : { "match" : { "author.first_name" : { "query" : "claire", "fuzziness" : 0.1 } } }, "must" : { "multi_match" : { "query" : "elasticsearch", "fields" : ["title^10", "body"] } } } }, "filter": { "and" : [ { "terms" : { "tags" : ["search"] } }, { "range" : { "published_on": {"from": "2013"} } }, { "term" : { "featured" : true } } ] } } } }' JSON-based Query DSL
  21. 21. curl  -­‐X  GET  localhost:9200/articles/_search  -­‐d  '{ "query" : { "filtered" : { "query" : { "bool" : { "must" : { "match" : { "author.first_name" : { "query" : "claire", "fuzziness" : 0.1 } } }, "must" : { "multi_match" : { "query" : "elasticsearch", "fields" : ["title^10", "body"] } } } }, "filter": { "and" : [ { "terms" : { "tags" : ["search"] } }, { "range" : { "published_on": {"from": "2013"} } }, { "term" : { "featured" : true } } ] } } } }' JSON-based Query DSL
  22. 22. curl  -­‐X  GET  localhost:9200/articles/_search  -­‐d  '{ "query" : { "filtered" : { "query" : { "bool" : { "must" : { "match" : { "author.first_name" : { "query" : "claire", "fuzziness" : 0.1 } } }, "must" : { "multi_match" : { "query" : "elasticsearch", "fields" : ["title^10", "body"] } } } }, "filter": { "and" : [ { "terms" : { "tags" : ["search"] } }, { "range" : { "published_on": {"from": "2013"} } }, { "term" : { "featured" : true } } ] } } } }' JSON-based Query DSL
  23. 23. curl  -­‐X  GET  localhost:9200/articles/_search  -­‐d  '{ "query" : { "filtered" : { "query" : { "bool" : { "must" : { "match" : { "author.first_name" : { "query" : "claire", "fuzziness" : 0.1 } } }, "must" : { "multi_match" : { "query" : "elasticsearch", "fields" : ["title^10", "body"] } } } }, "filter": { "and" : [ { "terms" : { "tags" : ["search"] } }, { "range" : { "published_on": {"from": "2013"} } }, { "term" : { "featured" : true } } ] } } } }' JSON-based Query DSL
  24. 24. “Find all articles with ‘search’ in their title or body, give matches in titles higher score” Full-text Search “Find all articles from year 2013 tagged ‘search’” Structured Search See custom_score and custom_filters_score queries Custom Scoring
  25. 25. Fetch document field ➝ Pick configured analyzer ➝ Parse text into tokens ➝ Apply token filters ➝ Store into index How Search Engine Works? ResultResultsQuery How Users See Search?
  26. 26. Mapping curl -X PUT localhost:9200/articles/_mapping -d '{ "article" : { "properties" : { "title" : { "type" : "string", "analyzer" : "czech" } } } }' Configuring document properties for the search engine
  27. 27. _analyze?pretty&format=text&text=Žluťoučký+kůň+skákal+přes+potok The _analyze API [žluťoučký:0-­‐>9:<ALPHANUM>] nn2:  n[kůň:10-­‐ >13:<ALPHANUM>]nn3:   n[skákal:14-­‐>20:<ALPHANUM>] nn4:  n[přes:21-­‐ >25:<ALPHANUM>]nn5:   n[potok:26-­‐>31:<ALPHANUM>] _analyze?pretty&format=text&text=Žluťoučký+kůň+skákal+přes +potok&analyzer=czech [žluťoučk:0-­‐>9:<ALPHANUM>]n n2:  n[koň:10-­‐ >13:<ALPHANUM>]nn3:   n[skákal:14-­‐>20:<ALPHANUM>] nn5:  n[potok:26-­‐ >31:<ALPHANUM>]n _analyze?text=...&tokenizer=X&filters=A,B,C
  28. 28. Slice & Dice
  29. 29. Query Facets
  30. 30. Location Product Tim e OLAP Cube Dimensions, measures, aggregations
  31. 31. Slice Dice Drill Down / Roll Up Show me sales numbers for all products across all locations in year 2013 Show me product A sales numbers across all locations over all years Show me products sales numbers in location X over all years
  32. 32. curl -X POST 'localhost:9200/articles/_search?search_type=count&pretty' -d '{ "facets": { "tag-cloug": { "terms" : { "field" : "tags" } } } }' “Tag Cloud” With the terms Facet "facets"  :  {        "tag-­‐cloug"  :  {            "terms"  :  [  {                "term"  :  "ruby",                "count"  :  2            },  {                "term"  :  "java",                "count"  :  2            },            ...            }  ]        }    }
  33. 33. curl -X GET 'localhost:9200/scores/_search/?search_type=count&pretty' -d '{ "facets": { "scores-per-subject" : { "terms_stats" : { "key_field" : "subject", "value_field" : "score" } } } }' Statistics on Student Scores With the terms_stats Facet "facets"  :  {        "scores-­‐per-­‐subject"  :  {            "_type"  :  "terms_stats",            "missing"  :  0,            "terms"  :  [  {                "term"  :  "math",                "count"  :  4,                "total_count"  :  4,                "min"  :  25.0,                "max"  :  92.0,                "total"  :  267.0,                "mean"  :  66.75            },  ...  ]        }    }
  34. 34. Facets Terms Terms Stats Statistical Range Histogram Date Histogram Filter Query Geo Distance
  35. 35. Above &  Beyond
  36. 36. Above & Beyond Bulk operations (For indexing and search operations) Percolator (“reversed search” — alerts, classification, …) Suggesters (“Did you mean …?”) Index aliases (Grouping or “renaming” of indices) Index templates (Automatic index configuration) Monitoring API (Amount of memory used, number of operations, …) …
  37. 37. thanks!

×