Elasticsearch Quick Introduction
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Elasticsearch Quick Introduction

  • 1,129 views
Uploaded on

Elasticsearch and MIT Sloan Data Analytics Hackathon ...

Elasticsearch and MIT Sloan Data Analytics Hackathon
Cambridge, MA - May 10, 2014

More in: Software , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,129
On Slideshare
1,118
From Embeds
11
Number of Embeds
1

Actions

Shares
Downloads
44
Comments
0
Likes
0

Embeds 11

http://localhost 11

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch and MIT Sloan Data Analytics Hackathon Cambridge, MA - May 10, 2014 Elasticsearch Quick Introduction
  • 2. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited About Me • Igor Motov • Developer at Elasticsearch Inc. • Github: imotov • Twitter: @imotov
  • 3. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited About Elasticsearch Inc. • Founded in 2012 By the people behind the Elasticsearch and Apache Lucene http://www.elasticsearch.com Headquarters: Amsterdam and Los Altos, CA • We provide Training (public & onsite) Development support Production support subscription (SLA)
  • 4. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited About Elasticsearch • Real time search and analytics engine JSON-oriented, Apache Lucene-based • Automatic Schema Detection Enables control of it when needed • Distributed Scales Up+Out, Highly Available • Multi-tenancy Dynamically create/delete indices • API centric Most functionality is exposed through an API
  • 5. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Basic Concepts • Cluster a group of nodes sharing the same set of indices • Node a running Elasticsearch instance (typically JVM process) • Index a set of documents of possibly different types stored in one or more shards • Type a set of documents in an index that share the same schema • Shard a Lucene index, allocated on one of the nodes
  • 6. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Basic Concepts - Document • JSON Object ! ! ! ! ! ! • Identified by index/type/id { "rank": 21, "city": "Boston", "state": "Massachusetts", "population2010": 617594, "land_area": 48.277, "density": 12793, "ansi": 619463, "location": { "lat": 42.332, "lon": 71.0202 }, "abbreviation": "MA" }
  • 7. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Downloading elasticsearch • http://www.elasticsearch.org/download/ Windows Everything else
  • 8. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited What’s in a distribution? . ├── LICENSE.txt ├── NOTICE.txt ├── README.textile ├── bin │   ├── elasticsearch │   ├── elasticsearch.in.sh │   └── plugin ├── config │   ├── elasticsearch.yml │   └── logging.yml ├── data │   └── elasticsearch ├── lib │   ├── elasticsearch-x.y.z.jar │   ├── ... │   └── └── logs    ├── elasticsearch.log    └── elasticsearch_index_search_slowlog.log executable scripts node config files data storage libs log files
  • 9. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Configuration (multicast) • Configuration config/elasticsearch.yml cluster.name: "elasticsearch-imotov" unique name
  • 10. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Configuration (stand-alone) • Configuration config/elasticsearch.yml cluster.name: "elasticsearch-imotov" network.host: "127.0.0.1" discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["localhost:9300", "localhost:9301", “localhost:9302"] unique name listen only on localhost disable multicast search for other nodes on localhost
  • 11. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Starting elasticsearch • Foreground ! ! • Background $ bin/elasticsearch $ bin/elasticsearch -d
  • 12. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Is it running? { "status" : 200, "name" : "Kamal", "version" : { "number" : "1.1.1", "build_hash" : "f1585f096d3f3985e73456debdc1a0745f512bbc", "build_timestamp" : "2014-04-16T14:27:12Z", "build_snapshot" : false, "lucene_version" : "4.7" }, "tagline" : "You Know, for Search" } $ curl -XGET "http://localhost:9200/?pretty"
  • 13. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Communicating with Elasticsearch • REST API Curl Ruby Python PHP Perl JavaScript (community supported) • Binary Protocol Java
  • 14. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Pick your client • Java included in distribution • Ruby, PHP, Perl, Python http://www.elasticsearch.org/blog/unleash-the-clients-ruby- python-php-perl/ • Everything Else http://www.elasticsearch.org/guide/clients/
  • 15. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Indexing a document $ curl -XPUT "http://localhost:9200/test-data/cities/21" -d '{ "rank": 21, "city": "Boston", "state": "Massachusetts", "population2010": 617594, "land_area": 48.277, "density": 12793, "ansi": 619463, "location": { "lat": 42.332, "lon": 71.0202 }, "abbreviation": "MA" }' {"ok":true,"_index":"test-data","_type":"cities","_id":"21","_version":1}
  • 16. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Getting a document { "_index" : "test-data", "_type" : "cities", "_id" : "21", "_version" : 1, "exists" : true, "_source" : { "rank": 21, "city": "Boston", "state": "Massachusetts", "population2010": 617594, "land_area": 48.277, "density": 12793, "ansi": 619463, "location": { "lat": 42.332, "lon": 71.0202 }, "abbreviation": "MA" } } $ curl -XGET "http://localhost:9200/test-data/cities/21?pretty"
  • 17. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Updating a document $ curl -XPUT "http://localhost:9200/test-data/cities/21" -d '{ "rank": 21, "city": "Boston", "state": "Massachusetts", "population2010": 617594, "population2012": 636479, "land_area": 48.277, "density": 12793, "ansi": 619463, "location": { "lat": 42.332, "lon": 71.0202 }, "abbreviation": "MA" }' {"ok":true,"_index":"test-data","_type":"cities","_id":"21","_version":2}
  • 18. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Searching $ curl -XGET 'http://localhost:9200/test-data/cities/_search?pretty' -d '{ "query": { "match": { "city": "Boston" } } }'
  • 19. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Searching { "took" : 5, "timed_out" : false, "_shards" : { "total" : 1, "successful" : 1, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 6.1357985, "hits" : [ { "_index" : "test-data", "_type" : "cities", "_id" : "21", "_score" : 6.1357985, "_source" : {"rank":"21","city":"Boston",...} } ] } }
  • 20. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Range Queries $ curl -XGET "http://localhost:9200/test-data/cities/_search?pretty" -d '{ "query": { "range": { "population2012": { "from": 500000, "to": 1000000 } } } }'
  • 21. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Boolean Queries $ curl -XGET "http://localhost:9200/test-data/cities/_search?pretty" -d '{ "query": { "bool": { "should": [{ "match": { "state": "Texas"} }, { "match": { "state": "California"} }], "must": { "range": { "population2012": { "from": 500000, "to": 1000000 } } }, "minimum_should_match": 1 } } }'
  • 22. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited MatchAll Query $ curl -XGET "http://localhost:9200/test-data/cities/_search?pretty" -d '{ "query": { "match_all": { } } }'
  • 23. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Sorting and Paging $ curl -XGET "http://localhost:9200/test-data/cities/_search?pretty" -d '{ "query": { "match_all": { } }, "sort": [ {"state": {"order": "asc"}}, {"population2010": {"order": "desc"}} ], "from": 0, "size": 20 }'
  • 24. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Analysis • By default string are - Divided into words (tokens) - All tokens are converted to lower-case
  • 25. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Analysis Example • “Elasticsearch is a powerful open source search and analytics engine.” 1. elasticsearch 2. is 3. a 4. powerful 5. open 6. source 7. search 8. and 9. analytics 10. engine
  • 26. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Customizing the mapping curl -XPUT 'http://localhost:9200/my_index/' -d '{ "settings": { "index": { "number_of_shards": 1, "number_of_replicas": 0 } }, "mappings": { "my_type": { "properties": { "description": { "type": "string" }, "sku": { "type": "string", "index": "not_analyzed" }, "count": { "type": "integer" }, "price": { "type": "float" }, "location": { "type": "geo_point" } } } } }' exact match analyzed text geo location
  • 27. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch Reference • http://www.elasticsearch.org/guide/
  • 28. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Ideas for hackathon • Explore data wikipedia twitter enron emails • Play with Kibana • Build Elasticsearch plugins • Get prizes
  • 29. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch Meetup http://www.meetup.com/Elasticsearch-Boston/
  • 30. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited We are hiring http://www.elasticsearch.com/about/jobs/