Your SlideShare is downloading. ×
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Hopper Elasticsearch Hackathon
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Hopper Elasticsearch Hackathon

2,943

Published on

Published in: Technology
0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,943
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
8
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch Quick Introduction Hopper Elasticsearch Hackathon Boston, MA - Sep 27, 2013
  • 2. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited About Me • Igor Motov • Developer at Elasticsearch Inc. • Github: imotov • Twitter: @imotov
  • 3. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited About Elasticsearch Inc. • Founded in 2012 • By the people behind the Elasticsearch and Apache Lucene • http://www.elasticsearch.com • Headquarters:Amsterdam and Los Altos, CA • We provide • Training (public & onsite) • Development support • Production support subscription (SLA)
  • 4. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited About Elasticsearch • Real time search and analytics engine • JSON-oriented,Apache Lucene-based • Automatic Schema Detection • Enables control of it when needed • Distributed • Scales Up+Out, Highly Available • Multi-tenancy • Dynamically create/delete indices • API centric • Most functionality is exposed through an API
  • 5. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Basic Concepts • Cluster • a group of nodes sharing the same set of indices • Node • a running Elasticsearch instance (typically JVM process) • Index • a set of documents of possibly different types • stored in one or more shards • Type • a set of documents in an index that share the same schema • Shard • a Lucene index, allocated on one of the nodes
  • 6. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Basic Concepts - Document • JSON Object • Identified by index/type/id { "rank": 21, "city": "Boston", "state": "Massachusetts", "population2010": 617594, "land_area": 48.277, "density": 12793, "ansi": 619463, "location": { "lat": 42.332, "lon": 71.0202 }, "abbreviation": "MA" }
  • 7. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Downloading elasticsearch • http://www.elasticsearch.org/download/ Windows Everything else
  • 8. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited What’s in a distribution? . ├── LICENSE.txt ├── NOTICE.txt ├── README.textile ├── bin │   ├── elasticsearch │   ├── elasticsearch.in.sh │   └── plugin ├── config │   ├── elasticsearch.yml │   └── logging.yml ├── data │   └── elasticsearch ├── lib │   ├── elasticsearch-x.y.z.jar │   ├── ... │   └── └── logs    ├── elasticsearch.log    └── elasticsearch_index_search_slowlog.log executable scripts node config files data storage libs log files
  • 9. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Configuration (multicast) • Configuration config/elasticsearch.yml cluster.name: "elasticsearch-imotov" unique name
  • 10. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Configuration (stand-alone) • Configuration config/elasticsearch.yml cluster.name: "elasticsearch-imotov" network.host: "127.0.0.1" discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["localhost"] unique name listen only on localhost disable multicast search for other nodes on localhost
  • 11. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Starting elasticsearch • Foreground • Background $ bin/elasticsearch -f $ bin/elasticsearch
  • 12. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Is it running? { "ok" : true, "status" : 200, "name" : "Hensley Fargus", "version" : { "number" : "0.90.5", "build_hash" : "c8714e8e0620b62638f660f6144831792b9dedee", "build_timestamp" : "2013-09-17T12:50:20Z", "build_snapshot" : false, "lucene_version" : "4.4" }, "tagline" : "You Know, for Search" } $ curl -XGET "http://localhost:9200/?pretty"
  • 13. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Communicating with Elasticsearch • REST API • Curl • Ruby • Python • PHP • Perl • JavaScript (community supported) • Binary Protocol • Java
  • 14. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Pick your client • Java • included in distribution • Ruby, PHP, Perl, Python • http://www.elasticsearch.org/blog/unleash-the-clients- ruby-python-php-perl/ • Everything Else • http://www.elasticsearch.org/guide/clients/
  • 15. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Indexing a document $ curl -XPUT "http://localhost:9200/test-data/cities/21" -d '{ "rank": 21, "city": "Boston", "state": "Massachusetts", "population2010": 617594, "land_area": 48.277, "density": 12793, "ansi": 619463, "location": { "lat": 42.332, "lon": 71.0202 }, "abbreviation": "MA" }' {"ok":true,"_index":"test-data","_type":"cities","_id":"21","_version":1}
  • 16. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Getting a document { "_index" : "test-data", "_type" : "cities", "_id" : "21", "_version" : 1, "exists" : true, "_source" : { "rank": 21, "city": "Boston", "state": "Massachusetts", "population2010": 617594, "land_area": 48.277, "density": 12793, "ansi": 619463, "location": { "lat": 42.332, "lon": 71.0202 }, "abbreviation": "MA" } } $ curl -XGET "http://localhost:9200/test-data/cities/21?pretty"
  • 17. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Updating a document $ curl -XPUT "http://localhost:9200/test-data/cities/21" -d '{ "rank": 21, "city": "Boston", "state": "Massachusetts", "population2010": 617594, "population2012": 636479, "land_area": 48.277, "density": 12793, "ansi": 619463, "location": { "lat": 42.332, "lon": 71.0202 }, "abbreviation": "MA" }' {"ok":true,"_index":"test-data","_type":"cities","_id":"21","_version":2}
  • 18. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Searching $ curl -XGET 'http://localhost:9200/test-data/cities/_search?pretty' -d '{ "query": { "match": { "city": "Boston" } } }'
  • 19. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Searching { "took" : 5, "timed_out" : false, "_shards" : { "total" : 1, "successful" : 1, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 6.1357985, "hits" : [ { "_index" : "test-data", "_type" : "cities", "_id" : "21", "_score" : 6.1357985, "_source" : {"rank":"21","city":"Boston",...} } ] } }
  • 20. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Range Queries $ curl -XGET "http://localhost:9200/test-data/cities/_search?pretty" -d '{ "query": { "range": { "population2012": { "from": 500000, "to": 1000000 } } } }'
  • 21. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Boolean Queries $ curl -XGET "http://localhost:9200/test-data/cities/_search?pretty" -d '{ "query": { "bool": { "should": [{ "match": { "state": "Texas"} }, { "match": { "state": "California"} }], "must": { "range": { "population2012": { "from": 500000, "to": 1000000 } } }, "minimum_should_match": 1 } } }'
  • 22. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited MatchAll Query $ curl -XGET "http://localhost:9200/test-data/cities/_search?pretty" -d '{ "query": { "match_all": { } } }'
  • 23. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Sorting and Paging $ curl -XGET "http://localhost:9200/test-data/cities/_search?pretty" -d '{ "query": { "match_all": { } }, "sort": [ {"state": {"order": "asc"}}, {"population2010": {"order": "desc"}} ], "from": 0, "size": 20 }'
  • 24. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Analysis • By default string are • Divided into words (tokens) • All tokens are converted to lower-case • English stop words are removed • a, an, and, are, as, at, be, but, by, for, if, in, into, is, it, no, not, of, on, or, such, that, the, their, then, there, these, they, this, to, was, will, with
  • 25. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Analysis Example • “Elasticsearch is a powerful open source search and analytics engine.” 1. elasticsearch 2. powerful 3. open 4. source 5. search 6. analytics 7. engine
  • 26. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Disabling stopwords in elasticsearch.yml index: analysis: analyzer: default: type: "custom" tokenizer: "standard" filters: ["lowercase"]
  • 27. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Customizing the mapping curl -XPUT 'http://localhost:9200/my_index/' -d '{ "settings": { "index": { "number_of_shards": 1, "number_of_replicas": 0 } }, "mappings": { "my_type": { "properties": { "description": { "type": "string" }, "sku": { "type": "string", "index": "not_analyzed" }, "count": { "type": "integer" }, "price": { "type": "float" }, "location": { "type": "geo_point" } } } } }' exact match analyzed text geo location
  • 28. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch Reference • http://www.elasticsearch.org/guide/
  • 29. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Ideas for hackathon • Explore data • wikipedia • twitter • enron emails • Play with Kibana • Build Elasticsearch plugins • Get elasticsearch T-shirt
  • 30. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch Meetup http://www.meetup.com/Elasticsearch-Boston/
  • 31. Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited We are hiring http://www.elasticsearch.com/about/jobs/

×