• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Mastering ElasticSearch with Ruby and Tire
 

Mastering ElasticSearch with Ruby and Tire

on

  • 4,024 views

A tutorial on what is ElasticSearch and how to use it effectively in a real project. ...

A tutorial on what is ElasticSearch and how to use it effectively in a real project.
The talk discusses how to integrate a search experience in an existing application, showing all the steps from downloading&configuring elastic search, to building the UI and wire the search logic (in a Rails application).

The talk was presented at RubyConf 2013.

Statistics

Views

Total Views
4,024
Views on SlideShare
3,765
Embed Views
259

Actions

Likes
10
Downloads
36
Comments
1

10 Embeds 259

http://www.scoop.it 174
http://eventifier.com 37
http://eventifier.co 17
https://twitter.com 16
http://librosweb.es 9
https://www.rebelmouse.com 2
http://www.linkedin.com 1
http://www.eventifier.com 1
http://ec2-54-243-189-159.compute-1.amazonaws.com 1
http://darya-ld1.linkedin.biz 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • karmiq has pointed me out to: https://github.com/rubygems/rubygems.org/pull/455

    It is a real integration of ElasticSearch into RubyGems (waiting to be integrated in the RubyGems code base), it's very worth to check it. Awesome job Karel!
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Mastering ElasticSearch with Ruby and Tire Mastering ElasticSearch with Ruby and Tire Presentation Transcript

    • Mastering ElasticSearch w/ Ruby (+ Tire) Luca Bonmassar, Gild RubyConf 2013 Tuesday, 5 November 13
    • Who am I? I’m Luca Bonmassar (@openmosix) # 31 # Italian living in San Francisco (and Stockholm) I work at Gild I love building products [for fun, profit and boredom] Tuesday, 5 November 13
    • Search use case You’re building a product User generated content Let (other) users find or discover this content Tuesday, 5 November 13
    • Search is NOT easy It usually starts as but then you want to support AND, OR, NOT, double quotes on multiple fields so and then it goes like Tuesday, 5 November 13
    • And so it goes... Tuesday, 5 November 13
    • Agenda Let’s define a “pet project” Boilerplate (download, install, scaffold, config, bla bla bla, yadda, yadda, yadda) Build a website w/ simple search Build a more advanced search What next (homework) Tuesday, 5 November 13
    • Project! (for fun and no profit) Tuesday, 5 November 13
    • Project: build a full textsearch for RubyGems.org Tuesday, 5 November 13
    • RubyGems Search Architecture RubyGems.org Web Spider Elastic Search Rails App (Tire) Tuesday, 5 November 13 MongoDB
    • RubyGems crawler github.com/openmosix/rubygems-crawler 1. Download all ruby gem names from https://rubygems.org/gems 2. Use gems API to download each gem info: read JSON - write JSON (MongoDB) Tuesday, 5 November 13
    • RubyGems Crawler II Tuesday, 5 November 13
    • RubyGems Search v1 Tuesday, 5 November 13 Code: github.com/openmosix/rubygems-search Spoiler: http://rubyconf.bonmassar.it
    • ElasticSearch Tuesday, 5 November 13
    • ElasticSearch II Open source search+analytics engine Distributed [near] Realtime search Multi tenant Built on Apache Lucene REST APIs JSON documents Tuesday, 5 November 13
    • ElasticSearch III 1. Download / set up a ES cluster 2. Define settings and data mapping (opt) 3. Index Data 4. Query Index MongoDB Elastic Search {    "ok"  :  true,    "status"  :  200,    "version"  :  {        "number"  :  "0.90.5",        "lucene_version"  :  "4.4"    },    "tagline"  :  "You  Know,  for  Search" } Tuesday, 5 November 13 curl  -­‐X  GET  'h.p://localhost:9200/ ruby_gems/_search?from=0&size=25&pre.y' > Curl
    • ES download + setup > wget http://download.elasticsearch.org/ elasticsearch/elasticsearch/ elasticsearch-0.90.6.tar.gz > tar zxvf elasticsearch-0.90.6.tar.gz > sudo mv elasticsearch-0.90.6 /usr/local/ Hint #1: you need Java Hint #2: you need Oracle Java Tuesday, 5 November 13
    • ES config > ls elasticsearch-0.90.6/config/ Logging.yml: where to log, how much to log Elasticsearch.yml: all server config. Defines: Name of the cluster (change it!!!) Node parameters (master/slave, store data/router) Sharing and # replicas Paths Plugins Memory (JVM, heap, memory locking) Network config “Gateway” (cluster backup) Recovery Discovery Slow log + GC log Default options are good enough for dev env Tuesday, 5 November 13
    • ES boot + test Start: bin/elasticsearch Test: curl http://localhost:9200/ { "ok" : true, "status" : 200, "name" : "Iron Fist", "version" : { "number" : "0.90.6", "lucene_version" : "4.5.1" }, "tagline" : "You Know, for Search" } Stop: curl -XPOST 'http://localhost:9200/ _shutdown' {"cluster_name":"elasticsearch","nodes": {"jm2Z3J4dSzebjJ7Px2fAJg":{"name":"Iron Fist"}}} Tuesday, 5 November 13
    • Profit (you are now an ElasticSearch expert - go and tell the world) Tuesday, 5 November 13
    • ElasticSearch operations Create a “RubyGem” Index Defines a “RubyGem” Index data mapping Index data (e.g. upload data from MongoDB to ES index = POST) Query (= GET) Tuesday, 5 November 13
    • Tire now Re-Tire ;( A ruby gem wrapping ElasticSearch REST APIs into a powerful ruby DSL ActiveModel integration Rake tasks and utilities to load and query ElasticSearch Tuesday, 5 November 13
    • Tire setup cat “gem ‘tire’” > Gemfile && bundle install > cat config/initializers/elasticsearch.rb ... Tire::Configuration.url('http://localhost:9200') Tire.configure { logger "#{Rails.root}/log/elasticsearchqueries.log" } if ENV['ES_LOG'] Tuesday, 5 November 13
    • Define an ES index (with Tire DSL) Tuesday, 5 November 13
    • Indexing Get a record Convert it to JSON format (to_indexed_json) Push it to Elastic Search (.update_index) ...under the hood... Tuesday, 5 November 13
    • Index (all data) Naive (POST on index for each record): Use bulk updates: ...under the hood... Tuesday, 5 November 13
    • Search I Tuesday, 5 November 13
    • Search II Tuesday, 5 November 13
    • Simple Search Tuesday, 5 November 13
    • Highlight matches Text ...add some CSS... Tuesday, 5 November 13
    • Advanced Search Tuesday, 5 November 13
    • Advanced Search II Tuesday, 5 November 13
    • Advanced Search III Tuesday, 5 November 13
    • Facets Tuesday, 5 November 13
    • ES facets Tuesday, 5 November 13
    • ES facets (running) Tuesday, 5 November 13
    • Facets - UI Tuesday, 5 November 13
    • Bonsai Cool I Tuesday, 5 November 13 Search Suggesters (Did you mean... ?)
    • Bonsai Cool II Tuesday, 5 November 13 “Similar to this” (aka “More Like This” API)
    • Bonsai Cool III Tuesday, 5 November 13 Percolate API
    • Deployment I Run your own cluster Some learnings: at least 3 nodes memory profiling / GC install very good monitoring (github.com/ karmi/elasticsearch-paramedic) more RAM is (always) better Check IOPS (if on AWS) Pros: Total control Cheaper (lot cheaper) Cons: Can be a nightmare / Require dedicated devop Tuesday, 5 November 13
    • Deployment II ElasticSearch as a service http://found.no http://searchly.com http://bonsai.io Pros: Get cluster up & running in a minute Focus on dev, not troubleshooting Professional support Cons: Expensive Can be in the wrong region / hosting provider Expensive Did I say expensive? Tuesday, 5 November 13
    • Thanks! Code: github.com/openmosix/rubygems-search github.com/openmosix/rubygems-crawler Demo (will be down by the end of rubyconf): http://rubyconf.bonmassar.it Say “hi”: Luca Bonmassar - luca@gild.com github.com/openmosix twitter.com/openmosix linkedin.com/in/lucabonmassar Tuesday, 5 November 13