• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Mastering ElasticSearch With Ruby (+Tire)
 

Mastering ElasticSearch With Ruby (+Tire)

on

  • 385 views

At RubyConf 2013, Co-founder and Chief Product and Technology Officer, Luca Bonmassar, covered how to build an ElasticSearch using the Ruby Tire Library. Here are the slides from the presentation.

At RubyConf 2013, Co-founder and Chief Product and Technology Officer, Luca Bonmassar, covered how to build an ElasticSearch using the Ruby Tire Library. Here are the slides from the presentation.

Statistics

Views

Total Views
385
Views on SlideShare
371
Embed Views
14

Actions

Likes
1
Downloads
8
Comments
0

1 Embed 14

http://192.168.6.184 14

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Mastering ElasticSearch With Ruby (+Tire) Mastering ElasticSearch With Ruby (+Tire) Presentation Transcript

    • Mastering ElasticSearch w/ Ruby (+ Tire) Luca Bonmassar, Gild RubyConf 2013 Tuesday, 5 November 13
    • Who am I? I’m Luca Bonmassar (@openmosix) # 31 # Italian living in San Francisco (and Stockholm) I work at Gild I love building products [for fun, profit and boredom] Tuesday, 5 November 13
    • Search use case You’re building a product User generated content Let (other) users find or discover this content Tuesday, 5 November 13
    • Search is NOT easy It usually starts as but then you want to support AND, OR, NOT, double quotes on multiple fields so and then it goes like Tuesday, 5 November 13
    • And so it goes... Tuesday, 5 November 13
    • Agenda Let’s define a “pet project” Boilerplate (download, install, scaffold, config, bla bla bla, yadda, yadda, yadda) Build a website w/ simple search Build a more advanced search What next (homework) Tuesday, 5 November 13
    • Project! (for fun and no profit) Tuesday, 5 November 13
    • Project: build a full textsearch for RubyGems.org Tuesday, 5 November 13
    • RubyGems Search Architecture RubyGems.org Web Spider Elastic Search Rails App (Tire) Tuesday, 5 November 13 MongoDB
    • RubyGems crawler github.com/openmosix/rubygems-crawler 1. Download all ruby gem names from https://rubygems.org/gems 2. Use gems API to download each gem info: read JSON - write JSON (MongoDB) Tuesday, 5 November 13
    • RubyGems Crawler II Tuesday, 5 November 13
    • RubyGems Search v1 Tuesday, 5 November 13 Code: github.com/openmosix/rubygems-search Spoiler: http://rubyconf.bonmassar.it
    • ElasticSearch Tuesday, 5 November 13
    • ElasticSearch II Open source search+analytics engine Distributed [near] Realtime search Multi tenant Built on Apache Lucene REST APIs JSON documents Tuesday, 5 November 13
    • ElasticSearch III 1. Download / set up a ES cluster 2. Define settings and data mapping (opt) 3. Index Data 4. Query Index MongoDB Elastic Search {    "ok"  :  true,    "status"  :  200,    "version"  :  {        "number"  :  "0.90.5",        "lucene_version"  :  "4.4"    },    "tagline"  :  "You  Know,  for  Search" } Tuesday, 5 November 13 curl  -­‐X  GET  'h.p://localhost:9200/ ruby_gems/_search?from=0&size=25&pre.y' > Curl
    • ES download + setup > wget http://download.elasticsearch.org/ elasticsearch/elasticsearch/ elasticsearch-0.90.6.tar.gz > tar zxvf elasticsearch-0.90.6.tar.gz > sudo mv elasticsearch-0.90.6 /usr/local/ Hint #1: you need Java Hint #2: you need Oracle Java Tuesday, 5 November 13
    • ES config > ls elasticsearch-0.90.6/config/ Logging.yml: where to log, how much to log Elasticsearch.yml: all server config. Defines: Name of the cluster (change it!!!) Node parameters (master/slave, store data/router) Sharing and # replicas Paths Plugins Memory (JVM, heap, memory locking) Network config “Gateway” (cluster backup) Recovery Discovery Slow log + GC log Default options are good enough for dev env Tuesday, 5 November 13
    • ES boot + test Start: bin/elasticsearch Test: curl http://localhost:9200/ { "ok" : true, "status" : 200, "name" : "Iron Fist", "version" : { "number" : "0.90.6", "lucene_version" : "4.5.1" }, "tagline" : "You Know, for Search" } Stop: curl -XPOST 'http://localhost:9200/ _shutdown' {"cluster_name":"elasticsearch","nodes": {"jm2Z3J4dSzebjJ7Px2fAJg":{"name":"Iron Fist"}}} Tuesday, 5 November 13
    • Profit (you are now an ElasticSearch expert - go and tell the world) Tuesday, 5 November 13
    • ElasticSearch operations Create a “RubyGem” Index Defines a “RubyGem” Index data mapping Index data (e.g. upload data from MongoDB to ES index = POST) Query (= GET) Tuesday, 5 November 13
    • Tire now Re-Tire ;( A ruby gem wrapping ElasticSearch REST APIs into a powerful ruby DSL ActiveModel integration Rake tasks and utilities to load and query ElasticSearch Tuesday, 5 November 13
    • Tire setup cat “gem ‘tire’” > Gemfile && bundle install > cat config/initializers/elasticsearch.rb ... Tire::Configuration.url('http://localhost:9200') Tire.configure { logger "#{Rails.root}/log/elasticsearchqueries.log" } if ENV['ES_LOG'] Tuesday, 5 November 13
    • Define an ES index (with Tire DSL) Tuesday, 5 November 13
    • Indexing Get a record Convert it to JSON format (to_indexed_json) Push it to Elastic Search (.update_index) ...under the hood... Tuesday, 5 November 13
    • Index (all data) Naive (POST on index for each record): Use bulk updates: ...under the hood... Tuesday, 5 November 13
    • Search I Tuesday, 5 November 13
    • Search II Tuesday, 5 November 13
    • Simple Search Tuesday, 5 November 13
    • Highlight matches Text ...add some CSS... Tuesday, 5 November 13
    • Advanced Search Tuesday, 5 November 13
    • Advanced Search II Tuesday, 5 November 13
    • Advanced Search III Tuesday, 5 November 13
    • Facets Tuesday, 5 November 13
    • ES facets Tuesday, 5 November 13
    • ES facets (running) Tuesday, 5 November 13
    • Facets - UI Tuesday, 5 November 13
    • Bonsai Cool I Tuesday, 5 November 13 Search Suggesters (Did you mean... ?)
    • Bonsai Cool II Tuesday, 5 November 13 “Similar to this” (aka “More Like This” API)
    • Bonsai Cool III Tuesday, 5 November 13 Percolate API
    • Deployment I Run your own cluster Some learnings: at least 3 nodes memory profiling / GC install very good monitoring (github.com/ karmi/elasticsearch-paramedic) more RAM is (always) better Check IOPS (if on AWS) Pros: Total control Cheaper (lot cheaper) Cons: Can be a nightmare / Require dedicated devop Tuesday, 5 November 13
    • Deployment II ElasticSearch as a service http://found.no http://searchly.com http://bonsai.io Pros: Get cluster up & running in a minute Focus on dev, not troubleshooting Professional support Cons: Expensive Can be in the wrong region / hosting provider Expensive Did I say expensive? Tuesday, 5 November 13
    • Thanks! Code: github.com/openmosix/rubygems-search github.com/openmosix/rubygems-crawler Demo (will be down by the end of rubyconf): http://rubyconf.bonmassar.it Say “hi”: Luca Bonmassar - luca@gild.com github.com/openmosix twitter.com/openmosix linkedin.com/in/lucabonmassar Tuesday, 5 November 13