Adding High Performance Search to your Grails App


Published on

These are slides from my presentation to the Grails Cafe Centro meetup in San Francisco on Aug 16th 2011.

Most of the session was practical, so the slides are brief and are mostly useful for the links they contain.

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Adding High Performance Search to your Grails App

  1. 1. Adding High Performance Search to your Grails App  Adam @Creeger API Architect,Ticketfly Grails Cafe Centro Meetup, Aug 16th 2011 San Francisco, CA
  2. 2. The next hour or so... <ul><ul><li>Light on the slides </li></ul></ul><ul><ul><li>Heavy on hands-on coding </li></ul></ul><ul><ul><li>Some audience participation! </li></ul></ul><ul><ul><li>Intro to ElasticSearch </li></ul></ul><ul><ul><li>Installing and using a local ElasticSearch instance </li></ul></ul><ul><ul><li>Adding basic search into a Grails app </li></ul></ul><ul><ul><li>Adding some more search functionality </li></ul></ul><ul><ul><li>Seeing the performance of ElasticSearch </li></ul></ul><ul><ul><li>ElasticSearch in the cloud (kind of) </li></ul></ul>
  3. 3. What is ElasticSearch? <ul><ul><li>High performance, distributed search server </li></ul></ul><ul><ul><li>Based on Lucene </li></ul></ul><ul><ul><li>Highly available </li></ul></ul><ul><ul><li>Schema optional </li></ul></ul><ul><ul><li>RESTful </li></ul></ul><ul><ul><li>Created by Shay Banon, author of Compass. </li></ul></ul><ul><ul><li>Very active open source project ( github impact graph ) </li></ul></ul><ul><ul><li>Still in &quot;beta&quot; </li></ul></ul>
  4. 4. What isn't ElasticSearch? <ul><ul><li>Quite as mature as Solr </li></ul></ul><ul><ul><li>100% stable </li></ul></ul><ul><ul><li>A replacement for your NoSQL layer </li></ul></ul>
  5. 5. My experience with ElasticSearch... <ul><ul><li>Ticketfly uses ElasticSearch to power our API and order search - about 1 million hits a day, 40ms average response time. </li></ul></ul><ul><ul><li>Small-ish indexes with a few million records </li></ul></ul><ul><ul><li>Not as intensive as many other uses of ElasticSearch. </li></ul></ul>
  6. 6. Hands on: Installing and Using ElasticSearch What you'll need: A unique ElasticSearch cluster name
  7. 7. Demo: Adding ElasticSearch to a Grails Application What you'll see: Installing the Grails plugin (a custom fork for now) Implement basic, but useful search
  8. 8. Demo: Making our Search more useful What you'll see: The power of the query string ElasticSearch's Groovy API Geo searching
  9. 9. Demo: Performance What you'll see: Bulk Indexing The performance of ElasticSearch compared with MySQL/GORM
  10. 10. Hands on: Using ES in the Cloud (kind of) What you'll need: The ElasticSearch installation you created earlier All be on the same network The same cluster name! Web App needs correct config:
  11. 11. Some tips for using ElasticSearch <ul><li>Use Aliases While Bulk Indexing </li></ul><ul><li>To avoid downtime while doing a Bulk Index: </li></ul><ul><li>1. Create an index with a Timestamped name: </li></ul><ul><li>e.g. org.cafecentro_20110814_145500 </li></ul><ul><li>2. Once the indexing is complete, use the Alias API to normalize the          name </li></ul><ul><li>e.g Alias org.cafecentro ->  org.cafecentro_20110814_145500 </li></ul><ul><li>which means: </li></ul><ul><li>http://localhost:9200/org.cafecentro/event/1234  </li></ul><ul><li>is equivalent to </li></ul><ul><li>http://localhost:9200/org.cafecentro_20110814_145500/event/1234 </li></ul><ul><li>3. Using the timestamp, clean up the old indexes </li></ul>
  12. 12. Some tips for using ElasticSearch <ul><li>Some General Tips </li></ul><ul><ul><li>Include the version number in the name your cluster. This means you can create a new cluster for an upgrade - don't rely on indexes being backwards-compatible. </li></ul></ul><ul><ul><li>Test your upgrades thoroughly in a staging environment. </li></ul></ul><ul><ul><li>Many concepts are very similar to Lucene - plenty of great documentation available. </li></ul></ul><ul><ul><li>Before using in production, understand how &quot;snapshots&quot; work and how to configure them:  and  will help. </li></ul></ul><ul><ul><li>The Google Group/Mailing List is very active. Many questions answered by Shay himself. </li></ul></ul>
  13. 13. Useful Resources <ul><ul><li> </li></ul></ul><ul><ul><li> </li></ul></ul><ul><ul><li>   </li></ul></ul><ul><ul><li> </li></ul></ul><ul><ul><li> </li></ul></ul><ul><li>From this presentation: </li></ul><ul><ul><li>  - Fork of the grails elasticsearch plugin </li></ul></ul><ul><ul><li> - The demo app used during this session </li></ul></ul>
  14. 14. Thank you! Adam @Creeger API Architect,Ticketfly