Finding Anything:  Real-time Search with IndexTank
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Finding Anything: Real-time Search with IndexTank

  • 1,307 views
Uploaded on

Presentation given at East Bay Ruby meetup group on 4/19/2011 @ U.C. Berkeley

Presentation given at East Bay Ruby meetup group on 4/19/2011 @ U.C. Berkeley

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,307
On Slideshare
1,307
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
2
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Finding anything: Real-time search with IndexTank
      Tim Spence April 19, 2011
  • 2. About the Presenter Tim Spence
    • Senior Infrastructure Engineer at MedHelp ( http://www.medhelp.org/ )
    • 3. Former .NET developer
    • 4. Recently converted to Ruby
    • 5. In love with Open Source Software
    • 6. More at http://whyhello.im/tim
  • 7. Agenda
    • State of search today
    • 8. Quick survey: how much time/effort did YOU spend implementing search on your webapp?
    • 9. Examples of services that need improved search
    • 10. IndexTank to the rescue
    • 11. Case study: reddit.com
  • 12. Agenda, continued
    • How I found out about IndexTank
    • 13. Two apps I built with IndexTank
    • 14. Live Demo
  • 15.  
  • 16. The State of Search Today
    • Not well implemented at all
      • Search works, but...
      • 17. Barely
    • How many pages of results do you typically browse through before finding what you were looking for?
    • 18. Or do you give up and head for google site search instead?
  • 19. Survey Time!
    • How much time/effort did YOU spend implementing search on your webapp?
    • 20. How many times have you iterated on your search feature?
    • 21. When was the last time someone thanked you for building a powerful, reliable search feature for your webapp?
  • 22. My Opinion
    • Search as an in-app feature is an afterthought
    • 23. Minimal implementation is the norm
    • 24. If it wasn't for MySQL/MS-SQL full text indexing, most apps probably wouldn't even have a search feature
    • 25. Most good web apps don't make it easy for users to find specific content outside of predetermined navigation
  • 26. Let's pick on some apps!
    • These are companies with great products, but their search comes up short
    • 27. Don't worry–they can take it!
  • 28. App #1: Github
  • 29. App #1: Github
  • 30. App #1: Github
    • Interface is decent
      • Search repos, code, users, or everything
      • 31. Search by language
    • However...
      • Can't do much with results but browse
      • 32. Check out this example
  • 33. App #1: Github
  • 34. App #1: Github
    • Why these results aren't so hot
      • Can't search by most recently maintained
      • 35. Can't search by most popular (most watched)
      • 36. Are you ready to browse 1,297 results?
    • Advanced search capabilities exist, but not the best interface
      • recency/popularity implemented, but require specific arguments
  • 37. App #2: Amazon Web Services
    • ”Hey, I bet I can find an AMI from the community for the exact EC2 setup I need”
    • 38. Fact: probably not
  • 39. App #2: Amazon Web Services
  • 40. App #2: Amazon Web Services
    • Notice something missing?
      • No search
      • 41. Only sort by date, title
    • Ready to browse 934 results?
      • I'd rather build my own AMI
    • Incredible missed opportunity
  • 44. Fact: Github & Amazon aren't the only ones
    • Lots of good web services
    • 45. Massive quantities of quality content
    • 46. Unfortunately not discoverable in meaningful ways
  • 47. Interlude: Sites with great search
    • Foodspotting
    • Medhelp
      • Content category
      • 50. Promoted content
    • Other sites I overlooked? Whose search do you like?
  • 51. What was the point of that last slide?
    • Search can be useful if it is valued as a feature
    • 52. Any company willing to invest in the resources can build and host a high quality search engine
    • 53. However, must you roll your own?
  • 54. Enter Search as a Service
    • No need for you to invest in additional infrastructure
    • 55. No need to reinvent the wheel
      • Search is a solved problem
      • 56. Let the experts refine it
  • 57. IndexTank to the rescue!
    • Hosted–no load on your infrastructure
    • 58. Powerful
      • We'll get into the details next
    • Always Improving
      • Search IS their product
    • Freemium
    • 59. Easy to implement
  • 60. Let's talk features
    • Real-time search
      • Real-time indexing–results immediately available
    • Custom scoring
    • 61. Autocomplete
    • 62. Faceting
    • 63. Geo search
    • 64. Advanced text search
  • 65.
    • Real-time search
    • Real-time indexing
      • results immediately available
    • Index multiple docs/sec
    • 66. Overwrite existing docs as you wish
      • Changes also immediately available
  • 67. Custom Scoring
    • Implementer has full control over how results are returned
    • 68. Choose which fields are searched
    • 69. Use pre-written scoring functions
    • 70. Or write your own
  • 71. Custom Scoring
  • 72. Everyone loves autocomplete
    • Saves users time
    • 73. Potentially avoids spelling errors
      • Not for hunters/peckers
    • Adds a degree of intelligence to the search process
  • 74. Faceting
    • Does it make sense for you to categorize documents in your index?
      • In all cases, YES
    • Consider your advanced users and the narrow results they seek
      • Don't make anyone sift through irrelevant results
  • 75. Faceting
  • 76. Geo
    • It's 2011
      • Location is more relevant than ever before
      • 77. Mobile is skyrocketing–every client has a GPS
    • IndexTank has built-in geo proximity search capability
  • 78. Geo
  • 79. Advanced Text Search (Beta)
    • Fuzzy search (Did you mean...?)
    • 80. Stemming
      • Alternate word forms (tense, possession, etc...)
    • Alternate spellings
      • Misspellings
  • 81. Other Benefits
    • Zero maintenance
    • 82. Scalability included for free
    • 83. Easy implementation
      • Clients available in many languages
      • 84. Excellent documentation–Let's check it out
    • Excellent support
      • Humans or bots? You decide
    • Dog food: their site search is done well
  • 85.  
  • 86. Case Study: reddit.com
    • High traffic news aggregator (> 1.0E9 pvs/mo) with tons of content
    • 87. Who remembers how bad reddit's search was?
      • When it even worked
    • Can't blame them for trying
      • Many attempts, but none worked
    • IndexTank excelled in all areas
    • 88. Let's check it out now
  • 89. My experience with IndexTank
    • Discovered through Heroku/IndexTank contest
    • 90. Built my first irl Rails app in an afternoon/evening w/ fellow hacker Chris Saylor (@cwsaylor)
    • 91. Didn't win the contest but learned how easy it is to quickly create highly targeted search
  • 92. App #1: Toxosis
    • Searchable database of toxic release data supplied by U.S. E.P.A.
    • 93. Hosted at http://toxosis.heroku.com/
    • 94. Search enabled on many fields including city/state/zip, toxin
    • 95. Additional fields can be added to index
      • When I have time, of course...
  • 96. More personal backstory
    • Still in the business of reinventing myself as a Rails developer
    • 97. How to get a Rails gig? Develop an app multiple Rails apps and show it them off
    • 98. Opportunities are everywhere–contests, hackathons, and weekend hacks for developer community
  • 99. App #2: SXSWdex
    • Searchable database of 2011 SXSW attendees
    • 100. Hosted at http://sxswdex.heroku.com/
    • 101. Design goal: do a better job than SXSW official site
    • 102. Search within bio, company, location, name
    • 103. Facets: company, city/state
  • 104. The moment we've all been waiting for
    • Let's build an app!
  • 105. Questions?
    • Q&A time with an IndexTank engineer