Scalling Solr in the Cloud - OSCON Data 2011
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Scalling Solr in the Cloud - OSCON Data 2011

on

  • 2,569 views

Solr, an open source enterprise search server, scales very well within an index (vertical scaling). It is when you have multiple indexes (horizontal scaling) that it starts to get hairy, which happens ...

Solr, an open source enterprise search server, scales very well within an index (vertical scaling). It is when you have multiple indexes (horizontal scaling) that it starts to get hairy, which happens a lot when you are hosting a cloud based solution for multiple users. In this session we will discuss these issue as well as the techniques of how to overcome them in-depth.

By: Andy Blyler & Lindsay Snider

Statistics

Views

Total Views
2,569
Views on SlideShare
2,310
Embed Views
259

Actions

Likes
3
Downloads
27
Comments
0

9 Embeds 259

http://www.oscon.com 178
http://lanyrd.com 62
http://app-dev.crowdbase.com 7
http://twitter.com 5
http://app-development.crowdbase.com 3
http://a0.twimg.com 1
http://crowdbase.com 1
http://staging.conferize.com 1
http://ruben.i.conferize.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Scalling Solr in the Cloud - OSCON Data 2011 Presentation Transcript

  • 1. Scaling Solr in the Cloud By @ablyler & @LindsaySnider FROM @NutshellCRM www.nutshell.com
  • 2. AgendaAbout the SpeakersAbout NutshellNutshell and SolrSolr Resource UsageScaling MethodsQuestions and Answers
  • 3. About the SpeakersBitLeap / Barracuda Networks Developers on the Backup Appliance / Cloud Scaled databases and storage systems Used SugarCRM & Salesforce.com
  • 4. About NutshellWeb and Mobile CRM applicationHeavy use of OpenSource technologies: Gentoo Nginx / PHP / ZendFramework / jQuery MySQL / Solr / Gearman Jenkins / Redmine / Cacti / Nagios
  • 5. Nutshell and SolrHeavy use of Solr for searching, table views, andde-duplicationUsed for searching / display: Accounts, Competitors, Contacts, Leads, Products, Sources, Teams, and UsersUsed for de-duplication: Accounts, Contacts
  • 6. Reads vs Writes 7000 5250 3500 1750 0Jul 20, 2011 3:00 AM Jul 20, 2011 6:00 AM Jul 20, 2011 9:00 AM Jul 20, 2011 12:00 PM Jul 20, 2011 3:00 PM Jul 20, 2011 6:00 PM Jul 20, 2011 9:00 PM Jul 21, 2011 12:00 AM Writes Reads Time Average Read Query Time: 2.11ms
  • 7. Demowww.nutshell.com
  • 8. Computer Resources CPU Disk I/O Disk RAM Storage
  • 9. CPU Disk I/O DiskRAM Storage
  • 10. SOLR ResourcesHeavy CPU Disk I/O Disk RAM StorageLight
  • 11. SOLR ResourcesHeavy CPU Disk I/O Disk RAM StorageLight
  • 12. SOLR ResourcesHeavy CPU Disk I/O Disk RAM StorageLight
  • 13. SOLR ResourcesHeavy CPU Disk I/O Disk RAM StorageLight
  • 14. SOLR ResourcesHeavy CPU Disk I/O Disk RAM StorageLight
  • 15. Dark Ages Age of Enlightenment Modern Era Today Auto Provisioning Setup new Solr Jetty app Create MySQL database Populate MySQL / Solr with demo data Send welcome email
  • 16. Dark Ages Age of Enlightenment Modern Era TodaySeparate Jetty per Customer Uses a ton of memory Separate schema / Solr for each customer Ran into upper limit morning before launch
  • 17. Dark Ages Age of Enlightenment Modern Era Today Auto provisioning Separate Jetty app for each customer
  • 18. Dark Ages Age of Enlightenment Modern Era Today5025 0 Auto provisioning Separate Jetty app for each customer
  • 19. Dark Ages Age of Enlightenment Modern Era Today Solr Core per Customer Allows for management of Solr on a per customer basis: creating / stopping Contained within a single Jetty app Shared schema between all cores Easily managed via simple HTTP API
  • 20. Dark Ages Age of Enlightenment Modern Era Today Fallback to MySQL Landing page of application Allows for graceful handling when Solr is down Abstracted within the application library
  • 21. Dark Ages Age of Enlightenment Modern Era Today Sun JVM to IcedTea JVM IcedTea JVM uses less memory than Sun JVM
  • 22. Dark Ages Age of Enlightenment Modern Era Today Separate Solr core per customer Fallback to MySQL for table data Migrated from Sun JVM to IcedTea JVM
  • 23. Dark Ages Age of Enlightenment Modern Era Today300150 0 Separate Solr core per customer Fallback to MySQL for table data Migrated from Sun JVM to IcedTea JVM
  • 24. Dark Ages Age of Enlightenment Modern Era TodayShared Schema Across Cores Decreases initialization time for each core Decreases memory usage
  • 25. Dark Ages Age of Enlightenment Modern Era Today Solr Index Field Selection More indexed fields = more used memory Only index fields that are searched Store other non-indexed fields for display
  • 26. Dark Ages Age of Enlightenment Modern Era Today Splitting of Reader / Writer Index building is CPU / disk intensive Writer = Solr with caching disabled Reader = Solr slave, that doesn’t build indexes
  • 27. Dark Ages Age of Enlightenment Modern Era Today Web Solr MySQL Slave (master- master)
  • 28. Dark Ages Age of Enlightenment Modern Era Today Web Solr MySQL Slave (master- master) Solr Master Range indexer Range indexer Range indexer Range indexer Document indexers Gearman workers
  • 29. Dark Ages Age of Enlightenment Modern Era Today Web Solr MySQL Slave (master- master) Solr Master Range indexer Range indexer Range indexer Range indexer Document indexers Gearman workers
  • 30. Dark Ages Age of Enlightenment Modern Era Today Web Solr MySQL Slave (master- master) Solr Master Range indexer Range indexer Range indexer Range indexer Document indexers Gearman workers
  • 31. Dark Ages Age of Enlightenment Modern Era Today Web Solr MySQL Slave (master- master) Solr Master Range indexer Range indexer Range indexer Range indexer Range indexers Reindex Manager
  • 32. Dark Ages Age of Enlightenment Modern Era Today Web Solr MySQL Slave (master- master) Solr Master Range indexer Range indexer Range indexer Range indexer Range indexers Reindex Manager
  • 33. Dark Ages Age of Enlightenment Modern Era Today Web Solr MySQL Slave (master- master) Solr Master Range indexer Range indexer Range indexer Range indexer Range indexers Reindex Manager
  • 34. Dark Ages Age of Enlightenment Modern Era Today Shared schema across cores Solr index field selection Splitting of reader / writer roles Intelligent indexing / reindexing
  • 35. Dark Ages Age of Enlightenment Modern Era Today1000500 0 Shared schema across cores Solr index field selection Splitting of reader / writer roles Intelligent indexing / reindexing
  • 36. Dark Ages Age of Enlightenment Modern Era TodayIntelligent Core Management Least recently used Solr cores are spun down Solr cores started on login Reindexing is a database flag, and happens on the next login
  • 37. Dark Ages Age of Enlightenment Modern Era Today Partitioning Pairs of readers and writers Partitioned based on account id Left pad the account id w/ zeros to length of two Reverse the account id Take last two digits of the account id
  • 38. Dark Ages Age of Enlightenment Modern Era Today Intelligent core spin down / up Partitioning of customers to separate Solr servers
  • 39. Dark Ages Age of Enlightenment Modern Era Today80004000 0 Intelligent core spin down / up Partitioning of customers to separate Solr servers
  • 40. SOLR Server8000 User
  • 41. SOLR SOLR SOLR Server SOLR Server SOLR Server SOLR8000 Server UserSOLR 8000 Server User 8000 Server 8000User 8000User Server 8000User 8000User User
  • 42. SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User
  • 43. SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User
  • 44. SOLR Server 8000 User SOLR Server 8000 User123456 SOLR Server 8000 User SOLR Server 8000 User SOLR Server 8000 User SOLR Server 8000 User SOLR Server 8000 User
  • 45. SOLR Server 8000 User SOLR Server 8000 User 123456 SOLR Server 8000 User1 2 3 4 5 6 SOLR Server 8000 User SOLR Server 8000 User SOLR Server 8000 User SOLR Server 8000 User
  • 46. SOLR Server 8000 User SOLR Server 8000 User 123456 SOLR Server 8000 User1 2 3 4 5 6 SOLR Server 8000 User SOLR Server 8000 User SOLR Server 8000 User SOLR Server 8000 User
  • 47. SOLR Server 8000 User SOLR Server 8000 User 123456 SOLR Server 8000 User1 2 3 4 5 6 SOLR Server 8000 User SOLR1 2 3 4 5 6 Server 8000 User SOLR Server 8000 User SOLR Server 8000 User
  • 48. SOLR Server 8000 User SOLR Server 8000 User 123456 SOLR Server 8000 User1 2 3 4 5 6 SOLR Server 8000 User SOLR1 2 3 4 5 6 Server 8000 User 65 SOLR Server 8000 User SOLR Server 8000 User
  • 49. SOLR Server 8000 User 01 SOLR 02 Server 8000 User 123456 03 SOLR 04 Server 8000 User1 2 3 4 5 6 65 SOLR Server 66 8000 User SOLR1 2 3 4 5 6 96 Server 8000 User 97 65 SOLR 98 Server 8000 User 99 SOLR Server 8000 User
  • 50. SOLR Server 8000 User 01 SOLR 02 Server 8000 User 123456 03 SOLR 04 Server 8000 User1 2 3 4 5 6 65 SOLR Server 66 8000 User SOLR1 2 3 4 5 6 96 Server 8000 User 97 65 SOLR 98 Server 8000 User 99 SOLR Server 8000 User
  • 51. SOLR Server 8000 User 01 SOLR 02 Server 8000 User 123456 03 SOLR 04 Server 8000 User1 2 3 4 5 6 65 SOLR Server 66 8000 User SOLR1 2 3 4 5 6 96 Server 8000 User 97 65 SOLR 98 Server 8000 User 99 SOLR Server 8000 User
  • 52. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR96 Server 8000 User97 SOLR98 Server 8000 User99 SOLR Server 8000 User
  • 53. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR96 Server 8000 User97 SOLR98 Server 8000 User99 SOLR Server 8000 User
  • 54. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR Server 8000 User SOLR96 Server 8000 User97 SOLR98 Server 8000 User99 SOLR Server 8000 User
  • 55. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR Server 8000 User SOLR96 Server 8000 User SOLR97 Server SOLR 8000 User98 Server 8000 User99 SOLR Server 8000 User
  • 56. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR Server 8000 User SOLR96 Server 8000 User SOLR97 Server SOLR 8000 User98 Server 8000 User99 SOLR Server 8000 User
  • 57. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR Server 8000 User SOLR96 Server 8000 User SOLR97 Server SOLR 8000 User98 Server 8000 User99 SOLR Server 8000 User
  • 58. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR Server 8000 User SOLR96 Server 8000 User SOLR97 Server SOLR 8000 User98 Server 8000 User99 SOLR Server 8000 User
  • 59. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR Server 8000 User96 SOLR97 Server SOLR 8000 User98 Server 8000 User99 SOLR Server 8000 User
  • 60. Scaling Solr in the Cloudauto provisioning 50 Usersseparate Jetty app for each customerseparate Solr core per customerfall back to MySQL for table data 300 Usersmigrated from Sun JVM to IcedTea JVMshared schema across coressolr index field selection 600 Userssplitting reader / writer rolesintelligent core spin down / up 7000 Usersparititioning of customer to separate Solr servers By @ablyler & @LindsaySnider FROM @NutshellCRM