Your SlideShare is downloading. ×
Scaling Solr in the Cloud  By @ablyler & @LindsaySnider              FROM         @NutshellCRM        www.nutshell.com
AgendaAbout the SpeakersAbout NutshellNutshell and SolrSolr Resource UsageScaling MethodsQuestions and Answers
About the SpeakersBitLeap / Barracuda Networks Developers on the Backup Appliance / Cloud Scaled databases and storage sys...
About NutshellWeb and Mobile CRM applicationHeavy use of OpenSource technologies: Gentoo Nginx / PHP / ZendFramework / jQu...
Nutshell and SolrHeavy use of Solr for searching, table views, andde-duplicationUsed for searching / display: Accounts, Co...
Reads vs Writes   7000   5250   3500   1750        0Jul 20, 2011 3:00 AM        Jul 20, 2011 6:00 AM    Jul 20, 2011 9:00 ...
Demowww.nutshell.com
SOLR ResourcesHeavy    CPU   Disk I/O                 Disk         RAM                StorageLight
SOLR ResourcesHeavy    CPU   Disk I/O                 Disk         RAM                StorageLight
SOLR ResourcesHeavy    CPU   Disk I/O                 Disk         RAM                StorageLight
SOLR ResourcesHeavy    CPU   Disk I/O                 Disk         RAM                StorageLight
Dark Ages   Age of Enlightenment   Modern Era   Today            Auto Provisioning Setup new Solr Jetty app Create MySQL d...
Dark Ages   Age of Enlightenment   Modern Era   TodaySeparate Jetty per Customer Uses a ton of memory Separate schema / So...
Dark Ages   Age of Enlightenment   Modern Era   Today Auto provisioning Separate Jetty app for each customer
Dark Ages   Age of Enlightenment   Modern Era   Today5025 0      Auto provisioning      Separate Jetty app for each customer
Dark Ages   Age of Enlightenment   Modern Era   Today      Solr Core per Customer Allows for management of Solr on a per c...
Dark Ages    Age of Enlightenment   Modern Era   Today            Fallback to MySQL Landing page of application Allows for...
Dark Ages   Age of Enlightenment   Modern Era   Today      Sun JVM to IcedTea JVM IcedTea JVM uses less memory than Sun JVM
Dark Ages   Age of Enlightenment   Modern Era   Today Separate Solr core per customer Fallback to MySQL for table data Mig...
Dark Ages   Age of Enlightenment   Modern Era   Today300150 0       Separate Solr core per customer       Fallback to MySQ...
Dark Ages   Age of Enlightenment   Modern Era   TodayShared Schema Across Cores Decreases initialization time for each cor...
Dark Ages   Age of Enlightenment   Modern Era   Today    Solr Index Field Selection More indexed fields = more used memory...
Dark Ages   Age of Enlightenment   Modern Era   Today  Splitting of Reader / Writer Index building is CPU / disk intensive...
Dark Ages    Age of Enlightenment   Modern Era           Today                            Web                             ...
Dark Ages   Age of Enlightenment       Modern Era        Today                              Web                           ...
Dark Ages   Age of Enlightenment       Modern Era        Today                              Web                           ...
Dark Ages   Age of Enlightenment       Modern Era        Today                              Web                           ...
Dark Ages   Age of Enlightenment     Modern Era              Today                              Web                       ...
Dark Ages   Age of Enlightenment     Modern Era            Today                              Web                         ...
Dark Ages   Age of Enlightenment     Modern Era              Today                              Web                       ...
Dark Ages   Age of Enlightenment   Modern Era   Today Shared schema across cores Solr index field selection Splitting of r...
Dark Ages   Age of Enlightenment   Modern Era   Today1000500  0        Shared schema across cores        Solr index field ...
Dark Ages   Age of Enlightenment   Modern Era   TodayIntelligent Core Management Least recently used Solr cores are spun d...
Dark Ages      Age of Enlightenment    Modern Era       Today                    Partitioning Pairs of readers and writers...
Dark Ages   Age of Enlightenment   Modern Era   Today Intelligent core spin down / up Partitioning of customers to separat...
Dark Ages   Age of Enlightenment   Modern Era   Today80004000   0        Intelligent core spin down / up        Partitioni...
SOLR  Server8000 User
SOLR   SOLR    SOLR  Server     SOLR   Server      SOLR    Server       SOLR8000 Server     UserSOLR 8000 Server      User...
SOLR  Server8000 User  SOLR  Server8000 User  SOLR  Server8000 User  SOLR  Server8000 User  SOLR  Server8000 User  SOLR  S...
SOLR  Server8000 User  SOLR  Server8000 User  SOLR  Server8000 User  SOLR  Server8000 User  SOLR  Server8000 User  SOLR  S...
SOLR           Server         8000 User           SOLR           Server         8000 User123456     SOLR           Server ...
SOLR                Server              8000 User                SOLR                Server              8000 User  123456...
SOLR                Server              8000 User                SOLR                Server              8000 User  123456...
SOLR                Server              8000 User                SOLR                Server              8000 User  123456...
SOLR                Server              8000 User                SOLR                Server              8000 User  123456...
SOLR                     Server                   8000 User              01                     SOLR              02     S...
SOLR                     Server                   8000 User              01                     SOLR              02     S...
SOLR                     Server                   8000 User              01                     SOLR              02     S...
SOLR       Server     8000 User01       SOLR02     Server     8000 User03       SOLR04     Server     8000 User65     SOLR...
SOLR       Server     8000 User01       SOLR02     Server     8000 User03       SOLR04     Server     8000 User65     SOLR...
SOLR       Server     8000 User01       SOLR02     Server     8000 User03       SOLR04     Server     8000 User65     SOLR...
SOLR       Server     8000 User01       SOLR02     Server     8000 User03       SOLR04     Server     8000 User65     SOLR...
SOLR       Server     8000 User01       SOLR02     Server     8000 User03       SOLR04     Server     8000 User65     SOLR...
SOLR       Server     8000 User01       SOLR02     Server     8000 User03       SOLR04     Server     8000 User65     SOLR...
SOLR       Server     8000 User01       SOLR02     Server     8000 User03       SOLR04     Server     8000 User65     SOLR...
SOLR       Server     8000 User01       SOLR02     Server     8000 User03       SOLR04     Server     8000 User65     SOLR...
Scaling Solr in the Cloudauto provisioning                                                       50 Usersseparate Jetty ap...
Scalling Solr in the Cloud - OSCON Data 2011
Scalling Solr in the Cloud - OSCON Data 2011
Scalling Solr in the Cloud - OSCON Data 2011
Scalling Solr in the Cloud - OSCON Data 2011
Upcoming SlideShare
Loading in...5
×

Scalling Solr in the Cloud - OSCON Data 2011

2,246

Published on

Solr, an open source enterprise search server, scales very well within an index (vertical scaling). It is when you have multiple indexes (horizontal scaling) that it starts to get hairy, which happens a lot when you are hosting a cloud based solution for multiple users. In this session we will discuss these issue as well as the techniques of how to overcome them in-depth.

By: Andy Blyler & Lindsay Snider

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,246
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
28
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "Scalling Solr in the Cloud - OSCON Data 2011"

  1. 1. Scaling Solr in the Cloud By @ablyler & @LindsaySnider FROM @NutshellCRM www.nutshell.com
  2. 2. AgendaAbout the SpeakersAbout NutshellNutshell and SolrSolr Resource UsageScaling MethodsQuestions and Answers
  3. 3. About the SpeakersBitLeap / Barracuda Networks Developers on the Backup Appliance / Cloud Scaled databases and storage systems Used SugarCRM & Salesforce.com
  4. 4. About NutshellWeb and Mobile CRM applicationHeavy use of OpenSource technologies: Gentoo Nginx / PHP / ZendFramework / jQuery MySQL / Solr / Gearman Jenkins / Redmine / Cacti / Nagios
  5. 5. Nutshell and SolrHeavy use of Solr for searching, table views, andde-duplicationUsed for searching / display: Accounts, Competitors, Contacts, Leads, Products, Sources, Teams, and UsersUsed for de-duplication: Accounts, Contacts
  6. 6. Reads vs Writes 7000 5250 3500 1750 0Jul 20, 2011 3:00 AM Jul 20, 2011 6:00 AM Jul 20, 2011 9:00 AM Jul 20, 2011 12:00 PM Jul 20, 2011 3:00 PM Jul 20, 2011 6:00 PM Jul 20, 2011 9:00 PM Jul 21, 2011 12:00 AM Writes Reads Time Average Read Query Time: 2.11ms
  7. 7. Demowww.nutshell.com
  8. 8. Computer Resources CPU Disk I/O Disk RAM Storage
  9. 9. CPU Disk I/O DiskRAM Storage
  10. 10. SOLR ResourcesHeavy CPU Disk I/O Disk RAM StorageLight
  11. 11. SOLR ResourcesHeavy CPU Disk I/O Disk RAM StorageLight
  12. 12. SOLR ResourcesHeavy CPU Disk I/O Disk RAM StorageLight
  13. 13. SOLR ResourcesHeavy CPU Disk I/O Disk RAM StorageLight
  14. 14. SOLR ResourcesHeavy CPU Disk I/O Disk RAM StorageLight
  15. 15. Dark Ages Age of Enlightenment Modern Era Today Auto Provisioning Setup new Solr Jetty app Create MySQL database Populate MySQL / Solr with demo data Send welcome email
  16. 16. Dark Ages Age of Enlightenment Modern Era TodaySeparate Jetty per Customer Uses a ton of memory Separate schema / Solr for each customer Ran into upper limit morning before launch
  17. 17. Dark Ages Age of Enlightenment Modern Era Today Auto provisioning Separate Jetty app for each customer
  18. 18. Dark Ages Age of Enlightenment Modern Era Today5025 0 Auto provisioning Separate Jetty app for each customer
  19. 19. Dark Ages Age of Enlightenment Modern Era Today Solr Core per Customer Allows for management of Solr on a per customer basis: creating / stopping Contained within a single Jetty app Shared schema between all cores Easily managed via simple HTTP API
  20. 20. Dark Ages Age of Enlightenment Modern Era Today Fallback to MySQL Landing page of application Allows for graceful handling when Solr is down Abstracted within the application library
  21. 21. Dark Ages Age of Enlightenment Modern Era Today Sun JVM to IcedTea JVM IcedTea JVM uses less memory than Sun JVM
  22. 22. Dark Ages Age of Enlightenment Modern Era Today Separate Solr core per customer Fallback to MySQL for table data Migrated from Sun JVM to IcedTea JVM
  23. 23. Dark Ages Age of Enlightenment Modern Era Today300150 0 Separate Solr core per customer Fallback to MySQL for table data Migrated from Sun JVM to IcedTea JVM
  24. 24. Dark Ages Age of Enlightenment Modern Era TodayShared Schema Across Cores Decreases initialization time for each core Decreases memory usage
  25. 25. Dark Ages Age of Enlightenment Modern Era Today Solr Index Field Selection More indexed fields = more used memory Only index fields that are searched Store other non-indexed fields for display
  26. 26. Dark Ages Age of Enlightenment Modern Era Today Splitting of Reader / Writer Index building is CPU / disk intensive Writer = Solr with caching disabled Reader = Solr slave, that doesn’t build indexes
  27. 27. Dark Ages Age of Enlightenment Modern Era Today Web Solr MySQL Slave (master- master)
  28. 28. Dark Ages Age of Enlightenment Modern Era Today Web Solr MySQL Slave (master- master) Solr Master Range indexer Range indexer Range indexer Range indexer Document indexers Gearman workers
  29. 29. Dark Ages Age of Enlightenment Modern Era Today Web Solr MySQL Slave (master- master) Solr Master Range indexer Range indexer Range indexer Range indexer Document indexers Gearman workers
  30. 30. Dark Ages Age of Enlightenment Modern Era Today Web Solr MySQL Slave (master- master) Solr Master Range indexer Range indexer Range indexer Range indexer Document indexers Gearman workers
  31. 31. Dark Ages Age of Enlightenment Modern Era Today Web Solr MySQL Slave (master- master) Solr Master Range indexer Range indexer Range indexer Range indexer Range indexers Reindex Manager
  32. 32. Dark Ages Age of Enlightenment Modern Era Today Web Solr MySQL Slave (master- master) Solr Master Range indexer Range indexer Range indexer Range indexer Range indexers Reindex Manager
  33. 33. Dark Ages Age of Enlightenment Modern Era Today Web Solr MySQL Slave (master- master) Solr Master Range indexer Range indexer Range indexer Range indexer Range indexers Reindex Manager
  34. 34. Dark Ages Age of Enlightenment Modern Era Today Shared schema across cores Solr index field selection Splitting of reader / writer roles Intelligent indexing / reindexing
  35. 35. Dark Ages Age of Enlightenment Modern Era Today1000500 0 Shared schema across cores Solr index field selection Splitting of reader / writer roles Intelligent indexing / reindexing
  36. 36. Dark Ages Age of Enlightenment Modern Era TodayIntelligent Core Management Least recently used Solr cores are spun down Solr cores started on login Reindexing is a database flag, and happens on the next login
  37. 37. Dark Ages Age of Enlightenment Modern Era Today Partitioning Pairs of readers and writers Partitioned based on account id Left pad the account id w/ zeros to length of two Reverse the account id Take last two digits of the account id
  38. 38. Dark Ages Age of Enlightenment Modern Era Today Intelligent core spin down / up Partitioning of customers to separate Solr servers
  39. 39. Dark Ages Age of Enlightenment Modern Era Today80004000 0 Intelligent core spin down / up Partitioning of customers to separate Solr servers
  40. 40. SOLR Server8000 User
  41. 41. SOLR SOLR SOLR Server SOLR Server SOLR Server SOLR8000 Server UserSOLR 8000 Server User 8000 Server 8000User 8000User Server 8000User 8000User User
  42. 42. SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User
  43. 43. SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User SOLR Server8000 User
  44. 44. SOLR Server 8000 User SOLR Server 8000 User123456 SOLR Server 8000 User SOLR Server 8000 User SOLR Server 8000 User SOLR Server 8000 User SOLR Server 8000 User
  45. 45. SOLR Server 8000 User SOLR Server 8000 User 123456 SOLR Server 8000 User1 2 3 4 5 6 SOLR Server 8000 User SOLR Server 8000 User SOLR Server 8000 User SOLR Server 8000 User
  46. 46. SOLR Server 8000 User SOLR Server 8000 User 123456 SOLR Server 8000 User1 2 3 4 5 6 SOLR Server 8000 User SOLR Server 8000 User SOLR Server 8000 User SOLR Server 8000 User
  47. 47. SOLR Server 8000 User SOLR Server 8000 User 123456 SOLR Server 8000 User1 2 3 4 5 6 SOLR Server 8000 User SOLR1 2 3 4 5 6 Server 8000 User SOLR Server 8000 User SOLR Server 8000 User
  48. 48. SOLR Server 8000 User SOLR Server 8000 User 123456 SOLR Server 8000 User1 2 3 4 5 6 SOLR Server 8000 User SOLR1 2 3 4 5 6 Server 8000 User 65 SOLR Server 8000 User SOLR Server 8000 User
  49. 49. SOLR Server 8000 User 01 SOLR 02 Server 8000 User 123456 03 SOLR 04 Server 8000 User1 2 3 4 5 6 65 SOLR Server 66 8000 User SOLR1 2 3 4 5 6 96 Server 8000 User 97 65 SOLR 98 Server 8000 User 99 SOLR Server 8000 User
  50. 50. SOLR Server 8000 User 01 SOLR 02 Server 8000 User 123456 03 SOLR 04 Server 8000 User1 2 3 4 5 6 65 SOLR Server 66 8000 User SOLR1 2 3 4 5 6 96 Server 8000 User 97 65 SOLR 98 Server 8000 User 99 SOLR Server 8000 User
  51. 51. SOLR Server 8000 User 01 SOLR 02 Server 8000 User 123456 03 SOLR 04 Server 8000 User1 2 3 4 5 6 65 SOLR Server 66 8000 User SOLR1 2 3 4 5 6 96 Server 8000 User 97 65 SOLR 98 Server 8000 User 99 SOLR Server 8000 User
  52. 52. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR96 Server 8000 User97 SOLR98 Server 8000 User99 SOLR Server 8000 User
  53. 53. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR96 Server 8000 User97 SOLR98 Server 8000 User99 SOLR Server 8000 User
  54. 54. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR Server 8000 User SOLR96 Server 8000 User97 SOLR98 Server 8000 User99 SOLR Server 8000 User
  55. 55. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR Server 8000 User SOLR96 Server 8000 User SOLR97 Server SOLR 8000 User98 Server 8000 User99 SOLR Server 8000 User
  56. 56. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR Server 8000 User SOLR96 Server 8000 User SOLR97 Server SOLR 8000 User98 Server 8000 User99 SOLR Server 8000 User
  57. 57. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR Server 8000 User SOLR96 Server 8000 User SOLR97 Server SOLR 8000 User98 Server 8000 User99 SOLR Server 8000 User
  58. 58. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR Server 8000 User SOLR96 Server 8000 User SOLR97 Server SOLR 8000 User98 Server 8000 User99 SOLR Server 8000 User
  59. 59. SOLR Server 8000 User01 SOLR02 Server 8000 User03 SOLR04 Server 8000 User65 SOLR Server66 8000 User SOLR Server 8000 User96 SOLR97 Server SOLR 8000 User98 Server 8000 User99 SOLR Server 8000 User
  60. 60. Scaling Solr in the Cloudauto provisioning 50 Usersseparate Jetty app for each customerseparate Solr core per customerfall back to MySQL for table data 300 Usersmigrated from Sun JVM to IcedTea JVMshared schema across coressolr index field selection 600 Userssplitting reader / writer rolesintelligent core spin down / up 7000 Usersparititioning of customer to separate Solr servers By @ablyler & @LindsaySnider FROM @NutshellCRM

×