Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
The Importance ofIndexes in MongoDBHow we increased the loading speed Profile Gliphs and InsightsJames Toyer, Lead Softwar...
What is Glipho?                   Social network for text                    based content                   Aims to bet...
Insights Page                 Load up the gliphs a writer                  has                 Iterate through, and     ...
Insights Document Structure Timestamp – when the action took place EntityId – identifier of the original entity ActionT...
Helpful Error Page…it’s all gone wrong!
Troubleshooting CPU spiking? NO Memory high? NO Disk IO high? NO Are there any actual regular hits happening? NO Do y...
Pre-Index performance•   3 passes on each filter page•   Average time for each page to load = 3.9 seconds•   “ListAll” met...
More Troubleshooting Is the code doing obviously stupid things? NO Has Linq screwed you over again? NO Do you trust the...
Know your query GetMongoQuery code   Output
Index analysis    Without action field           With action field Query structure             Query structure Query ti...
Post-Index performance  Pre-index performance           Post-index performance Page load time:                Page load ...
Gliph listings for Writers                  Problems:                     Slow loading                     Sometimes er...
What did I learn? Know exactly what queries are being run Don’t do a “best guess” on an index. Test them out Don’t “for...
Any Questions?       james@glipho.com       glipho.com/james            @jamestoyer
Upcoming SlideShare
Loading in …5
×

The importance of indexes in mongo db

2,682 views

Published on

  • Be the first to comment

The importance of indexes in mongo db

  1. 1. The Importance ofIndexes in MongoDBHow we increased the loading speed Profile Gliphs and InsightsJames Toyer, Lead Software Engineer at Glipho
  2. 2. What is Glipho?  Social network for text based content  Aims to better engage writers and readers  Original content only  Not an aggregator  Automatically share to Facebook, LinkedIn and Twitter
  3. 3. Insights Page  Load up the gliphs a writer has  Iterate through, and sum, actions for each gliph  Load and sum actions for the writers profile  This can be over 100 calls to the database  We know it’s inefficient but it does the job for now
  4. 4. Insights Document Structure Timestamp – when the action took place EntityId – identifier of the original entity ActionType – the type of the entity (probably should be entity type) Action – the actual action that took place
  5. 5. Helpful Error Page…it’s all gone wrong!
  6. 6. Troubleshooting CPU spiking? NO Memory high? NO Disk IO high? NO Are there any actual regular hits happening? NO Do you know anything? NOCrack out the code performance tools…
  7. 7. Pre-Index performance• 3 passes on each filter page• Average time for each page to load = 3.9 seconds• “ListAll” method calls the database• “ListAll” is iterated over for each gliph in the database and the profile (in this case ~10 times)• Average time in “ListAll” 256ms
  8. 8. More Troubleshooting Is the code doing obviously stupid things? NO Has Linq screwed you over again? NO Do you trust the driver? PROBABLY Check the database  ~ 400,000 documents (now ~690,000)  No indexes
  9. 9. Know your query GetMongoQuery code Output
  10. 10. Index analysis Without action field With action field Query structure  Query structure Query time before index:  Query time before index:  334ms  409ms Index  Index Query time after index:  Query time after index:  >1ms  >1ms
  11. 11. Post-Index performance Pre-index performance Post-index performance Page load time:  Page load time:  3.9s  72ms “ListAll” method execution  “ListAll” method execution time: time:  256ms  >0.2ms  Page load time deceased by 98%  “ListAll” method execution time decreased by 99%
  12. 12. Gliph listings for Writers  Problems:  Slow loading  Sometimes erroring out  Reasons:  Indexes were no longer accurate  Code had changed  Solution:  New indexes  Remove old indexes
  13. 13. What did I learn? Know exactly what queries are being run Don’t do a “best guess” on an index. Test them out Don’t “forget” to add indexes Ensure your indexes evolve as your queries do
  14. 14. Any Questions? james@glipho.com glipho.com/james @jamestoyer

×