The importance of indexes in mongo db

The Importance ofIndexes in MongoDBHow we increased the loading speed Profile Gliphs and InsightsJames Toyer, Lead Software Engineer at Glipho

What is Glipho?  Social network for text based content  Aims to better engage writers and readers  Original content only  Not an aggregator  Automatically share to Facebook, LinkedIn and Twitter

Insights Page  Load up the gliphs a writer has  Iterate through, and sum, actions for each gliph  Load and sum actions for the writers profile  This can be over 100 calls to the database  We know it’s inefficient but it does the job for now

Insights Document Structure Timestamp – when the action took place EntityId – identifier of the original entity ActionType – the type of the entity (probably should be entity type) Action – the actual action that took place

Helpful Error Page…it’s all gone wrong!

Troubleshooting CPU spiking? NO Memory high? NO Disk IO high? NO Are there any actual regular hits happening? NO Do you know anything? NOCrack out the code performance tools…

Pre-Index performance• 3 passes on each filter page• Average time for each page to load = 3.9 seconds• “ListAll” method calls the database• “ListAll” is iterated over for each gliph in the database and the profile (in this case ~10 times)• Average time in “ListAll” 256ms

More Troubleshooting Is the code doing obviously stupid things? NO Has Linq screwed you over again? NO Do you trust the driver? PROBABLY Check the database  ~ 400,000 documents (now ~690,000)  No indexes

Know your query GetMongoQuery code Output

Index analysis Without action field With action field Query structure  Query structure Query time before index:  Query time before index:  334ms  409ms Index  Index Query time after index:  Query time after index:  >1ms  >1ms

Post-Index performance Pre-index performance Post-index performance Page load time:  Page load time:  3.9s  72ms “ListAll” method execution  “ListAll” method execution time: time:  256ms  >0.2ms  Page load time deceased by 98%  “ListAll” method execution time decreased by 99%

Gliph listings for Writers  Problems:  Slow loading  Sometimes erroring out  Reasons:  Indexes were no longer accurate  Code had changed  Solution:  New indexes  Remove old indexes

What did I learn? Know exactly what queries are being run Don’t do a “best guess” on an index. Test them out Don’t “forget” to add indexes Ensure your indexes evolve as your queries do

Any Questions? james@glipho.com glipho.com/james @jamestoyer

The importance of indexes in mongo db

by mongodb/10gen on Jan 23, 2013

Accessibility

Upload Details

Usage Rights

1 Embed 4

Statistics

The importance of indexes in mongo db Presentation Transcript