Indexing documents


Published on

Alberto Lerner, software engineer at 10gen, presents at MongoUK (London) and MongoFR (Paris) in June 2010

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Indexing documents

  1. 1. Indexing Documents in MongoDB<br />Alberto LernerSoftware Engineer – 10Gen<br /><br />
  2. 2. Indexing Basics<br />MongoDB can use separate tree structures to index a collection<br />When processing a search criteria, MongoDB will try to avoid going through a collection, taking advantage of existing indices<br />
  3. 3. Team Work <br />MongoDB’s job: use an index, if possible<br />SearchCriteria<br />using index<br />scanning the collection<br />
  4. 4. Your Job<br />To provide indices for important queries<br />Important queries?<br />Very frequently used<br />Especially low response time required<br />
  5. 5. Creating an Index<br />You have an automatic one over _id<br />Others can be created with ‘ensureIndex’<br /># index over attribute ‘name’<br />db.<collection>.ensureIndex({name:1})<br /># compound keys, ascending/descending<br />db.<collection>.ensureIndex({name:1, date:-1 })<br /># unique keys<br />db.<collection>.ensureIndex({sku:1}, {unique:true})<br /># building in background<br />db.<collection>.ensureIndex( …, {background:true})<br />
  6. 6. Simple Search Criteria<br />Search criteria is the index key or a prefix thereof<br />db.<collection>.find({sku:1234}) # index over sku<br />db.<collection>.find({sku:1234}) # index over sku, <xxx><br />
  7. 7. More Exact Matching<br /># index over sku<br />….find({sku: {$in:[1234,5678])<br /># index over ‘product.sku’<br />….find({“product.sku”:1234})<br /># a tricky query, would need index on ‘product’ instead<br />….find({product: {sku:1234}}) <br />{ _id:1, product: {sku:1234} } # matches<br />
  8. 8. Range Criteria<br />Search criteria may return several results<br />db.<collection>.findOne({sku: {$gt:1234}}) <br />db.<collection>.find({sku: {$gt:5678,$lt:5699}})<br />
  9. 9. Range Criteria (cont)<br /># index over sku<br />….find({sku:/^12/}<br />
  10. 10. Other Operations<br /># index over sku<br />….update({sku:1234},{$inc:{sold:1}}})<br />….remove({sku:1234})<br />
  11. 11. Index Covering<br />Sometimes, all the needed information is in the index itself<br />….count({color:blue}) # index over color<br />….find({sku:1234},{color:1}) # index over sku, color<br />
  12. 12. Missing fields<br />All documents have an entry on an index<br />A missing field is indexed as a NULL<br /># matches all documents without sku<br /># if index over sku is unique, there could be only one<br />….find({sku:NULL}) <br /># will be using a sku index, but not there yet<br />….find({{sku:{$exits:true}})<br />
  13. 13. Array Matching<br />A field that contains an array will have one entry in the index per element in the array<br />{ _id: “abcd”, x:[2,10]} will appear in all the following queries using an index over x<br />….find({x:2})<br />….find({x:10})<br />….find({x:[2,10]})<br />….find({x:{$gt:5}}) # because of 10<br />
  14. 14. Indexes and Ordering<br />Sort elimination is also accomplished though using indexes<br />….find({sku:{$gt:56678}).sort({sku:1})<br />….find().sort({sku:-1}) # can traverse backwards<br />
  15. 15. Is It Using the Index?<br />explain() tool allows you to see whether an index is being chosen<br />db.<collection>.find({sku:{$gt:5}}).explain()<br />{<br />“cursor” : “BtreeCursor sku_1”,<br />…<br />}<br />
  16. 16. Hinting<br />Sometimes we may force or avoid the use of an index<br />Usually, it should not be necessary to intervene<br /># forces use of index over sku<br />….find{{sku:1, …}).hint({sku:1})<br /># prevents any index to be used<br />….find({sku:1,…}).hint({$natural:1})<br />
  17. 17. When Indexes Don’t Help<br /># negation<br />….find({sku:{$ne:9876}})<br /># index helps to filter string sku’s, though<br />….find({sku:/88/}) # generic regex<br /># $where may contain very expressive searches<br /># we don’t even try<br />….find({$where:”this.sku==1234”})<br />
  18. 18. Many indices?<br />Evaluating search criteria currently uses just one index, even if more than one would be possible<br />The choice is based on previous executions; if an index “worked well” for a query before, it’ll likely be again<br />Exception: $or can use more than one index<br />
  19. 19. So When to Index?<br />There’s a trade off between search criteria efficiency and insertion/update/deletion of keys<br />Also, there is (a quite high) limit on number of indexes per collection (that we keep bumping up)<br />
  20. 20. Indexes Resources<br />Indexes are memory mapped as well; <br />Be mindful of number of indexes and choice of keys<br /># In ‘indexSizes’, individual indexes in collection<br />db.<collection>.stats()<br /># All indexes in collection<br />db.<collection>.TotalIndexSize()<br />
  21. 21. Take away<br />The picture to keep in mind<br />SearchCriteria<br />
  22. 22. Questions?<br /><br />