SlideShare a Scribd company logo
1 of 67
Indexing with
   MongoDB
        Luke Ehresman
CopperEgg - www.copperegg.com
What is an index?
What is an index?

• Pointers to documents
What is an index?

• Pointers to documents
• Efficiently organized for quick scanning
What is an index?

• Pointers to documents
• Efficiently organized for quick scanning
• Maintained in a tree structure
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}




                 6 documents scanned
How do we find x = 6 with an index?
How do we find x = 6 with an index?

                        {x:4}
                <                 >



        {x:2}                             {x:6}
    <           >                     <           >


{x:1}           {x:3}           {x:5}             {x:7}
How do we find x = 6 with an index?

                        {x:4}
                <                 >



        {x:2}                             {x:6}
    <           >                     <           >


{x:1}           {x:3}           {x:5}             {x:7}
How do we find x = 6 with an index?

                        {x:4}
                <                 >



        {x:2}                             {x:6}
    <           >                     <           >


{x:1}           {x:3}           {x:5}             {x:7}


    Only 2 documents scanned
In reality, it’s even better than that...
In reality, it’s even better than that...


                                    {x:0, name:‘George’, ....}
                                    {x:1, name:‘Jane’, ....}
                                    {x:2, name:‘Judy’, ....}
                                    {x:3, name:‘Elroy’, ....}
                                    {x:4, name:‘Astro’, ....}
                                    {x:5, name:‘Rosie’, ....}
                                    {x:6, name:‘Fred’, ....}
                                    {x:7, name:‘Wilma’, ....}
                                    {x:8, name:‘Barney’, ....}
                                    {x:9, name:‘Betty’, ....}
In reality, it’s even better than that...


                        {x:4}                             {x:0, name:‘George’, ....}
                                                          {x:1, name:‘Jane’, ....}
                <                 >                       {x:2, name:‘Judy’, ....}
                                                          {x:3, name:‘Elroy’, ....}
                                                          {x:4, name:‘Astro’, ....}
                                                          {x:5, name:‘Rosie’, ....}
        {x:2}                             {x:6}           {x:6, name:‘Fred’, ....}
                                                          {x:7, name:‘Wilma’, ....}
    <           >                     <           >       {x:8, name:‘Barney’, ....}
                                                          {x:9, name:‘Betty’, ....}

{x:1}           {x:3}           {x:5}             {x:7}
In reality, it’s still even better than that...
In reality, it’s still even better than that...

•MongoDB stores data and indexes in extents
In reality, it’s still even better than that...

•MongoDB stores data and indexes in extents
•Only loads extents into memory as needed
In reality, it’s still even better than that...

•MongoDB stores data and indexes in extents
•Only loads extents into memory as needed
•The full index does not need to be in memory
In reality, it’s still even better than that...

•MongoDB stores data and indexes in extents
•Only loads extents into memory as needed
•The full index does not need to be in memory
                            {x:4}
                    <                 >



            {x:2}                             {x:6}
        <           >                     <           >


    {x:1}           {x:3}           {x:5}             {x:7}
When to index?
When to index?

• Frequently queried fields
When to index?

• Frequently queried fields
• Low response time
When to index?

• Frequently queried fields
• Low response time
• Sorting
When to index?

• Frequently queried fields
• Low response time
• Sorting
• Avoid full collection scans
Things to know...
Things to know...
• Indexes maintain order
Things to know...
• Indexes maintain order
• Indexes will slow writes
Things to know...
• Indexes maintain order
• Indexes will slow writes
• Indexes take up space
Things to know...
• Indexes maintain order
• Indexes will slow writes
• Indexes take up space
• Index maintenance will cause writes
Things to know...
• Indexes maintain order
• Indexes will slow writes
• Indexes take up space
• Index maintenance will cause writes
• “unique” parameter forces uniqueness
How to create an index
How to create an index
    db.people.ensureIndex({...})
How to create an index
        db.people.ensureIndex({...})

{_id:1} is already created for each collection
How to create an index
            db.people.ensureIndex({...})

   {_id:1} is already created for each collection

      You can call ensureIndex multiple times
(it only creates an index if it doesn’t already exist)
How to create an index
            db.people.ensureIndex({...})

   {_id:1} is already created for each collection

      You can call ensureIndex multiple times
(it only creates an index if it doesn’t already exist)

     Blocks your mongod process unless you
             specify background:true.
Indexes on missing fields
Indexes on missing fields

null is assumed if a field is missing
Indexes on missing fields

         null is assumed if a field is missing

db.people.insert({name:‘Fred’, children:[‘Pebbles’]});
db.people.insert({name:‘Pebbles’})

db.people.ensureIndex({children:1})

db.people.find({children:null});
(returns Pebbles and uses the index)
Indexes on missing fields

         null is assumed if a field is missing

db.people.insert({name:‘Fred’, children:[‘Pebbles’]});
db.people.insert({name:‘Pebbles’})

db.people.ensureIndex({children:1})

db.people.find({children:null});
(returns Pebbles and uses the index)

         Optional sparse indexes exclude
           empty fields for efficiency
Compound indexes
Compound indexes

{
    name: “George”,
    age:45,
    gender:”M”,
    ...
}
Compound indexes

{
    name: “George”,        {gender:1, age:-1}
    age:45,
    gender:”M”,
    ...
}
Compound indexes

{
    name: “George”,          {gender:1, age:-1}
    age:45,
    gender:”M”,
    ...
}                         gender:”M”
                            age:45
                                              {name:‘George’, ....}
                            age:12
                                              {name:‘Jane’, ....}
                            age:3
                                              {name:‘Judy’, ....}
                          gender:”F”
                                              {name:‘Elroy’, ....}
                            age:44
                                              {name:‘Astro’, ....}
                            age:16
                                              {name:‘Rosie’, ....}
                            age:10
Indexes on arrays
Indexes on arrays
{
    name: “George”,
    age:45,
    gender:”M”,
    children: [‘Judy’, ‘Elroy’]
    ...
}
Indexes on arrays
{
    name: “George”,
    age:45,
                                  {children:1}
    gender:”M”,
    children: [‘Judy’, ‘Elroy’]
    ...
}
Indexes on arrays
{
    name: “George”,
    age:45,
                                  {children:1}
    gender:”M”,
    children: [‘Judy’, ‘Elroy’]
    ...
}
                                  Actually creates a new entry
                                     in the index for each
                                     element of the array.
Indexes on arrays
 {
     name: “George”,
     age:45,
                                   {children:1}
     gender:”M”,
     children: [‘Judy’, ‘Elroy’]
     ...
 }
                                   Actually creates a new entry
                                      in the index for each
                                      element of the array.
children:‘Judy’                         {name:‘George’, ....}
children:‘Elroy’                        {name:‘Fred’, ...}
children:‘Bam-Bam’                      {name:‘Barney’ ...}
Covered indexes
Covered indexes
Queries that can be resolved with only the index
(does not need to fetch the original document)
Covered indexes
Queries that can be resolved with only the index
(does not need to fetch the original document)


{
    name: “George”,
    age:45,
    gender:”M”,
    children: [‘Judy’, ‘Elroy’]
    ...
}
Covered indexes
Queries that can be resolved with only the index
(does not need to fetch the original document)


{
    name: “George”,
    age:45,                       db.people.ensureIndex({name:1, age:1});
    gender:”M”,                   db.people.find({name:‘George’}, {_id:0, age:1});
    children: [‘Judy’, ‘Elroy’]
    ...
}
How does it know which index to use?
How does it know which index to use?

db.people.ensureIndex({name:1});
db.people.ensureIndex({age:1});

db.people.find({age:{$gte:21}});
How does it know which index to use?

db.people.ensureIndex({name:1});
db.people.ensureIndex({age:1});

db.people.find({age:{$gte:21}});

 scan


 index on name


 index on age
How does it know which index to use?

db.people.ensureIndex({name:1});
db.people.ensureIndex({age:1});

db.people.find({age:{$gte:21}});

 scan

                            terminated
 index on name


 index on age         remember
Query performance analysis


db.people.find({name:‘Fred’}).explain()

{
    “cursor” : “BasicCursor”,
    “indexBounds” : {},
    “nscanned” : 88731,
    “nscannedObjects” : 88731,
    “n” : 1,
    “millis” : 108
}
Query performance analysis


db.people.ensureIndex({name:1})
db.people.find({name:‘Fred’}).explain()

{
    “cursor” : “BtreeCursor name_1”,
    ...
}
Query performance analysis


db.people.ensureIndex({name:1})
db.people.find({name:‘Fred’}).explain()

{
    “cursor” : “BtreeCursor name_1”,
    ...
}



                     Demo
Credits
• Much of this information was gleaned from
  these two presentations:
• http://speakerdeck.com/u/mongodb/p/
  indexing-and-query-optimizer-kevin-
  hanson-10gen
• http://www.slideshare.net/mongodb/
  indexing-with-mongodb

More Related Content

What's hot

Coding Horrors
Coding HorrorsCoding Horrors
Coding HorrorsMark Baker
 
Php code for online quiz
Php code for online quizPhp code for online quiz
Php code for online quizhnyb1002
 
Improving RDF Search Performance with Lucene and SIREN
Improving RDF Search Performance with Lucene and SIRENImproving RDF Search Performance with Lucene and SIREN
Improving RDF Search Performance with Lucene and SIRENMike Hugo
 
Gareth hayes. non alphanumeric javascript-php and shared fuzzing
Gareth hayes. non alphanumeric javascript-php and shared fuzzingGareth hayes. non alphanumeric javascript-php and shared fuzzing
Gareth hayes. non alphanumeric javascript-php and shared fuzzingYury Chemerkin
 
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...MongoDB
 
Real World CouchDB
Real World CouchDBReal World CouchDB
Real World CouchDBJohn Wood
 
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDBMongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDBMongoDB
 
Airline reservation project using JAVA in NetBeans IDE
Airline reservation project using JAVA in NetBeans IDEAirline reservation project using JAVA in NetBeans IDE
Airline reservation project using JAVA in NetBeans IDEHimanshiSingh71
 
Building a Scalable Inbox System with MongoDB and Java
Building a Scalable Inbox System with MongoDB and JavaBuilding a Scalable Inbox System with MongoDB and Java
Building a Scalable Inbox System with MongoDB and Javaantoinegirbal
 
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDBMongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDBMongoDB
 
Strongly Typed Languages and Flexible Schemas
Strongly Typed Languages and Flexible SchemasStrongly Typed Languages and Flexible Schemas
Strongly Typed Languages and Flexible SchemasNorberto Leite
 
Webinar: Strongly Typed Languages and Flexible Schemas
Webinar: Strongly Typed Languages and Flexible SchemasWebinar: Strongly Typed Languages and Flexible Schemas
Webinar: Strongly Typed Languages and Flexible SchemasMongoDB
 
Schema Design with MongoDB
Schema Design with MongoDBSchema Design with MongoDB
Schema Design with MongoDBrogerbodamer
 
ETL for Pros: Getting Data Into MongoDB
ETL for Pros: Getting Data Into MongoDBETL for Pros: Getting Data Into MongoDB
ETL for Pros: Getting Data Into MongoDBMongoDB
 
MongoDB Schema Design: Four Real-World Examples
MongoDB Schema Design: Four Real-World ExamplesMongoDB Schema Design: Four Real-World Examples
MongoDB Schema Design: Four Real-World ExamplesMike Friedman
 
Dev Jumpstart: Schema Design Best Practices
Dev Jumpstart: Schema Design Best PracticesDev Jumpstart: Schema Design Best Practices
Dev Jumpstart: Schema Design Best PracticesMongoDB
 

What's hot (17)

Coding Horrors
Coding HorrorsCoding Horrors
Coding Horrors
 
Php code for online quiz
Php code for online quizPhp code for online quiz
Php code for online quiz
 
Improving RDF Search Performance with Lucene and SIREN
Improving RDF Search Performance with Lucene and SIRENImproving RDF Search Performance with Lucene and SIREN
Improving RDF Search Performance with Lucene and SIREN
 
Gareth hayes. non alphanumeric javascript-php and shared fuzzing
Gareth hayes. non alphanumeric javascript-php and shared fuzzingGareth hayes. non alphanumeric javascript-php and shared fuzzing
Gareth hayes. non alphanumeric javascript-php and shared fuzzing
 
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...
 
Real World CouchDB
Real World CouchDBReal World CouchDB
Real World CouchDB
 
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDBMongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDB
 
Airline reservation project using JAVA in NetBeans IDE
Airline reservation project using JAVA in NetBeans IDEAirline reservation project using JAVA in NetBeans IDE
Airline reservation project using JAVA in NetBeans IDE
 
Building a Scalable Inbox System with MongoDB and Java
Building a Scalable Inbox System with MongoDB and JavaBuilding a Scalable Inbox System with MongoDB and Java
Building a Scalable Inbox System with MongoDB and Java
 
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDBMongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDB
 
MongoDB With Style
MongoDB With StyleMongoDB With Style
MongoDB With Style
 
Strongly Typed Languages and Flexible Schemas
Strongly Typed Languages and Flexible SchemasStrongly Typed Languages and Flexible Schemas
Strongly Typed Languages and Flexible Schemas
 
Webinar: Strongly Typed Languages and Flexible Schemas
Webinar: Strongly Typed Languages and Flexible SchemasWebinar: Strongly Typed Languages and Flexible Schemas
Webinar: Strongly Typed Languages and Flexible Schemas
 
Schema Design with MongoDB
Schema Design with MongoDBSchema Design with MongoDB
Schema Design with MongoDB
 
ETL for Pros: Getting Data Into MongoDB
ETL for Pros: Getting Data Into MongoDBETL for Pros: Getting Data Into MongoDB
ETL for Pros: Getting Data Into MongoDB
 
MongoDB Schema Design: Four Real-World Examples
MongoDB Schema Design: Four Real-World ExamplesMongoDB Schema Design: Four Real-World Examples
MongoDB Schema Design: Four Real-World Examples
 
Dev Jumpstart: Schema Design Best Practices
Dev Jumpstart: Schema Design Best PracticesDev Jumpstart: Schema Design Best Practices
Dev Jumpstart: Schema Design Best Practices
 

Viewers also liked

Fast querying indexing for performance (4)
Fast querying   indexing for performance (4)Fast querying   indexing for performance (4)
Fast querying indexing for performance (4)MongoDB
 
Indexing with MongoDB
Indexing with MongoDBIndexing with MongoDB
Indexing with MongoDBMongoDB
 
Performance Tuning on the Fly at CMP.LY
Performance Tuning on the Fly at CMP.LYPerformance Tuning on the Fly at CMP.LY
Performance Tuning on the Fly at CMP.LYMongoDB
 
MongoDB and Indexes - MUG Denver - 20160329
MongoDB and Indexes - MUG Denver - 20160329MongoDB and Indexes - MUG Denver - 20160329
MongoDB and Indexes - MUG Denver - 20160329Douglas Duncan
 
Indexing and Performance Tuning
Indexing and Performance TuningIndexing and Performance Tuning
Indexing and Performance TuningMongoDB
 
Indexing and Query Optimizer (Aaron Staple)
Indexing and Query Optimizer (Aaron Staple)Indexing and Query Optimizer (Aaron Staple)
Indexing and Query Optimizer (Aaron Staple)MongoSF
 
MongoDB Days UK: Indexing and Performance Tuning
MongoDB Days UK: Indexing and Performance TuningMongoDB Days UK: Indexing and Performance Tuning
MongoDB Days UK: Indexing and Performance TuningMongoDB
 
MongoDB Indexing Constraints and Creative Schemas
MongoDB Indexing Constraints and Creative SchemasMongoDB Indexing Constraints and Creative Schemas
MongoDB Indexing Constraints and Creative SchemasMongoDB
 
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]Filip Tepper
 
Geo-Indexing w/MongoDB
Geo-Indexing w/MongoDBGeo-Indexing w/MongoDB
Geo-Indexing w/MongoDBLalit Kapoor
 
User Data Management with MongoDB
User Data Management with MongoDB User Data Management with MongoDB
User Data Management with MongoDB MongoDB
 
Webinar: Index Tuning and Evaluation
Webinar: Index Tuning and EvaluationWebinar: Index Tuning and Evaluation
Webinar: Index Tuning and EvaluationMongoDB
 
Indexing and Query Optimizer (Richard Kreuter)
Indexing and Query Optimizer (Richard Kreuter)Indexing and Query Optimizer (Richard Kreuter)
Indexing and Query Optimizer (Richard Kreuter)MongoDB
 
MongoDB Performance Tuning
MongoDB Performance TuningMongoDB Performance Tuning
MongoDB Performance TuningMongoDB
 
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...MongoDB
 
Webinar: Working with Graph Data in MongoDB
Webinar: Working with Graph Data in MongoDBWebinar: Working with Graph Data in MongoDB
Webinar: Working with Graph Data in MongoDBMongoDB
 
Optimizing MongoDB: Lessons Learned at Localytics
Optimizing MongoDB: Lessons Learned at LocalyticsOptimizing MongoDB: Lessons Learned at Localytics
Optimizing MongoDB: Lessons Learned at Localyticsandrew311
 

Viewers also liked (19)

Fast querying indexing for performance (4)
Fast querying   indexing for performance (4)Fast querying   indexing for performance (4)
Fast querying indexing for performance (4)
 
Indexing with MongoDB
Indexing with MongoDBIndexing with MongoDB
Indexing with MongoDB
 
Performance Tuning on the Fly at CMP.LY
Performance Tuning on the Fly at CMP.LYPerformance Tuning on the Fly at CMP.LY
Performance Tuning on the Fly at CMP.LY
 
MongoDB and Indexes - MUG Denver - 20160329
MongoDB and Indexes - MUG Denver - 20160329MongoDB and Indexes - MUG Denver - 20160329
MongoDB and Indexes - MUG Denver - 20160329
 
Indexing and Performance Tuning
Indexing and Performance TuningIndexing and Performance Tuning
Indexing and Performance Tuning
 
Indexing and Query Optimizer (Aaron Staple)
Indexing and Query Optimizer (Aaron Staple)Indexing and Query Optimizer (Aaron Staple)
Indexing and Query Optimizer (Aaron Staple)
 
MongoDB Days UK: Indexing and Performance Tuning
MongoDB Days UK: Indexing and Performance TuningMongoDB Days UK: Indexing and Performance Tuning
MongoDB Days UK: Indexing and Performance Tuning
 
MongoDB Indexing Constraints and Creative Schemas
MongoDB Indexing Constraints and Creative SchemasMongoDB Indexing Constraints and Creative Schemas
MongoDB Indexing Constraints and Creative Schemas
 
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
 
Geo-Indexing w/MongoDB
Geo-Indexing w/MongoDBGeo-Indexing w/MongoDB
Geo-Indexing w/MongoDB
 
User Data Management with MongoDB
User Data Management with MongoDB User Data Management with MongoDB
User Data Management with MongoDB
 
Webinar: Index Tuning and Evaluation
Webinar: Index Tuning and EvaluationWebinar: Index Tuning and Evaluation
Webinar: Index Tuning and Evaluation
 
Phplx mongodb
Phplx mongodbPhplx mongodb
Phplx mongodb
 
Indexing and Query Optimizer (Richard Kreuter)
Indexing and Query Optimizer (Richard Kreuter)Indexing and Query Optimizer (Richard Kreuter)
Indexing and Query Optimizer (Richard Kreuter)
 
Indexing
IndexingIndexing
Indexing
 
MongoDB Performance Tuning
MongoDB Performance TuningMongoDB Performance Tuning
MongoDB Performance Tuning
 
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
 
Webinar: Working with Graph Data in MongoDB
Webinar: Working with Graph Data in MongoDBWebinar: Working with Graph Data in MongoDB
Webinar: Working with Graph Data in MongoDB
 
Optimizing MongoDB: Lessons Learned at Localytics
Optimizing MongoDB: Lessons Learned at LocalyticsOptimizing MongoDB: Lessons Learned at Localytics
Optimizing MongoDB: Lessons Learned at Localytics
 

Similar to MongoDB Indexing Guide

Data Representation - Day 2
Data Representation - Day 2Data Representation - Day 2
Data Representation - Day 2blprnt
 
Processing & Dataviz
Processing & DatavizProcessing & Dataviz
Processing & Datavizblprnt
 
JSON-stat & JS: the JSON-stat Javascript Toolkit
JSON-stat & JS: the JSON-stat Javascript ToolkitJSON-stat & JS: the JSON-stat Javascript Toolkit
JSON-stat & JS: the JSON-stat Javascript ToolkitXavier Badosa
 
Haskell in the Real World
Haskell in the Real WorldHaskell in the Real World
Haskell in the Real Worldosfameron
 
Thinking Functionally In Ruby
Thinking Functionally In RubyThinking Functionally In Ruby
Thinking Functionally In RubyRoss Lawley
 

Similar to MongoDB Indexing Guide (6)

Data Representation - Day 2
Data Representation - Day 2Data Representation - Day 2
Data Representation - Day 2
 
Processing & Dataviz
Processing & DatavizProcessing & Dataviz
Processing & Dataviz
 
JSON-stat & JS: the JSON-stat Javascript Toolkit
JSON-stat & JS: the JSON-stat Javascript ToolkitJSON-stat & JS: the JSON-stat Javascript Toolkit
JSON-stat & JS: the JSON-stat Javascript Toolkit
 
Haskell in the Real World
Haskell in the Real WorldHaskell in the Real World
Haskell in the Real World
 
Trees
TreesTrees
Trees
 
Thinking Functionally In Ruby
Thinking Functionally In RubyThinking Functionally In Ruby
Thinking Functionally In Ruby
 

Recently uploaded

Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostZilliz
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 

Recently uploaded (20)

Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage CostLeverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
Leverage Zilliz Serverless - Up to 50X Saving for Your Vector Storage Cost
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 

MongoDB Indexing Guide

  • 1. Indexing with MongoDB Luke Ehresman CopperEgg - www.copperegg.com
  • 2. What is an index?
  • 3. What is an index? • Pointers to documents
  • 4. What is an index? • Pointers to documents • Efficiently organized for quick scanning
  • 5. What is an index? • Pointers to documents • Efficiently organized for quick scanning • Maintained in a tree structure
  • 6. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 7. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 8. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 9. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 10. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 11. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 12. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 13. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 14. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9} 6 documents scanned
  • 15. How do we find x = 6 with an index?
  • 16. How do we find x = 6 with an index? {x:4} < > {x:2} {x:6} < > < > {x:1} {x:3} {x:5} {x:7}
  • 17. How do we find x = 6 with an index? {x:4} < > {x:2} {x:6} < > < > {x:1} {x:3} {x:5} {x:7}
  • 18. How do we find x = 6 with an index? {x:4} < > {x:2} {x:6} < > < > {x:1} {x:3} {x:5} {x:7} Only 2 documents scanned
  • 19. In reality, it’s even better than that...
  • 20. In reality, it’s even better than that... {x:0, name:‘George’, ....} {x:1, name:‘Jane’, ....} {x:2, name:‘Judy’, ....} {x:3, name:‘Elroy’, ....} {x:4, name:‘Astro’, ....} {x:5, name:‘Rosie’, ....} {x:6, name:‘Fred’, ....} {x:7, name:‘Wilma’, ....} {x:8, name:‘Barney’, ....} {x:9, name:‘Betty’, ....}
  • 21. In reality, it’s even better than that... {x:4} {x:0, name:‘George’, ....} {x:1, name:‘Jane’, ....} < > {x:2, name:‘Judy’, ....} {x:3, name:‘Elroy’, ....} {x:4, name:‘Astro’, ....} {x:5, name:‘Rosie’, ....} {x:2} {x:6} {x:6, name:‘Fred’, ....} {x:7, name:‘Wilma’, ....} < > < > {x:8, name:‘Barney’, ....} {x:9, name:‘Betty’, ....} {x:1} {x:3} {x:5} {x:7}
  • 22. In reality, it’s still even better than that...
  • 23. In reality, it’s still even better than that... •MongoDB stores data and indexes in extents
  • 24. In reality, it’s still even better than that... •MongoDB stores data and indexes in extents •Only loads extents into memory as needed
  • 25. In reality, it’s still even better than that... •MongoDB stores data and indexes in extents •Only loads extents into memory as needed •The full index does not need to be in memory
  • 26. In reality, it’s still even better than that... •MongoDB stores data and indexes in extents •Only loads extents into memory as needed •The full index does not need to be in memory {x:4} < > {x:2} {x:6} < > < > {x:1} {x:3} {x:5} {x:7}
  • 28. When to index? • Frequently queried fields
  • 29. When to index? • Frequently queried fields • Low response time
  • 30. When to index? • Frequently queried fields • Low response time • Sorting
  • 31. When to index? • Frequently queried fields • Low response time • Sorting • Avoid full collection scans
  • 33. Things to know... • Indexes maintain order
  • 34. Things to know... • Indexes maintain order • Indexes will slow writes
  • 35. Things to know... • Indexes maintain order • Indexes will slow writes • Indexes take up space
  • 36. Things to know... • Indexes maintain order • Indexes will slow writes • Indexes take up space • Index maintenance will cause writes
  • 37. Things to know... • Indexes maintain order • Indexes will slow writes • Indexes take up space • Index maintenance will cause writes • “unique” parameter forces uniqueness
  • 38. How to create an index
  • 39. How to create an index db.people.ensureIndex({...})
  • 40. How to create an index db.people.ensureIndex({...}) {_id:1} is already created for each collection
  • 41. How to create an index db.people.ensureIndex({...}) {_id:1} is already created for each collection You can call ensureIndex multiple times (it only creates an index if it doesn’t already exist)
  • 42. How to create an index db.people.ensureIndex({...}) {_id:1} is already created for each collection You can call ensureIndex multiple times (it only creates an index if it doesn’t already exist) Blocks your mongod process unless you specify background:true.
  • 44. Indexes on missing fields null is assumed if a field is missing
  • 45. Indexes on missing fields null is assumed if a field is missing db.people.insert({name:‘Fred’, children:[‘Pebbles’]}); db.people.insert({name:‘Pebbles’}) db.people.ensureIndex({children:1}) db.people.find({children:null}); (returns Pebbles and uses the index)
  • 46. Indexes on missing fields null is assumed if a field is missing db.people.insert({name:‘Fred’, children:[‘Pebbles’]}); db.people.insert({name:‘Pebbles’}) db.people.ensureIndex({children:1}) db.people.find({children:null}); (returns Pebbles and uses the index) Optional sparse indexes exclude empty fields for efficiency
  • 48. Compound indexes { name: “George”, age:45, gender:”M”, ... }
  • 49. Compound indexes { name: “George”, {gender:1, age:-1} age:45, gender:”M”, ... }
  • 50. Compound indexes { name: “George”, {gender:1, age:-1} age:45, gender:”M”, ... } gender:”M” age:45 {name:‘George’, ....} age:12 {name:‘Jane’, ....} age:3 {name:‘Judy’, ....} gender:”F” {name:‘Elroy’, ....} age:44 {name:‘Astro’, ....} age:16 {name:‘Rosie’, ....} age:10
  • 52. Indexes on arrays { name: “George”, age:45, gender:”M”, children: [‘Judy’, ‘Elroy’] ... }
  • 53. Indexes on arrays { name: “George”, age:45, {children:1} gender:”M”, children: [‘Judy’, ‘Elroy’] ... }
  • 54. Indexes on arrays { name: “George”, age:45, {children:1} gender:”M”, children: [‘Judy’, ‘Elroy’] ... } Actually creates a new entry in the index for each element of the array.
  • 55. Indexes on arrays { name: “George”, age:45, {children:1} gender:”M”, children: [‘Judy’, ‘Elroy’] ... } Actually creates a new entry in the index for each element of the array. children:‘Judy’ {name:‘George’, ....} children:‘Elroy’ {name:‘Fred’, ...} children:‘Bam-Bam’ {name:‘Barney’ ...}
  • 57. Covered indexes Queries that can be resolved with only the index (does not need to fetch the original document)
  • 58. Covered indexes Queries that can be resolved with only the index (does not need to fetch the original document) { name: “George”, age:45, gender:”M”, children: [‘Judy’, ‘Elroy’] ... }
  • 59. Covered indexes Queries that can be resolved with only the index (does not need to fetch the original document) { name: “George”, age:45, db.people.ensureIndex({name:1, age:1}); gender:”M”, db.people.find({name:‘George’}, {_id:0, age:1}); children: [‘Judy’, ‘Elroy’] ... }
  • 60. How does it know which index to use?
  • 61. How does it know which index to use? db.people.ensureIndex({name:1}); db.people.ensureIndex({age:1}); db.people.find({age:{$gte:21}});
  • 62. How does it know which index to use? db.people.ensureIndex({name:1}); db.people.ensureIndex({age:1}); db.people.find({age:{$gte:21}}); scan index on name index on age
  • 63. How does it know which index to use? db.people.ensureIndex({name:1}); db.people.ensureIndex({age:1}); db.people.find({age:{$gte:21}}); scan terminated index on name index on age remember
  • 64. Query performance analysis db.people.find({name:‘Fred’}).explain() { “cursor” : “BasicCursor”, “indexBounds” : {}, “nscanned” : 88731, “nscannedObjects” : 88731, “n” : 1, “millis” : 108 }
  • 67. Credits • Much of this information was gleaned from these two presentations: • http://speakerdeck.com/u/mongodb/p/ indexing-and-query-optimizer-kevin- hanson-10gen • http://www.slideshare.net/mongodb/ indexing-with-mongodb

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n
  31. \n
  32. \n
  33. \n
  34. \n
  35. \n
  36. \n
  37. \n
  38. \n
  39. \n
  40. \n
  41. \n
  42. \n
  43. \n
  44. \n
  45. \n
  46. \n
  47. \n
  48. \n
  49. \n
  50. \n
  51. \n
  52. \n
  53. \n
  54. \n