4. • Motivation
• Who am I?
• Three Items to be aware of:
• Blocking Stages
• Using the $or operator
• Case-insensitivity
Roadmap
5. The Power of Query Optimization
Query tuning results in:
• Improved performance
• Reduced resource utilization
This may lead to:
• Improved stability and predictability
• A smaller hardware footprint
Not uncommon to observe efficiency improvements greater than 99%
6. • Technical Services Engineer (Support)
• 2.5 year tenure
• Member of the Technical Experts program
• Focus: Queries and Indexing
• Previously: Data Warehouse workload optimization
About Me
7. • Technical Services Engineer (Support)
• 2.5 year tenure
• Member of the Technical Experts program
• Focus: Queries and Indexing
• Previously: Data Warehouse workload optimization
About Me
8. Meet Asya
• DBA at Acme Game, Inc.
• MongoDB Champion
Meet Stakeholders
• Others at Acme, Inc.
• Developers
• Leadership
• RDBMS Historically
10. • Game nearly complete
• Developers have learned a lot from Asya
Stakeholder Concerns
11. • Game nearly complete
• Developers have learned a lot from Asya
Stakeholder Concerns
Indexes support the efficient
execution of queries in MongoDB
12. • Game nearly complete
• Developers have learned a lot from Asya
Stakeholder Concerns
Indexes support the efficient
execution of queries in MongoDB
13. • Game nearly complete
• Developers have learned a lot from Asya
Stakeholder Concerns
Ace Bob Sue
… …Indexes support the efficient
execution of queries in MongoDB
14. • Game nearly complete
• Developers have learned a lot from Asya
Stakeholder Concerns
• App being stress tested
15. • Game nearly complete
• Developers have learned a lot from Asya
Stakeholder Concerns
• App being stress tested
• Concerns over current performance
16. Stakeholder Concern #1
Developers created index
db.games.createIndex({ gamerTag: 1 })
This query takes several seconds to execute:
db.games.find( { gamerTag: "Ace" } ).sort({score:-1})
Adding the index on score does not help!
db.games.createIndex({ score: -1 })
17. Stakeholder Concern #1
Developers created index
{db.games.createIndex({ gamerTag: 1 })
This query takes several seconds to execute:
db.games.find( { gamerTag: "Ace" } ).sort({score:-1})
Adding the index on score does not help!
{db.games.createIndex({ score: 1 })
“Clearly MongoDB
is not webscale!”
19. Blocking Operation
● Formally:
■ “An operation which must process all input before it can begin to produce
any output.”
● Opposite of the often desirable “fully pipelined” plan which can stream results
back as soon as they are found.
● Commonly observed when a sort is added to a query
48. Blocking Stages
• $sort
• In aggregation and find
• $group
• $bucket
• $count
• $facet
Are there any other blocking
operations?
49. Working with blocking stages
For sorting:
Add a supporting index
Worth the overhead in almost all circumstances
For other stages:
Do you need the blocking stage?
Offload to secondary member
52. Stakeholder Concern #1
Performance of
db.games.find( { gamerTag: "Ace" } ).sort({score:-1})
db.games.createIndex({ gamerTag: 1, score:-1 })
"That’ll work great!”
53. Stakeholder Concern #2
The $and version of a query
returns quickly:
db.games.find({
$and : [
{ gamerTag: "Ace" },
{ score: {$gt: 9000} }
]
})
But the $or version is slow:
db.games.find({
$or : [
{ gamerTag: "Ace" },
{ score: {$gt: 9000} }
]
})
54. Stakeholder Concern #2
The $and version of a query
returns quickly:
db.games.find({
$and : [
{ gamerTag: "Ace" },
{ score: {$gt: 9000} }
]
})
But the $or version is slow:
db.games.find({
$or : [
{ gamerTag: "Ace" },
{ score: {$gt: 9000} }
]
})
We just created an index with both
those fields… Can it be used?
111. Stakeholder Concern #3
“Wait wait wait, we can’t even FIND the gamers!”
A basic search on gamerTag takes several seconds already:
db.games.find({gamerTag: /Ace/i})
“This query is SLOWER with the index than it is without it!”
122. Recommendations
Case insensitive index!
Collations available since 3.4
db.games.createIndex( { gamerTag: 1},
{ collation: { locale: 'en', strength: 2 } } )
Store a transformed (eg toLower()) copy of the string
127. Work Smarter Not Harder
• Understand the business logic
• Index appropriately
• Is it the right index to support the query?
• Be aware of:
• Blocking Stages
• Usage of $or
• Case sensitivity
• Leverage the Performance Advisor
128. Work Smarter Not Harder
• Understand the business logic
• Index appropriately
• Is it the right index to support the query?
• Be aware of:
• Blocking Stages
• Usage of $or
• Case sensitivity
• Leverage the Performance Advisor