The document provides a comprehensive guide on optimizing MongoDB queries, focusing on how to structure compound indexes effectively using the e-s-r (equality, sort, range) principle. It discusses the importance of placing equality predicates first, followed by sort predicates, and then range predicates to enhance performance and avoid blocking sorts. Additionally, it covers different types of predicates and their implications for indexing strategies.

1. Chris Harris
Tips and Tricks++ for Querying MongoDB

2. Chris Harris
Lead Technical Expert for Queries and Indexing

3. Introductions
Diego

4. Introductions
I identified slow queries.
What index do I make?

5. Roadmap
§ E-S-R
§ Overview
§ Predicate Type Check
§ Sequential Prefix
§ Exceptions?
§ Multikey Index
§ Definition
§ Semantics
§ Performance

6. Roadmap
§ E-S-R
§ Overview
§ Predicate Type Check
§ Sequential Prefix
§ Exceptions?
§ Multikey Index
§ Definition
§ Semantics
§ Performance

7. Equality Sort Range "Rule"

8. E – S - R
The ordering of index keys in a
compound index is critically
important. E-S-R provides
guidance that is useful in most
cases:
• Equality first
• Sort next
• Range last

9. E – S - R
The ordering of index keys in a
compound index is critically
important. E-S-R provides
guidance that is useful in most
cases:
• Equality first
• Sort next
• Range last
What is the difference between
Equality and Range?

10. Definitions
Equality Fields
An exact match on a single
value. For example:
• {x:123}
• {x:{$eq:123}}
• {x:"123"}
• {"x.y":123}

11. Definitions
Equality Fields
An exact match on a single
value. For example:
• {x:123}
• {x:{$eq:123}}
• {x:"123"}
• {"x.y":123}
Sort
The (entire) requested sort.
.sort({x:1, y:-1})

12. Definitions
Equality Fields
An exact match on a single
value. For example:
• {x:123}
• {x:{$eq:123}}
• {x:"123"}
• {"x.y":123}
Sort
The (entire) requested sort.
.sort({x:1, y:-1})
Range Predicates
Any predicates that are not
exact matches. Some
operators include:
• {x:{$gt:0}}
• {x:{$lte:1000}}

13. Equality
Equality keys are placed first in any order
If present in the query shape, equality fields should always form the
prefix for the index.
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )

14. Equality
Equality keys are placed first in any order
If present in the query shape, equality fields should always form the
prefix for the index.
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )

15. Equality
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
{gamertag: "Ace", score: 100}
{gamertag: "Ace", score: 99,999}
{gamertag: "Bob", score: 15,000}
{gamertag: "Bob", score: 50,000}

16. Equality
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
{} {}
{}{}
100 15,000 50,000 99,999
Ace Bob Bob Ace
{gamertag: "Ace", score: 100}
{gamertag: "Ace", score: 99,999}
{gamertag: "Bob", score: 15,000}
{gamertag: "Bob", score: 50,000}

17. Equality
100 15,000 50,000 99,999
Ace Bob Bob Ace
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
{} {} {} {}

18. Equality
100 15,000 50,000 99,999
Ace Bob Bob Ace
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
{} {} {} {}

19. Equality
100 15,000 50,000 99,999
Ace Bob Bob Ace
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
{} {} {} {}

20. Equality
100 15,000 50,000 99,999
Ace Bob Bob Ace
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
{} {} {} {}

21. Equality
100 15,000 50,000 99,999
Ace Bob Bob Ace
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
{} {} {} {}

22. Equality
100 15,000 50,000 99,999
Ace Bob Bob Ace
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
{} {} {} {}

23. Equality
100 15,000 50,000 99,999
Ace Bob Bob Ace
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
{} {} {} {}

24. Equality
100 15,000 50,000 99,999
Ace Bob Bob Ace
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
{} {} {} {}

25. Equality
Ace Bob Bob Ace
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
100 99,999 15,000 50,000
{} {} {} {}

26. Equality
100
Ace Bob
99,999 15,000 50,000
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
{} {} {} {}

27. Equality
100
Ace Bob
99,999 15,000 50,000
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
{} {} {} {}

28. Equality
100
Ace Bob
99,999 15,000 50,000
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
{} {} {} {}

29. Equality before Range
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
R
E R
E

30. Sort
Sort fields are placed next
Placing sort predicates after sequential equality keys allow for the
index to:
• Provide a non-blocking sort.
• Minimize the amount of scanning required.
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )

31. Sort
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
100 15,000 50,000 99,999
Ace Bob Bob Ace
{} {}{} {}

32. Sort
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
100 15,000 50,000 99,999
Ace Bob Bob Ace
{} {}{} {}

33. Sort
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
{}
100 15,000 50,000 99,999
Ace Bob Bob Ace
{} {} {}

34. Sort
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
15,000 50,000 99,999
Ace Bob Bob Ace
{} {}{} {}
100

35. Sort
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
100 15,000 50,000 99,999
Ace Bob Bob Ace
{} {}{} {}

36. Sort
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
100 15,000 50,000 99,999
Ace Bob Bob Ace
{} {}{} {}

37. Sort
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
100 15,000 50,000 99,999
Ace Bob Bob Ace
{} {}{} {}

38. Sort
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
100 15,000 50,000 99,999
Ace Bob Bob Ace
{} {}{} {}

39. Sort
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
100 15,000 50,000 99,999
Ace Bob Bob Ace
{} {}{} {}

40. Sort
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
100 15,000 50,000 99,999
Ace Bob Bob Ace
{} {}{} {}

41. Sort
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
100 15,000 50,000 99,999
Ace Bob Bob Ace
{} {}{} {}

42. Sort
Ace Bob Bob Ace
100 99,999 15,000 50,000
{} {} {} {}
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )

43. Sort
100
Ace Bob
{}
99,999 15,000 50,000
{} {}
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
{}

44. Sort
100
Ace Bob
{}
99,999 15,000 50,000
{} {}
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
{}

45. Sort
100
Ace Bob
{}
99,999 15,000 50,000
{} {}
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
{}

46. Sort
100
Ace Bob
{}
99,999 15,000 50,000
{} {}
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
{}

47. Sort after Equality
db.games.find( {gamertag: "Ace"} ).sort( {score: 1} )
S
E S
E

48. Range
Range fields are usually last
Generally range predicates should be placed last.
This allows them to still participate in filtering the data, but does not
force a blocking sort.
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )

49. Range after Equality
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
R
E R
E

50. Range
db.games.find( {gamertag: "Ace", score: {$gt: 9000}} )
Should range come after sort
too?
R
E R
E

51. Range
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})
Should range come after sort
too?
R
E R
E

52. Range
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})
100 15,000 50,000 99,999
Ace Bob Cali Ace
{} {}{} {}

53. Range
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})
100 15,000 50,000 99,999
Ace Bob Cali Ace
{} {}{} {}{1}{3}{2}

54. {1}{3}{2}
Range
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})
100 15,000 50,000 99,999
Ace Bob Cali Ace
{}

55. {1}{3}{2}
Range
100 15,000 50,000 99,999
Ace Bob Cali Ace
{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

56. {1}{3}{2}
Range
100 15,000 50,000 99,999
Ace Bob Cali Ace
{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

57. {1}{3}{2}
Range
100 15,000 50,000 99,999
Ace Bob Cali Ace
{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

58. {1}{3}{2}
Range
100 15,000 50,000 99,999
Ace Bob Cali Ace
{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

59. {1}{3}{2}
Range
100 15,000 50,000 99,999
Ace Bob Cali Ace
{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

60. {1}{3}{2}
Range
100 15,000 50,000 99,999
Ace Bob Cali Ace
{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

61. {1}{3}{2}
Range
100 15,000 50,000 99,999
Ace Bob Cali Ace
{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

62. {1}{3}{2}
Range
{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

63. {1}{3}{2}
Range
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})
Blocking SortBlocking Sort
BS

64. {1}{3}{2}
Range
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})
BS

65. Range
100 15,000 50,000 99,999
Ace Bob Cali Ace
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

66. Range
Ace Bob Cali Ace
100 15,000 50,00099,999
{1} {3}{2}{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

67. Range
100 15,000 50,00099,999
Ace Bob Cali
{1} {3}{2}{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

68. Range
100 15,000 50,00099,999
Ace Bob Cali
{1} {3}{2}{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

69. Range
100 15,000 50,00099,999
Ace Bob Cali
{1} {3}{2}{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

70. Range
100 15,000 50,00099,999
Ace Bob Cali
{1} {3}{2}{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

71. Range
100 15,000 50,00099,999
Ace Bob Cali
{1} {3}{2}{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

72. Range
100 15,000 50,00099,999
Ace Bob Cali
{1} {3}{2}{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

73. Range
100 15,000 50,00099,999
Ace Bob Cali
{1} {3}{2}{}
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})

74. Range after Sort
db.games.find( {score: {$gt: 9000}} ).sort({gamertag: 1})
BS
R S
R
S

75. Roadmap
§ E-S-R
§ Overview
§ Predicate Type Check
§ Sequential Prefix
§ Exceptions?
§ Multikey Index
§ Definition
§ Semantics
§ Performance

76. Predicate Type Check
What are the types of the following operators?

77. Predicate Type Check - Inequality
Inequality operators, such as the
following, are E, S, or R?
§ $ne:123
§ $nin:[1,2]
Range!

78. Predicate Type Check - Inequality
Inequality operators, such as the
following, are E, S, or R?
§ $ne:123
§ $nin:[1,2]
Range!
How do you know?

79. Predicate Type Check - Inequality
Inequality operators, such as the
following, are E, S, or R?
§ $ne:123
§ $nin:[1,2]
Range!
"indexBounds" : {
"x" : [
"[MinKey, 123.0)",
"(123.0, MaxKey]"
]
}

80. Predicate Type Check - Inequality
Range!
"indexBounds" : {
"x" : [
"[MinKey, 123.0)",
"(123.0, MaxKey]"
]
}
50 123 999

81. Predicate Type Check - Inequality
Range!
"indexBounds" : {
"x" : [
"[MinKey, 123.0)",
"(123.0, MaxKey]"
]
}
50 123 999

82. Predicate Type Check - Regex
Regex operators, such as the
following, are E, S, or R?
§ {str:/car/}
§ {str:/^car/i}
Range!

83. Predicate Type Check - Regex
Regex operators, such as the
following, are E, S, or R?
§ {str:/car/}
§ {str:/^car/i}
Range!

84. Predicate Type Check - Regex
Regex operators, such as the
following, are E, S, or R?
§ {str:/car/}
§ {str:/^car/i}
Range!
planecar racecar

85. Predicate Type Check - Regex
Regex operators, such as the
following, are E, S, or R?
§ {str:/car/}
§ {str:/^car/i}
Range!
planecar racecar

86. Predicate Type Check - Regex
Regex operators, such as the
following, are E, S, or R?
§ {str:/car/}
§ {str:/^car/i}
Range!
planecar racecar

87. Predicate Type Check - Regex
Regex operators, such as the
following, are E, S, or R?
§ {str:/car/}
§ {str:/^car/i}
Range!

88. Predicate Type Check - Regex
Regex operators, such as the
following, are E, S, or R?
§ {str:/car/}
§ {str:/^car/i}
Range!
raincarry Carpool

89. Predicate Type Check - Regex
Regex operators, such as the
following, are E, S, or R?
§ {str:/car/}
§ {str:/^car/i}
Range!
raincarry Carpool

90. Predicate Type Check - Regex
Regex operators, such as the
following, are E, S, or R?
§ {str:/car/}
§ {str:/^car/i}
Range!
raincarry Carpool

91. Predicate Type Check - $in
$in filters, as demonstrated below,
are E, S, or R?
§ {field:{$in:[1,3]}}
… it depends! Can be Equality or
Range in terms of the key ordering

92. Predicate Type Check - $in
{field:{$in:[1,3]}}
"indexBounds" : {
"field" : [
"[1.0, 1.0]",
"[3.0, 3.0]"
],
"sortField" : [
"[MinKey, MaxKey]"
]
}
1 2 3
C G B F A D

93. Predicate Type Check - $in
{field:{$in:[1,3]}}
"indexBounds" : {
"field" : [
"[1.0, 1.0]",
"[3.0, 3.0]"
],
"sortField" : [
"[MinKey, MaxKey]"
]
}
1 2 3
C G B F A D

94. Predicate Type Check - $in
{field:{$in:[1,3]}}
"indexBounds" : {
"field" : [
"[1.0, 1.0]",
"[3.0, 3.0]"
],
"sortField" : [
"[MinKey, MaxKey]"
]
}
1 2 3
C G B F A D

95. Predicate Type Check - $in
{field:{$in:[1,3]}}
"indexBounds" : {
"field" : [
"[1.0, 1.0]",
"[3.0, 3.0]"
],
"sortField" : [
"[MinKey, MaxKey]"
]
}
1 2 3
C G B F A D

96. Predicate Type Check - $in
{field:{$in:[1,3]}}
"indexBounds" : {
"field" : [
"[1.0, 1.0]",
"[3.0, 3.0]"
],
"sortField" : [
"[MinKey, MaxKey]"
]
}
1 2 3
C G B F A D

97. Predicate Type Check - $in
{field:{$in:[1,3]}}
"indexBounds" : {
"field" : [
"[1.0, 1.0]",
"[3.0, 3.0]"
],
"sortField" : [
"[MinKey, MaxKey]"
]
}
1 2 3
C G B F A D
BS

98. Predicate Type Check - $in
{field:{$in:[1,3]}}
"indexBounds" : {
"field" : [
"[1.0, 1.0]",
"[3.0, 3.0]"
],
"sortField" : [
"[MinKey, MaxKey]"
]
}

99. Predicate Type Check - $in
{field:{$in:[1,3]}}

100. Predicate Type Check - $in
{field:{$in:[1,3]}}
"indexBounds" : {
"field" : [
"[1.0, 1.0]"
],
"sortField" : [
"[MinKey, MaxKey]"
]
}
1 2 3
C G B F A D
{} {} {} {} {} {}

101. Predicate Type Check - $in
1 2 3
C G B F A D
1 2 3
C G B F A D
{} {} {} {} {} {} {} {} {} {} {} {}
{field:{$in:[1,3]}}
"indexBounds" : {
"field" : [
"[1.0, 1.0]"
],
"sortField" : [
"[MinKey, MaxKey]"
]
}
"indexBounds" : {
"field" : [
"[3.0, 3.0]"
],
"sortField" : [
"[MinKey, MaxKey]"
]
}

102. Predicate Type Check - $in
{field:{$in:[1,3]}}
"indexBounds" : {
"field" : [
"[1.0, 1.0]"
],
"sortField" : [
"[MinKey, MaxKey]"
]
}
"indexBounds" : {
"field" : [
"[3.0, 3.0]"
],
"sortField" : [
"[MinKey, MaxKey]"
]
}
1 2 3
C G B F A D
1 2 3
C G B F A D
Sort Merge
{} {} {} {} {} {} {} {} {} {} {} {}

103. Predicate Type Check - $in
BS
Depends on the length of the $in list
S M

104. Roadmap
§ E-S-R
§ Overview
§ Predicate Type Check
§ Sequential Prefix
§ Exceptions?
§ Multikey Index
§ Definition
§ Semantics
§ Performance

105. Sequential Equality Keys
Does it matter if there is missing predicate in the equality key list?
For example, given:
§ Query: {gamertag:"Ace", game: "Halo"}
§ Index: {gamertag:1, date:1, game:1}
Is that index as efficient as the following one?
{gamertag:1, game:1, date:1}

106. Sequential Equality Keys
Answer: It depends! … but probably not
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"date" : [
"[MinKey, MaxKey]"
],
"game" : [
"["Halo", "Halo"]"
]
}

107. Sequential Equality Keys
Ace
Mario Halo
2016 2017
Mario Halo
2018 2019
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"date" : [
"[MinKey, MaxKey]"
],
"game" : [
"["Halo", "Halo"]"
]
}

108. Sequential Equality Keys
Ace
Mario Halo
2016 2017
Mario Halo
2018 2019
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"date" : [
"[MinKey, MaxKey]"
],
"game" : [
"["Halo", "Halo"]"
]
}

109. Sequential Equality Keys
Ace
Mario Halo
2016 2017
Mario Halo
2018 2019
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"date" : [
"[MinKey, MaxKey]"
],
"game" : [
"["Halo", "Halo"]"
]
}

110. Sequential Equality Keys
Ace
Mario Halo
2016 2017
Mario Halo
2018 2019
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"date" : [
"[MinKey, MaxKey]"
],
"game" : [
"["Halo", "Halo"]"
]
}

111. Sequential Equality Keys
Ace
Mario Halo
2016 2017
Mario Halo
2018 2019
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"date" : [
"[MinKey, MaxKey]"
],
"game" : [
"["Halo", "Halo"]"
]
}

112. Sequential Equality Keys
Ace
Mario Halo
2016 2017
Mario Halo
2018 2019
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"date" : [
"[MinKey, MaxKey]"
],
"game" : [
"["Halo", "Halo"]"
]
}

113. Sequential Equality Keys
Ace
Mario Halo
2016 2017
Mario Halo
2018 2019
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"date" : [
"[MinKey, MaxKey]"
],
"game" : [
"["Halo", "Halo"]"
]
}

114. Sequential Equality Keys
Ace
Mario Halo
2016 2017
Mario Halo
2018 2019
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"date" : [
"[MinKey, MaxKey]"
],
"game" : [
"["Halo", "Halo"]"
]
}

115. Sequential Equality Keys
Ace
Mario Halo
2016 2017
Mario Halo
2018 2019
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"date" : [
"[MinKey, MaxKey]"
],
"game" : [
"["Halo", "Halo"]"
]
}

116. "indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"game" : [
"["Halo", "Halo"]"
]
}
Sequential Equality Keys
Ace
Mario Halo
2016 2017
Mario Halo
2018 2019
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"date" : [
"[MinKey, MaxKey]"
],
"game" : [
"["Halo", "Halo"]"
]
}

117. Halo
Sequential Equality Keys
Ace
Mario Halo
2016 2017
Mario
2018 2019
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"date" : [
"[MinKey, MaxKey]"
],
"game" : [
"["Halo", "Halo"]"
]
}
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"game" : [
"["Halo", "Halo"]"
]
}

118. "indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"game" : [
"["Halo", "Halo"]"
]
}
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"game" : [
"["Halo", "Halo"]"
]
}
"indexBounds" : {
"gamertag" : [
"["Ace", "Ace"]"
],
"game" : [
"["Halo", "Halo"]"
],
"date" : [
"[MinKey, MaxKey]"
]
}
Sequential Equality Keys
Ace
2016 2017Mario2018 2019Halo

119. Sequential Equality Keys
R
E
E E
E
E
E

120. Roadmap
§ E-S-R
§ Overview
§ Predicate Type Check
§ Sequential Prefix
§ Exceptions?
§ Multikey Index
§ Definition
§ Semantics
§ Performance

121. Exceptions
Is this "rule" always optimal?
Not always.
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})

122. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
1 5,000 9,0012 9,000
… …
2001 2000 2015 2019 2019

123. Exceptions
Ace
1 5,000 9,0012 9,000
… …
2001 2000 2015 2019 2019
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})

124. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
1 5,000 9,0012 9,000
… …
2001 2000 2015 2019 2019

125. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
1 5,000 9,0012 9,000
… …
2001 2000 2015 2019 2019

126. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
1 5,000 9,0012 9,000
… …
2001 2000 2015 2019 2019

127. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
1 5,000 9,0012 9,000
… …
2001 2000 2015 2019 2019

128. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
1 5,000 9,0012 9,000
… …
2001 2000 2015 2019 2019

129. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
1 5,000 9,0012 9,000
… …
2001 2000 2015 2019 2019

130. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
1 5,000 9,0012 9,000
… …
2001 2000 2015 2019 2019

131. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
1 5,000 9,0012 9,000
… …
2001 2000 2015 2019 2019

132. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
1 5,000 9,0012 9,000
… …
2001 2000 2015 2019 2019

133. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
1 5,000 9,0012 9,000
… …
2001 2000 2015 2019 2019

134. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
1 5,000 9,0012 9,000
… …
2001 2000 2015 2019 2019
"executionStats" : {
"nReturned" : 2,
"executionTimeMillis" : 23,
"totalKeysExamined" : 9001,
"totalDocsExamined" : 2,

135. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
1 5,000 9,0012 9,000
… …
2001 2000 2015 2019 2019

136. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
2001 2000 2015 2019 2019
{score:2} {score:1} {score:5000} {score:9001} {score:9000}
… …

137. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
… …
2000 2001 2015 2019 2019
{score:2} {score:1} {score:5000} {score:9001} {score:9000}

138. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
… …
2000 2001 2015 2019 2019
{score:2} {score:1} {score:5000} {score:9001} {score:9000}

139. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
… …
2000 2001 2015 2019 2019
{score:2} {score:1} {score:5000} {score:9001} {score:9000}
BS
"executionStats" : {
"nReturned" : 2,
"executionTimeMillis" : 0,
"totalKeysExamined" : 2,
"totalDocsExamined" : 2,

140. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
Ace
… …
2000 2001 2015 2019 2019
{score:2} {score:1} {score:5000}
"executionStats" : {
"nReturned" : 2,
"executionTimeMillis" : 0,
"totalKeysExamined" : 2,
"totalDocsExamined" : 2,
"executionStages" : {
"stage" : "SORT",
"memUsage" : 154,
"memLimit" : 33554432,
{score:9001}{score:9000}
BS

141. Exceptions
db.games
.find({gamertag:"Ace", date:{$gte:2019}})
.sort({score:1})
……… …
BS
S
R
E
R
E

142. E-S-R Guidance
A good starting place applicable to most use cases
Place keys in the following order:
• Equality first
• Sort next
• Range last
Remember that equality is an exact match of a single value

143. E-S-R Guidance
A good starting place applicable to most use cases
Place keys in the following order:
• Equality first
• Sort next
• Range last
Remember that equality is an exact match of a single value
I see isMultiKey:true in
the explain output. What
does that mean?

144. Roadmap
§ E-S-R
§ Overview
§ Predicate Type Check
§ Sequential Prefix
§ Exceptions?
§ Multikey Index
§ Definition
§ Semantics
§ Performance

145. Multikey
Index
A special type of index which
supports efficient querying for
array fields

146. Index Nomenclature
Single Field Index
The index entries reference
a single field in the
document
Can be Multikey
Compound Index
The index entries reference
more than one field in the
document
Can be Multikey
Multikey Index
There is separate entry in
the index for every array in
the document

147. Single Field Index
Ace Doyen
{ gamertag: "Ace",
games:["Mario", "Halo"] }
{ gamertag: "Doyen",
games:["Fortnite", "GTA", "Minecraft"] }{} {}

148. Single Field Index - Multikey
{} {}
MarioHaloGTAFortnite
Minecraft

149. DoyenAce
Compound Index - Multikey
{} {}
MarioHaloGTAFortnite
Minecraft

150. Compound Index - Multikey
HaloMinecraftGTAFortnite
Mario
AceDoyen
{}{}
We tried to use arrays but get
incorrect results and queries are
slow. What gives?

151. Roadmap
§ E-S-R
§ Overview
§ Predicate Type Check
§ Sequential Prefix
§ Exceptions?
§ Multikey Index
§ Definition
§ Semantics
§ Performance

152. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }

153. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }

154. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }

155. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
db.matches.find({ "players.name": "Ace",
"players.rank": 1 })

156. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
db.matches.find({ "players.name": "Ace",
"players.rank": 1 })

157. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
db.matches.find({ "players.name": "Ace",
"players.rank": 1 })

158. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
"You can specify the query such that either a single array element meets these conditions

159. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
"You can specify the query such that either a single array element meets these conditions or
any combination of array elements meets the conditions."

160. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
"You can specify the query such that either a single array element meets these conditions or
any combination of array elements meets the conditions."
db.matches.find({ "players.name": "Ace", "players.rank": 1 })

161. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
"You can specify the query such that either a single array element meets these conditions or
any combination of array elements meets the conditions."
db.matches.find({ "players.name": "Ace", "players.rank": 1 })

162. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
"You can specify the query such that either a single array element meets these conditions or
any combination of array elements meets the conditions."
db.matches.find({ "players.name": "Ace", "players.rank": 1 })

163. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
"You can specify the query such that either a single array element meets these conditions or
any combination of array elements meets the conditions."
db.matches.find({ "players.name": "Ace", "players.rank": 1 })

164. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
"You can specify the query such that either a single array element meets these conditions or
any combination of array elements meets the conditions."
db.matches.find({ "players.name": "Ace", "players.rank": 1 })

165. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
"You can specify the query such that either a single array element meets these conditions or
any combination of array elements meets the conditions."
db.matches.find({ "players.name": "Ace", "players.rank": 1 })

166. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
"You can specify the query such that either a single array element meets these conditions or
any combination of array elements meets the conditions."
db.matches.find({ "players.name": "Ace", "players.rank": 1 })

167. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
"You can specify the query such that either a single array element meets these conditions or
any combination of array elements meets the conditions."
db.matches.find({ "players.name": "Ace", "players.rank": 1 })

168. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
"You can specify the query such that either a single array element meets these conditions or
any combination of array elements meets the conditions."
db.matches.find({ "players.name": "Ace", "players.rank": 1 })

169. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
"You can specify the query such that either a single array element meets these conditions or
any combination of array elements meets the conditions."
db.matches.find({ "players.name": "Ace", "players.rank": 1 })

170. Semantics
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
{ _id: 3, players: [ { name: "Doyen", rank : 1 },
{ name: "Cali", rank : 2 } ] }
"You can specify the query such that either a single array element meets these conditions or
any combination of array elements meets the conditions."
db.matches.find({ "players.name": "Ace", "players.rank": 1 })

171. Semantics - $elemMatch
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
db.matches.find({ "players.name": "Ace", "players.rank": 1 })

172. Semantics - $elemMatch
{ _id: 1, players: [ { name: "Doyen", rank : 1 },
{ name: "Ace", rank : 2 } ] }
{ _id: 2, players: [ { name: "Ace", rank : 1 },
{ name: "Doyen", rank : 2 } ] }
db.matches.find({ "players.name": "Ace", "players.rank": 1 })
db.matches.find({ "players":
{ $elemMatch: { name: "Ace", "rank": 1} }
})

173. Semantics - $elemMatch
Use $elemMatch to query multiple fields of a single array element
Not necessary when querying on a single predicate

174. Semantics - $elemMatch
Use $elemMatch to query multiple fields of a single array element
Not necessary when querying on a single predicate
Okay, that change got us
correct results. But what
about the performance?

175. Roadmap
§ E-S-R
§ Overview
§ Predicate Type Check
§ Sequential Prefix
§ Exceptions?
§ Multikey Index
§ Definition
§ Semantics
§ Performance

176. Multikey Path Tracking
{
"_id" : 1,
"game" : "Halo",
"players" : [
"Ace",
"Doyen",
"Cali",
"Bob"
],
"date" : ISODate("2019-02-15T00:00:00Z")
}

177. Multikey Path Tracking
{
"_id" : 1,
"game" : "Halo",
"players" : [
"Ace",
"Doyen",
"Cali",
"Bob"
],
"date" : ISODate("2019-02-15T00:00:00Z")
}
db.matches.find({
players:"Ace",
date:{
$gte:ISODate("2019-02-01"), $lte:ISODate("2019-02-28")
}
})

178. Multikey Path Tracking
{
"_id" : 1,
"game" : "Halo",
"players" : [
"Ace",
"Doyen",
"Cali",
"Bob"
],
"date" : ISODate("2019-02-15T00:00:00Z")
}
db.matches.find({
players:"Ace",
date:{
$gte:ISODate("2019-02-01"), $lte:ISODate("2019-02-28")
}
})
.explain()
…
"indexBounds" : {
"players" : [
"["Ace", "Ace"]"
],
"date" : [
"(true, new Date(1551312000000)]"
]
}

179. Multikey Path Tracking
{
"_id" : 1,
"game" : "Halo",
"players" : [
"Ace",
"Doyen",
"Cali",
"Bob"
],
"date" : ISODate("2019-02-15T00:00:00Z")
}
db.matches.find({
players:"Ace",
date:{
$gte:ISODate("2019-02-01"), $lte:ISODate("2019-02-28")
}
})
.explain()
…
"indexBounds" : {
"players" : [
"["Ace", "Ace"]"
],
"date" : [
"(true, new Date(1551312000000)]"
]
}

180. Multikey Path Tracking
{
"_id" : 1,
"game" : "Halo",
"players" : [
"Ace",
"Doyen",
"Cali",
"Bob"
],
"date" : ISODate("2019-02-15T00:00:00Z")
}
db.matches.find({
players:"Ace",
date:{
$gte:ISODate("2019-02-01"), $lte:ISODate("2019-02-28")
}
})
.explain()
…
"indexBounds" : {
"players" : [
"["Ace", "Ace"]"
],
"date" : [
"(true, new Date(1551312000000)]"
]
}

181. Multikey Path Tracking
{
"_id" : 1,
"game" : "Halo",
"players" : [
"Ace",
"Doyen",
"Cali",
"Bob"
],
"date" : ISODate("2019-02-15T00:00:00Z")
}
db.matches.find({
players:"Ace",
date:{
$gte:ISODate("2019-02-01"), $lte:ISODate("2019-02-28")
}
})
.explain()
…
"indexBounds" : {
"players" : [
"["Ace", "Ace"]"
],
"date" : [
"(true, new Date(1551312000000)]"
]
}

182. Multikey Path Tracking
Before MongoDB 3.4:
§ Database only contained coarse isMultiKey flag for index
§ Did not know which field was the array
§ Conservative behavior to ensure correct results

183. Multikey Path Tracking
Before MongoDB 3.4:
"stage" : "IXSCAN",
"indexName" : "players_1_date_1",
"isMultiKey" : true,
"indexBounds" : {
"players" : [
"["Ace", "Ace"]"
],
"date" : [
"(true, new Date(1551312000000)]"
]
}
3.4+:
...
"multiKeyPaths" : {
"players" : [ "players" ],
"date" : [ ]
},
"indexBounds" : {
"players" : [
"["Ace", "Ace"]"
],
"date" : [
"[new Date(1548979200000),
new Date(1551312000000)]"
]
}

184. Multikey Path Tracking
Take advantage of the improvement
§ Index needs to be rebuilt when running 3.4 binaries
§ Just running on 3.4 is not sufficient
§ Look for multiKeyPaths field in explain output
§ v:2 index would be a deterministic check
§ Multikey index cannot cover queries on the array field

185. Multikey Path Tracking
Take advantage of the improvement
§ Index needs to be rebuilt when running 3.4 binaries
§ Just running on 3.4 is not sufficient
§ Look for multiKeyPaths field in explain output
§ v:2 index would be a deterministic check
§ Multikey index cannot cover queries on the array field
Wow! Everything is running at
Web Scale now!

186. Summary

187. § E-S-R
§ Overview
§ Predicate Type Check
§ Sequential Prefix
§ Exceptions?
§ Multikey Index
§ Definition
§ Semantics
§ Performance

188. Q&A about Q&I

189. Additional Resources

190. Key Ordering Resources
Blog Post
My colleague Jesse wrote an article
which explores the performance
characteristics of different
arrangements.
MongoDB University
My colleague Kirby covers many
indexing topics in greater detail,
including key ordering, in M201:
MongoDB Performance.

191. Please provide Session Feedback
1. Go to slido.com
2. Enter event code #MDBW19
3. Click on Room Name and Provide Session
Feedback
Feedback poll will remain open for 10 minutes after the talk
ends
Questions?
Meet me now in the Leaf Lounge on the second floor of the
Partner Pavilion
YOU ARE IN MURRAY HILL