1. Building a Mobile App!
Part 1
{ name: ‘Bryan Reinero’,
title: ‘Developer Advocate’,
twitter: ‘@blimpyacht’,
code: ‘github.com/breinero’
email: ‘bryan@mongdb.com’ }

2. 2
Track Agenda
• Part 1: Architecture and Application Design
• Part 2: Geo-Spatial Indexing and Queries
• Part 3:Deployment Readiness

3. 3
Mobile Applications
• Location based data
• User relevance
• Context Rich
• Social Engagement

4. 4
The Scavenger Hunt App
Users create
scavenger hunts by
“pinning” waypoints

5. 5
The Scavenger Hunt App
Players search for a
scavenger hunt
near their current
position

6. 6
The Scavenger Hunt App
Basic Requirements

7. 7
The Scavenger Hunt App
Basic Requirements
• Mark target points

8. 8
The Scavenger Hunt App
Basic Requirements
• Mark target points
• Identify our users

9. 9
The Scavenger Hunt App
Basic Requirements
• Mark target points
• Identify our users
• Mark users’ progress
during hunts

10. Schema Design

11. 11
The Scavenger Hunt App
Users create
scavenger hunts by
“pinning” waypoints

12. 12
Waypoint
{ _id: ObjectId(),
user: UUID,
tour: UUDI
name: "Doug's Dogs",
desc: "The best hot-dog",
clues: [
"Hungry for a Coney Island?",
"Ask for Dr. Frankenfurter",
"Look for the hot dog stand"
],
"geometry": {
"type": "Point",
"coordinates": [125.6, 10.1] }
};

13. 13
Waypoint
{ _id: ObjectId(),
user: UUID,
tour: UUDI
name: "Doug's Dogs",
desc: "The best hot-dog",
clues: [
"Hungry for a Coney Island?",
"Ask for Dr. Frankenfurter",
"Look for the hot dog stand"
],
"geometry": {
"type": "Point",
"coordinates": [125.6, 10.1] }
};

14. 14
Waypoint
{ _id: ObjectId(),
user: UUID,
tour: UUDI
name: "Doug's Dogs",
desc: "The best hot-dog",
clues: [
"Hungry for a Coney Island?",
"Ask for Dr. Frankenfurter",
"Look for the hot dog stand"
],
"geometry": {
"type": "Point",
"coordinates": [125.6, 10.1] }
};
Geospacial Index:
ensureIndex(
{ geometry: “2dsphere” }
)

15. 15
Waypoint
Mobile Client

16. 16
Waypoint
Application ServerMobile Client
RESTful Service
scavenger.com/waypoints?lat=40.87&lon=-73.87&max=100
HTTP GET

17. 17
Waypoint
Application Server DatabaseMobile Client
RESTful Service
scavenger.com/waypoints?lat=40.87&lon=-73.87&max=100
HTTP GET

18. 18
Waypoint
Application Server DatabaseMobile Client
RESTful Service
scavenger.com/waypoints?lat=40.87&lon=-73.87&max=100
HTTP GET
Query
{ ‘$geoNear’: {
‘$geometry’: {
type: "Point",
coordinates: [ -174.9559, 40.6544 ]
},
‘maxDistance’: 100
}

19. 19
The Scavenger Hunt App
Basic Requirements
• Mark target points
• Identify our users
• Mark users’ progress
during hunts

20. 20
The Scavenger Hunt App
{
_id: ObjectId(),
user: UUID,
huntId: UUID,
timestamp: ISODate(),
geometry: {
type: "Point",
coordinates: [ long, lat ]
}
}

21. 21
The Scavenger Hunt App
Social Requirements
• Allow users to follow one
another
• Allow users to exchange
messages
• Send users notifications

22. 25
Social Interactions
Eratosthenes
Democritus
Hypatia
Shemp
Euripides

23. 26
User Collection
{
_id: UUID,
name: “Bryan Reinero’,
email: “bryan@mongodb.com”,
pass: “dontyouwishyouknew”,
description: “I am just a guy”,
followers: [
“Achille”,
“Jason”,
“Stephan”,
“Norm”
]
}

24. 27
Social Interactions
Followers Collection
{ follower: ‘Shemp’, followed: ‘Euripides’},
{ follower:‘Shemp’, followed: ’Eratosthenes”},
{ follower: “Eratosthenes’, followed: ‘Shemp’ },
…
Eratosthenes
Democritus
Hypatia
Shemp
Euripides

25. 28
Social Interactions
Eratosthenes
Democritus
Hypatia
Shemp
Euripides
Followers Collection
{ follower: ‘Shemp’, followed: ‘Euripides’},
{ follower:‘Shemp’, followed: ’Eratosthenes”},
{ follower: “Eratosthenes’, followed: ‘Shemp’ },
…
! (Euripides -> Shemp )

26. 29
Social Interactions
db.followers.find( { follower:‘Shemp’ } );
Followers Collection
{ follower: ‘Shemp’, followed: ‘Euripides’},
{ follower:‘Shemp’, followed: ’Eratosthenes”},
{ follower: “Eratosthenes’, followed: ‘Shemp’ },
…

27. 30
Notifications / Messaging
Message
{
sender: “Hypatia”,
recipients: [
“Democritus”,
“Euripides”,
“Eratosthenes”
]
date: ISODate(),
message: “truth is a point of view, and
so is changeable”
}

28. 31
Notifications / Messaging
Message
{
sender: “Hypatia”,
recipients: [
“Democritus”,
“Euripides”,
“Eratosthenes”
]
date: ISODate(),
message: “truth is a point of view, and
so is changeable”
}
db.messages.find(
{ recipients: “Democritus”}
);

29. 32
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
Notifications / Messaging
db.messages.find( ‘outbox.user’: ‘Hypatia’ );
db.messages.find( ‘outbox.user’: ‘Euripides’);
db.messages.find( ‘outbox.user’: ‘Democritus’);
db.messages.find( ‘outbox.user’: ‘Shemp’);

30. 33
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
Notifications / Messaging
db.messages.find( ‘outbox.user’: ‘Hypatia’ );
db.messages.find( ‘outbox.user’: ‘Euripides’);
db.messages.find( ‘outbox.user’: ‘Democritus’);
db.messages.find( ‘outbox.user’: ‘Shemp’);

31. 34
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
Notifications / Messaging
db.messages.find( ‘outbox.user’: ‘Hypatia’ );
db.messages.find( ‘outbox.user’: ‘Euripides’);
db.messages.find( ‘outbox.user’: ‘Democritus’);
db.messages.find( ‘outbox.user’: ‘Shemp’);

32. 35
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
OUTBOX
Notifications / Messaging
db.messages.find( ‘outbox.user’: ‘Hypatia’ );
db.messages.find( ‘outbox.user’: ‘Euripides’);
db.messages.find( ‘outbox.user’: ‘Democritus’);
db.messages.find( ‘outbox.user’: ‘Shemp’);

33. 36
One Document per Message per Recipient
{
sender: “Hypatia”,
recipient: “Democritus”,
date: ISODate(),
message: “truth is a point of view, and so is changeable”
}
{
sender: “Hypatia”,
recipient: “Euripides”,
date: ISODate(),
message: “truth is a point of view, and so is changeable”
}
{
sender: “Hypatia”,
recipient: “Eratosthenes”,
date: ISODate(),
message: “truth is a point of view, and so is changeable”
}

34. 37
Inbox Buckets
Hypatia
Message
Message
Message
Hypatia
Message
Message
Message
Hypatia
Message
Message
Message
…

35. 38
Notifications / Messaging
{user: “Hypatia”,
ctime: ISODate(),
mtime: ISODate(),
count: 10,
inbox: [
<message>,
<message>,
<message>,
…
]
}

36. 39
Write to Bucket
function addToBucket( user, item,bucketSize )
{
var query = {
user: user,
count: { $lt: bucketSize }
};
db.notifications.update(
query,
{
$push: { messages: item },
$set: { mtime: ISODate() },
$inc: { count: 1 },
$setOnInsert: {
startDate: ISODate() }
},
{ upsert: true }
);
}

37. 40
Write to Bucket
Parameters
• user bucket owner
• item notification message
• bucketSize limits number of messages / document
function addToBucket( user, item, bucketSize )
{
var query = {
user: user,
count: { $lt: bucketSize }
};
db.notifications.update(
query,
{
$push: { messages: item },
$set: { mtime: ISODate() },
$inc: { count: 1 },
$setOnInsert: {
startDate: ISODate() }
},
{ upsert: true }
);
}

38. 41
Write to Bucket
Parameters
• user bucket owner
• item notification message
• bucketSize limits number of messages / document
Query
Don’t update full bucket
function addToBucket( user, item,bucketSize )
{
var query = {
user: user,
count: { $lt: bucketSize }
};
db.notifications.update(
query,
{
$push: { messages: item },
$set: { mtime: ISODate() },
$inc: { count: 1 },
$setOnInsert: {
startDate: ISODate() }
},
{ upsert: true }
);
}

39. 42
Write to Bucket
Parameters
• user bucket owner
• item notification message
• bucketSize limits number of messages / document
Query
Don’t update full bucket
Update
• Append item to message array,
• Update the mtime with current timestamp
• Increment the size of the bucket
function addToBucket( user, item,bucketSize )
{
var query = {
user: user,
count: { $lt: bucketSize }
};
db.notifications.update(
query,
{
$push: { messages: item },
$set: { mtime: ISODate() },
$inc: { count: 1 },
$setOnInsert: {
startDate: ISODate() }
},
{ upsert: true }
);
}

40. 43
Write to Bucket
Parameters
• user bucket owner
• item notification message
• bucketSize limits number of messages / document
Query
Don’t update full bucket
Update
• Append item to message array,
• Update the mtime with current timestamp
• Increment the size of the bucket
Upsert
Create a new bucket
function addToBucket( user, item,bucketSize )
{
var query = {
user: user,
count: { $lt: bucketSize }
};
db.notifications.update(
query,
{
$push: { messages: item },
$set: { mtime: ISODate() },
$inc: { count: 1 },
$setOnInsert: {
startDate: ISODate() }
},
{ upsert: true }
);
}

41. 44
Personal Timeline / Hotlist
{
_id: UUID,
user: ”Democritus",
hotList" : [
{
message: "New scavenger hunt tomorrow!",
url: "http://bit.ly/1hKn9ff",
date" : ISODate()
},
{
message: "Get 50% off at Toga City",
url: "http://bit.ly/1KnlFHQ",
date: ISODate()
}
],
atime: ISODate("20150313T04:38:43.606Z")
}

42. 45
Personal Timeline / Hotlist
Notifications of highest
user relevance
{
_id: UUID,
user: ”Democritus",
hotList" : [
{
message: "New scavenger hunt tomorrow!",
url: "http://bit.ly/1hKn9ff",
date" : ISODate()
},
{
message: "Get 50% off at Toga City",
url: "http://bit.ly/1KnlFHQ",
date: ISODate()
}
],
atime: ISODate("20150313T04:38:43.606Z")
}

43. 50
Production Ready Architecture
Application Servers Replica SetMobile Client
L.B.

44. Thanks!
{ name: ‘Bryan Reinero’,
title: ‘Developer Advocate’,
twitter: ‘@blimpyacht’,
code: ‘github.com/breinero’
email: ‘bryan@mongdb.com’ }

45. What’s My Place in the Universe?
Using Geo Indexes to Solve Existential Crises
Building a Mobile App with MongoDB Part 2

46. 56
The Scavenger Hunt App
Users create
scavenger hunts by
“pinning” waypoints

47. 57
The Scavenger Hunt App
Players search for a
scavenger hunt
near their current
position

48. 58
The Scavenger Hunt App
Basic Requirements

49. 59
The Scavenger Hunt App
Basic Requirements
• Mark target points

50. 60
The Scavenger Hunt App
Basic Requirements
• Mark target points
• Identify our users

51. 61
The Scavenger Hunt App
Basic Requirements
• Mark target points
• Identify our users
• Mark users’ progress
during hunts

52. 62
Waypoint
{ _id: ObjectId(),
user: UUID,
tour: UUDI
name: "Doug's Dogs",
desc: "The best hot-dog",
clues: [
"Hungry for a Coney Island?",
"Ask for Dr. Frankenfurter",
"Look for the hot dog stand"
],
"geometry": {
"type": "Point",
"coordinates": [125.6, 10.1] }
};

53. 63
Waypoint
{ _id: ObjectId(),
user: UUID,
tour: UUDI
name: "Doug's Dogs",
desc: "The best hot-dog",
clues: [
"Hungry for a Coney Island?",
"Ask for Dr. Frankenfurter",
"Look for the hot dog stand"
],
"geometry": {
"type": "Point",
"coordinates": [125.6, 10.1] }
};
Geospacial Index:
ensureIndex(
{ geometry: “2dsphere” }
)

54. 64
Waypoint
db.waypoints.find(
{
geometry: {
$near:{
{ type : "Point" ,
coordinates :
[ -173, 40.7 ]
},
$maxDistance : 100
}
}});

55. 65
GEOJSON
• Point
• MultiPoint
• LineString
• MultiLineString
• Polygon
• MultiPolygon
• GeometryCollection
http://geojson.org/

56. Crisis 1:
The sense of being alone
and isolated in the world

57. 67
The Checkpoint Document
{
_id: ObjectId(),
user: UUID,
huntId: UUID,
timestamp: ISODate(),
geometry: {
type: "Point",
coordinates: [125.6, 10.1]
}
}

58. 68
The Checkpoint Document
{
_id: ObjectId(),
user: UUID,
huntId: UUID,
timestamp: ISODate(),
geometry: {
type: "Point",
coordinates: [125.6, 10.1]
}
}
db.checkpoints.ensureIndex(
timestamp: -1,
geometry: “2dsphere”
);

59. 69
The Checkpoint Document
db.checkpoints.find(
{
date: {$gt: <10mins ago> }
geometry: {
$near:{
{ type : "Point" ,
coordinates :
[ -173, 40.7 ]
},
$maxDistance : 100 }
}});
{
_id: ObjectId(),
user: UUID,
huntId: UUID,
timestamp: ISODate(),
geometry: {
type: "Point",
coordinates: [125.6, 10.1]
}
}

60. 70
The Checkpoint Document
db.checkpoints.find(
{
date: {$gt: <10mins ago> }
geometry: {
$near:{
{ type : "Point" ,
coordinates :
[ -173, 40.7 ]
},
$maxDistance : 100 }
}});
{
_id: ObjectId(),
user: UUID,
huntId: UUID,
timestamp: ISODate(),
geometry: {
type: "Point",
coordinates: [125.6, 10.1]
}
}

61. 71
The Scavenger Hunt App

62. 72
Polygon

63. 73
Polygon
Defines a business service area

64. 74
Polygon
{
"type": "Polygon",
"coordinates" : [
[
[ -73.969581, 40.760331 ],
[ -73.974487, 40.762245 ],
[ -73.977692, 40.763598],
[ -73.979508, 40.761269 ],
[ -73.982364, 40.762358 ],
[ -73.983692, 40.760497 ],
[ -73.972821, 40.755861 ],
[ -73.969581, 40.760331 ]
]
]
} Defines a business service area

65. 75
Polygon
Defines a business service area
{
"type": "Polygon",
"coordinates" : [
[
[ -73.969581, 40.760331 ],
[ -73.974487, 40.762245 ],
[ -73.977692, 40.763598],
[ -73.979508, 40.761269 ],
[ -73.982364, 40.762358 ],
[ -73.983692, 40.760497 ],
[ -73.972821, 40.755861 ],
[ -73.969581, 40.760331 ]
]
]
}

66. 76
Polygon

67. 77
$geowithin
{ geometry: {
$geoWithin: {
$geometry : {
'type': "Polygon",
'coordinates': [
[
[ -73.975181, 40.758494 ],
[ -73.973336, 40.760965 ],
[ -73.974924, 40.761663 ],
[ -73.976748, 40.759160 ],
[ -73.975181, 40.758494 ]
]
]
}}}}

68. 78
$geoIntersects
{
$geoIntersects: {
$geometry: {
"type": "Point",
"coordinates": [
-73.975010,
40.760071
]
}
}
}

69. 79
Complex Polygons

70. 80
Complex Polygons
{
"type": "Polygon",
"coordinates" : [
[ [ -73.969581, 40.760331 ],
[ -73.974487, 40.762245 ],
[ -73.977692, 40.763598],
[ -73.979508, 40.761269 ],
[ -73.982364, 40.762358 ],
[ -73.983692, 40.760497 ],
[ -73.972821, 40.755861 ],
[ -73.969581, 40.760331 ]
],
[[ -73.975181, 40.758494 ],
[ -73.973336, 40.760965 ],
[ -73.974924, 40.761663 ],
[ -73.976748, 40.759160 ],
[ -73.975181, 40.758494 ]]
]
}

71. 81
Complex Polygons

72. 82
Complex Polygons
{
$geoIntersects: {
$geometry: {
"type": "Point",
"coordinates": [
-73.975010,
40.760071
]
}
}
}

73. 83
Rendering Geo Elements
{ type: "GeometryCollection",
geometries: [
{ type: "Point",
coordinates: [
-73.975010,
40.760071
]
},
{ type: "Polygon",
coordinates: [[
[-73.975181, 40.758494 ],
[-73.973336, 40.760965 ],
[-73.974924, 40.761663 ],
[-73.976748, 40.75916 ],
[-73.975181, 40.758494 ]
]]
}
]
}

74. 84
Complex Polygons

75. 85
Complex Polygons
{ "type": "Polygon",
"coordinates" : [
[
[ -73.969581, 40.760331 ],
[ -73.974487, 40.762245 ],
[ -73.977692, 40.763598],
…
],
[
[ -73.975181, 40.758494 ],
[ -73.973336, 40.760965 ],
[ -73.974924, 40.761663 ],
..
],
[
[ -73.979437, 40.755390 ],
[ -73.976953, 40.754362 ],
[ -73.978364, 40.752448 ],
…] ]
}

76. 86
Complex Polygons
{ "type": "Polygon",
"coordinates" : [
[
[ -73.969581, 40.760331 ],
[ -73.974487, 40.762245 ],
[ -73.977692, 40.763598],
…
],
[
[ -73.975181, 40.758494 ],
[ -73.973336, 40.760965 ],
[ -73.974924, 40.761663 ],
..
],
[
[ -73.979437, 40.755390 ],
[ -73.976953, 40.754362 ],
[ -73.978364, 40.752448 ],
…] ]
}
$err : Can't canonicalize query
BadValue Secondary loops not
contained by first exterior loop -
secondary loops must be holes

77. 87
Complex Polygons
{ "type": "Polygon",
"coordinates" : [
[
[ -73.969581, 40.760331 ],
[ -73.974487, 40.762245 ],
[ -73.977692, 40.763598],
…
],
[
[ -73.975181, 40.758494 ],
[ -73.973336, 40.760965 ],
[ -73.974924, 40.761663 ],
..
],
[
[ -73.979437, 40.755390 ],
[ -73.976953, 40.754362 ],
[ -73.978364, 40.752448 ],
…] ]
}
$err : Can't canonicalize query
BadValue Secondary loops not
contained by first exterior loop -
secondary loops must be holes

78. 88
{ "type": "Polygon",
"coordinates" : [
[
[ -73.969581, 40.760331 ],
[ -73.974487, 40.762245 ],
[ -73.977692, 40.763598],
…
],
]
}
{ "type": "Polygon",
"coordinates" : [
[
[
[ -73.979437, 40.755390 ],
[ -73.976953, 40.754362 ],
[ -73.978364, 40.752448 ],
…] ]
}
Complex Polygons

79. 89
Complex Polygons
‘$or’ : [ {
"geometry" : {
"$geoWithin" : {
"$geometry" : {
"type" : "Polygon",
"coordinates" : [
...
]
}…,
{
"geometry" : {
"$geoWithin" : {
"$geometry" : {
"type" : "Polygon",
"coordinates" : [
...
]
}…

80. 90
Complex Polygons

81. 91
Complex Polygons
{
$geoIntersects: {
$geometry: { "type": "LineString",
"coordinates": [
[-73.979543, 40.761132],
[-73.974715, 40.759127],
[-73.973363, 40.760969],
[-73.970059, 40.759600]
]
}
}

82. Crisis 2:
Shattering of one's sense of reality,
or how the world is.

83. 93
Honolulu to Tokyo

84. 94
Honolulu to Tokyo

85. 95
Am I inside or outside the box?

86. 96
The Correct Interpretation

87. 97
Coordinate Reference System
• A system to locate geographical entities
• WGS84 datum
• Geoid - the shape of the surface of the oceans
• EGM96 gravitational model
"crs": {
"type": "name",
"properties": {
"name": "urn:ogc:def:crs:OGC:1.3:CRS84"
}
}

88. 98
Geoid
Credit: ESA

89. 99
Right Hand Rule

90. Crisis 3:
Anew-found grasp or
appreciation of one's mortality.

91. 101
BigPolygon in Action

92. 102
BigPolygon in Action

93. 103
BigPolygon in Action

94. Crisis 4:
Awareness of one's freedom
and the
consequences of accepting freedom.

95. Questions?

96. Thanks!
{ name: ‘Bryan Reinero’,
title: ‘Developer Advocate’,
twitter: ‘@blimpyacht’,
code: ‘github.com/breinero’
email: ‘bryan@mongdb.com’ }

97. Deploy Like a Boss
Building a Mobile App with MongoDB Part 3

98. 108
Deploy
with
Joy!

99. 109

100. 110

101. 111
Production Checklist
Proper Infrastructure
Proper Configuration
Proper Monitoring
Emergency Procedures

102. 112
Infrastructure Sizing
• RAM
• CPU
• Disk Size
• I/O Bandwidth
• Availability

103. 113
Sizing
• Indexes need to be in RAM
• Working set needs to be in RAM
• I/O Bandwidth
- write load
- Index updates
- Working set migration
{ _id: ObjectId(),
tour: UUID,
user: UUID,
name: "Doug's Dogs",
desc: "The best hot-dog",
clues: [
"Hungry for a Coney Island?",
"Ask for Dr. Frankenfurter",
"Look for the hot dog stand"
]
"geometry": {
"type": "Point",
"coordinates": [125.6, 10.1]
}
}

104. 117
Load Testing

105. 118
Load Testing
• Test it like you use it, benchmarks don’t count

106. 119
Load Testing
• Test it like you use it, benchmarks don’t count
• Test to failure

107. 120
Load Testing
• Test it like you use it, benchmarks don’t count
• Test to failure
• Instrument your code!

108. 121
Load Testing
• Test it like you use it, benchmarks don’t count
• Test to failure
• Instrument your code!
https://github.com/breinero/Firehose
https://github.com/ParsePlatform/flashback

109. 122
Load Testing
• Test it like you use it, benchmarks don’t count
• Test to failure
• Instrument your code!
There’s me

110. 123
Growth
0
2
4
6
8
10
12
1 2 3 4 5 6 7 8 9 10
Saturation
Warn
Load
1K Ops / Second
time

111. 124
Growth
0
2
4
6
8
10
12
1 2 3 4 5 6 7 8 9 10
Saturation
Warn
Load
Memory

112. 125
Growth
0
2
4
6
8
10
12
1 2 3 4 5 6 7 8 9 10
Saturation
Warn
Load
Input Output

113. 126

114. 127
Monitoring
Baseline
• MongoDB Management Service (MMS)
• MongoDB Ops Manager
• Nagios, Zenoss, …
Detailed Query Specific
• mongotop
• db.currentOp()
• Query Profiler
• mtools

115. 128
Fosrensics
2014-08-08T21:15:25.181-0500 [conn1026]
getmore claimsPoc.claims
cursorid:100012502307 ntoreturn:0
keyUpdates:0 numYields:1406953
locks(micros) r:11887558422 nreturned:289
reslen:4208149 28795759ms
2014-08-07T15:31:51.714-0500 [conn7]
command claimsPoc.$cmd command:
createIndexes { createIndexes: "claims",
indexes: [ { key: { Claims.ICN: 1.0 }, name:
"Claims.ICN_1" } ] } keyUpdates:0
numYields:0 locks(micros) r:14476
w:25176930351 reslen:113 25176955ms

116. 129
Logging

117. 130
Logging
• Save and Rotate
• Don’t use --quiet
• --logpath != --dbpath
• Use component verbosity
for debugging

118. 131
Security

119. 132
Security
• Firewall
• Bind ip
• Encrypt Networks
• Enable Access Control
• Don’t enable REST interface
• Auditing
Limit Exposure
and use
Principal of Least Privileges

120. 133
Tuning
Best Practices
• Disable Transparent hugepages
• NTP to synchronize time
• Set ulimits
• Use XFS or Ext4
• Don’t use NFS
• Disable NUMA
• Have swap
Read Production Notes
Tunables
• Set IO Scheduler NOOP
• Adjust readaheads ( MMapV1 )
• Avoid cgroups
• SE Linux (?)
• RAID

121. 134
Availability
http://avstop.com/ac/flighttrainghandbook/imagel4b.jpg

122. 135
Availability
S S
DC1 DC2
P
Avoid Critical Data Centers

123. 136
Availability
P S
DC1 DC2
S
DC3

124. 137
Availability
P S
DC1 DC2
S
AWS

125. 138
Availability
P S
DC1 DC2
Arbiter
AWS

126. 139
Availability
P
DC1
Arbiter
AWS
S
DC2
Down for maintenance

127. 140
Emergency Procedures
https://spinoff.nasa.gov/spinoff2002/images/070.jpg

128. 141
Emergency Procedures
https://spinoff.nasa.gov/spinoff2002/images/070.jpg
Backup and Recovery
• File System Snapshot
• MMS Cloud
• Ops Manager
• Mongodump

129. 142
Backups and Recovery
https://spinoff.nasa.gov/spinoff2002/images/070.jpg
PERFORM DRILLS
OFTEN AND
ROUTINELY

130. 143
Emergency Procedures
https://spinoff.nasa.gov/spinoff2002/images/070.jpg
Document your Procedures
• Include ETAs
• Follow procedures in
docs.mongodb.org

131. 144
Production Ready Architecture
L.B.

132. 145
Production Ready Architecture
L.B.
Unindexed queries

133. 146
Production Ready Architecture
L.B.
Unindexed queries Leads to collection scans

134. 147
Production Ready Architecture
L.B.
Unindexed queries Leads to collection scans
Results in high latencies

135. 148
Classic Failure Scenario
L.B.
Unindexed queries Leads to collection scans
Results in high latenciesCauses memory exhaustion

136. 149
Production Ready Architecture
L.B.
Unindexed queries Leads to collection scans
Results in high latenciesCauses memory exhaustion
CASCADING FAILURE

137. 150
Circuit Breaker
Trigger Conditions
• Latency stats.getMean() >= max
• OpsPerSecond stats.getN() >= max
• ConcurrentOperations stats.getN()*stats.getMean() >= max

138. 151
Circuit Breaker
Trigger Conditions
• Latency stats.getMean() >= max
• OpsPerSecond stats.getN() >= max
• ConcurrentOperations stats.getN()*stats.getMean() >= max
https://github.com/breinero/Firehose

139. 152
Production Ready Architecture
L.B.

140. 156
Client Side
• Don’t use ensureIndex() in application
• Look out for connection bombs
--maxConnect
• DO use operation timeouts
• DON’T cause socket timeouts
Lower keepalives
• Avoid retry bombs

141. 157
Requirements & Specs
Make a DevOps Contract
• Database Access Requirements
• Database Access Fulfillment Specification
• Cluster Configuration
• Monitoring and Alerting Specification

142. 158
Monitoring
• Opcounters
• Memory
• Page Faults
• Queues
• Replication Lag
• Oplog Window
• Background Flush Average
• Disk space

143. Thanks!
{ name: ‘Bryan Reinero’,
title: ‘Developer Advocate’,
twitter: ‘@blimpyacht’,
code: ‘github.com/breinero’
email: ‘bryan@mongdb.com’ }