How to leverage what's new in MongoDB 3.6

1 2 APR IL , 2018
# M D B l o c a l
How to leverage
what’s new in MongoDB
3.6

# M D B l o c a l
Maxime Beugnet
Developer Advocate EMEA @MongoDB Paris
Twitter : @MBeugnet
Github : MaBeuLux88
Email : maxime@mongodb.com
Meetups
Hackathons
Workshops
Conferences

# M D B l o c a l
MONGODB 3.6
THEMES

# M D B l o c a l
To make your life easier…

# M D B l o c a l# M D B l o c a l
Realtime
Expressibili
ty Analytics Application
Availability
Operations

# M D B l o c a l
REALTIME
CHANGE STREAMS

updatereplaceinsertdeleteMongoDB Application Action

db.coll.watch()
db.coll.watch([{$match: {operationType: “insert”}}])
db.coll.watch([], {fullDocument:“updateLookup”})
db.coll.watch([], {resumeAfter:<cachedResumeToken>})
Change streams
• $match
• $project
• $addFields
• $replaceRoot
• $redact

cursor = db.users.watch([ ], {"fullDocument":"updateLookup"});
while ( !cursor.isExhausted() ) {
if ( cursor.hasNext() ) {
print(tojson(cursor.next()));
}
}
Change streams => cursors

•Resumable
•Targeted
Changes
•Total ordering
•Durability
•Security
•Ease of use
•Idempotence
Characteristics of Change Streams

# M D B l o c a l
EXPRESSIBILITY
More expressive query language
Array updates

db.products.find( {
$expr: {
$gt: [ "$currRating" , "$prevRating" ]
}
} )
Comparing Fields within a document

db.products.find( {
$expr: {
$gt: [ {$subtract:
["$currRating" ,"$prevRating"] }, 2]
}
} )

db.supplies.find( {
$expr: {
$lt:[ {
$cond: {
if: { $gte: ["$qty", 100] },
then: { $divide: ["$price", 2] },
else: { $divide: ["$price", 4] }
}
}, 5 ]
}
})

Expressive Array updates
Update all matching
items in an array
Match Nested Arrays

MongoDB V3.4
• $
• $addToSet
• $pop
• $pull
• $push
• $pullAll
{ _id: 1,
grades: [
{ exam: 80, quizz: 75},
{ exam: 85, quizz: 90}
]
}
db.students.update(
{ _id: 1, "grades.exam": 85 },
{ $set: { "grades.$.quizz" : 95 } }
)

MongoDB V3.6
• $[<id>]
• $[]
{ _id: 1,
grades: [
{ exam: 80, quizz: 75},
{ exam: 85, quizz: 90}
]
}
db.students.update( { },
{ $set: { "grades.$[elem].quizz" : 100 } },
{ multi: true, arrayFilters: [ { "elem.exam": { $gte: 80 } } ] }
)
db.students.update( { },
{ $set: { "grades.$[].quizz" : 100 } },
{ multi: true }
)

{ _id: 1, name: "X",
Medications: [
{ id: 23, name: "DrugName99", Sched:"I", Rx: [
{ id:13, Qty: 60, started: "2009-01-01" },
{ id:77, Qty: 30, started: "2011-02-01", current:true }
]},
{ id: 41, name: "OtherDrugName", Sched: "II, Rx: […] },
{ id: 59, name: "ThirdDrug", Sched:"I", Rx:[
{ id:994, Qty: 60, started: "2012-01-01", current:true },
{ id:1034, Qty: 90, started: "2007-02-01" }
]},
"lastVisit": ISODate("2017-01-22T13:01:13.000Z")
}

{ _id: 1, name: "X",
Medications: [
{ id: 23, Sched:"I", Rx: [
{ id:13, Qty: 60},
{ id:77, Qty: 30, current:true}
]},
{ id: 41, Sched: "II" },
{ id: 59, Sched:"I", Rx:[
{ id:994, Qty: 60, current:true},
{ id:1034, Qty: 90}
]}
]
}
db.patientRx.update(
{},
{"$set": {"Medications.$[med].Rx.$[rx].Qty": 20}},
{ "multi": true,
"arrayFilters": [
{"med.Sched": "I"},
{"rx.current":true, "rx.Qty":{$gt:30}}
]}
)

{ _id: 1, name: "X",
Medications: [
{ id: 23, Sched:"I", Rx: [
{ id:13, Qty: 60},
{ id:77, Qty: 30, current:true}
]},
{ id: 41, Sched: "II" },
{ id: 59, Sched:"I", Rx:[
{ id:994, Qty: 60, current:true},
{ id:1034, Qty: 90}
]}
}
db.patientRx.update(
{"Medications": {"$elemMatch": {
"Sched": "II",
"Rx": {"$elemMatch": {
"current": true,
"Qty": {"$gt": 30}
}}
}}},
{"$set": {"Medications.$[med].Rx.$[rx].Qty": 20}},
{ "multi": true,
"arrayFilters": [
{"med.Sched": "I"},
{"rx.current":true, "rx.Qty":{$gt:30}}
]}
)

# M D B l o c a l
ANALYTICS
New operators
Timezone support
Expressive $lookup
R Driver
BI Connector

New operators

New operators : $arrayToObject
{ "_id" : 1, dimensions: [ { "k": "l", "v": 25} , { "k": "w", "v": 10 }, { "k": "uom", "v": "cm" } ] }
{ "_id" : 2, dimensions: [ [ "l", 50 ], [ "w", 25 ], [ "uom", "cm" ] ] }
{ "_id" : 3, dimensions: [ [ "l", 50 ], [ "l", 25 ], [ "l", "cm" ] ] }
db.inventory.aggregate( [ { $project: { dimensions: { $arrayToObject: "$dimensions" } } } ] )
{ "_id" : 1, "dimensions" : { "l" : 25, "w" : 10, "uom" : "cm" } }
{ "_id" : 2, "dimensions" : { "l" : 50, "w" : 25, "uom" : "cm" } }
{ "_id" : 3, "dimensions" : { "l" : 50 } }

New operators : $objectToArray
{ $objectToArray: { item: "foo",
qty: 25,
size: { len: 25, w: 10, uom: "cm" }
} }
[ { "k" : "item", "v" : "foo" },
{ "k" : "qty", "v" : 25 },
{ "k" : "size", "v" : { "len" : 25, "w" : 10, "uom" : "cm" } } ]

New operators : $mergeObjects
{ $mergeObjects: [ { a: 1 },
{ a: 2, b: 2 },
{ a: 3, b: null, c: 3 } ] }
{ a: 3, b: null, c: 3 }

New operators : $dateFromString
{ _id: 1, date: "2017-02-08T12:10:40.787, timezone: "America/New_York" },
{ _id: 2, date: "2017-02-08" , timezone: "-05:00" },
{ _id: 3 }
db.logmessages.aggregate( [
{ $project: {
date: {
$dateFromString: { dateString: '$date', timezone: '$timezone’ }
}
}
} ] )
{ "_id" : 1, "date" : ISODate("2017-02-08T17:10:40.787Z") }
{ "_id" : 2, "date" : ISODate("2017-02-08T05:00:00Z") }
{ "_id" : 3, "date" : null }

New operators : $dateFromParts
db.sales.aggregate([ {
$project: {
date: { $dateFromParts: { 'year' : 2017, 'month' : 2, 'day': 8, 'hour' : 12 } },
date_iso: { $dateFromParts: { 'isoWeekYear' : 2017, 'isoWeek' : 6, 'isoDayOfWeek' : 3, 'hour' : 12 } },
date_timezone: { $dateFromParts: { 'year' : 2016, 'month' : 12, 'day' : 31, 'hour' : 23, 'minute' : 46,
'second' : 12, 'timezone' : 'America/New_York’ } }
}
}])
{
"_id" : 1,
"date" : ISODate("2017-02-08T12:00:00Z"),
"date_iso" : ISODate("2017-02-08T12:00:00Z"),
"date_timezone" : ISODate("2017-01-01T04:46:12Z")
}

New operators : $dateToParts
db.sales.aggregate([ {
$project: {
date: { $dateToParts: { date: "$date" } },
date_iso: { $dateToParts: { date: "$date", iso8601: true } },
date_timezone: { $dateToParts: { date: "$date", timezone: "America/New_York" } }
}
}])
{
"_id" : 2,
"date" : { "year" : 2017, "month" : 1, "day" : 1, "hour" : 1,
"minute" : 29, "second" : 9, "millisecond" : 123 },
"date_iso" : { "isoWeekYear" : 2016, "isoWeek" : 52, "isoDayOfWeek" : 7, "hour" : 1,
"minute" : 29, "second" : 9, "millisecond" : 123 },
"date_timezone" : { "year" : 2016, "month" : 12, "day" : 31, "hour" : 20,
"minute" : 29, "second" : 9, "millisecond" : 123 }
}

Timezone support
$dayOfYear, $dayOfMonth, $dayOfWeek, $year, $month, $week, $hour,
$minute, $second, $millisecond, $isoDayOfWeek, $isoWeek, $isoWeekYear
{ $operator: { date: <isoDateExpression>, timezone: <tzExpression> } }

>db.collection.aggregate({$lookup:{
from:"coll2",
localField:"x",
foreignField:"y",
as:"coll2details"
}}])
More expressive $lookup
>db.collection.aggregate({$lookup:{
from:"coll2",
let: {x: "$x"},
pipeline: [
{$match: {$expr:
{$eq: [ "$y", "$$x"] }
} }
]
as:"coll2details"
}}])
Prior to 3.6 New in 3.6

$lookup example
orders:
{
line_items : [
{ id: 123,
title : “USB Battery”,
price: 15.0 },
{ id: 512,
title : “Hip T-shirt”,
price : 45.0 }
]
}
db.orders.aggregate([
{$unwind: "$line_items"},
{$lookup:{
from: "reviews",
let: {p_id: "$line_items.id"},
pipeline: [
{$match: {$expr: {$eq: ["$p_id","$$p_id"]}}},
{$group: { _id: 1, rating: {$avg:"$rating"}}}
], as: "avgRating" }
}
])

Recommended MongoDB R driver for data scientists, developers & statisticians
• MongoDB read & write concerns to control data consistency & durability
• Idiomatic, native language access to the database
• Data security with enterprise authentication mechanisms
• BSON data type support, e.g., Decimal 128 for high precision scientific &
financial analysis
R Driver for MongoDB

MongoDB Connector for BI
+ many more

MongoDB Connector for BI
• Faster
• Takes advantage of 3.6 expressive Lookup to push more
computation into the database
• Supports Show Status function to enable deeper performance
optimization
• Simpler
• Lifecycle management with Ops Manager
• Schema sampling and mapping now managed by the mongosqld
process, rather than separate utility (mongodrdl)
• Authenticate via client-side plugins, rather than managing TLS
certificates. Kerberos support added

# M D B l o c a l
APPLICATIONS AVAILABILITY
Retryable writes
DNS seed list
Read Concern: Available & Tunable
consistency

Retryable writes
PS
S
Application P
write successful :D
write unsuccessful :O

Retryable writes
Application MongoDB
{ _id: 1,
team : "Manchester United",
gameID: "game123",
coach : "José Mourinho",
score : 2,
league: "Premier League"
}
{ _id: 2,
team : "Chelsea",
gameID: "game456",
coach : "Antonio Conte",
score : 2,
league: "Premier League"
}
db.games.update(
{gameID:"game123"},
{$inc: {score:1}}
)
db.games.update(
{team: "Chelsea"},
{$set: {coach:"Zidane"}}
)

• Automatic Drivers logic
• Network errors
• Elections
• NOT for logic errors
• Safe
• For both non-idempotent and idempotent writes
• NOT for multi: true
Characteristics of Retryable Writes

Retryable writes
uri = "mongodb://example.com:27017/?retryWrites=true"
client = MongoClient(uri)
database = client.database
collection = database.collection

DNS seed list
"mongodb://example1.com:27017,example2.com:27017,example3.com:27017
"mongodb+srv://my-dns-server.mongodb.com
"mongodb://example1.com:27017

Tunable consistency
Availability Consistencymagic

• readConcern (Read Isolation)
• Local
• Majority
• Linearizable
• writeConcern (write acknowledgement)
• <number> (i.e. 1)
• Majority
• Tag
What is readConcern and writeConcern?

Tunable consistency
mongos
3 1
2
Shard 1 Shard 2 Shard 3
readConcern: available

Tunable consistency
readConcern: available is equivalent to readConcern: local on replica sets
you can pass readConcern: available to a primary in a sharded cluster
readConcern: available is default for secondaries in a sharded cluster
Secondaries in sharded clusters will now respect readConcern : local for safe reads

Tunable consistency
mongos
3 1
2
Shard 1 Shard 2 Shard 3
Global Logical Clock
wait until cluster time has moved past the last time you saw
Causal consistency:
guarantees monotonic,
logically consistent reads
from any replica node in
the same user session

Tunable consistency
//start client session, which is causally consistent by default
try (ClientSession session =
client.startSession(ClientSessionOptions.builder().build())) {
//Run causally related operations within the session
collection.insertOne(session, ... );
collection.updateOne(session, ...);
try (MongoCursor<Document> cursor =
collection.find(session).filter(...).iterator()) {
while (cursor.hasNext()) {
Document cur = cursor.next();
}
}

# M D B l o c a l
OPERATIONS
JSON Schema
Network security
Session management
End-to-end Compression

Network Security
Bind to localhost by Default
IP Whitelisting
• Associate IP addresses or ranges with roles in auth
• If a the IP restrictions are not met, fail to authenticate
• Able to restrict __system user to authenticate from only cluster
nodes

Network Security
192.168.1.25
Application
Application
System Administrator
192.168.1.48
172.16.4.13
172.16.4.88
172.33.20.62
172.33.20.11
Restrict each user’s
authentication based on:
• Client IP Address Range
and/or
• Server IP Listen Address

JSON Schema
Enforces strict schema structure over a complete collection
for data governance & quality
• Builds on document validation introduced by restricting new content that
can be added to a document
• Enforces presence, type, and values for document content, including
nested array
• Simplifies application logic
Tunable: enforce document structure, log warnings, or allow
complete schema flexibility
Queryable: identify all existing documents that do not comply

JSON Schema
db.createCollection( "orders",
{validator: {$jsonSchema:
{properties:
{line_items:
{type: "array",
items:
{properties:
{title: {type: "string"},
price: {type: "number", minimum: 0.0} },
required: ["_id", "title", "price"],
additionalProperties: false}}},
required: ["line_items"]}}}
)
http://json-schema.org/

Session Management
Server sessions
• Every Operation is wrapped in a server session by default in 3.6
• killSessions by user
Client sessions
• Every operation within a defined client session have causal consistency
• Not by default, must be explicitly defined

MongoDB 3.6 adds compression
of wire protocol traffic between
client and database
• Up to 80% bandwidth savings
MongoDB End to End
Compression
• Wire protocol
• Intra-cluster
• Indexes in memory
• Storage
Application
MongoDB Primary
Replica
Wire Protocol
Compression
MongoDB Secondary Replica
Single ViewMongoDB Secondary Replica
MongoDB Secondary Replica
Intra-Cluster
Compression
Compression of
Data on Disk
Compression of
Indexes in Memory
End to End
Compression

How to leverage what's new in MongoDB 3.6

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to How to leverage what's new in MongoDB 3.6

Similar to How to leverage what's new in MongoDB 3.6 (20)

Recently uploaded

Recently uploaded (20)

How to leverage what's new in MongoDB 3.6