This document discusses MongoDB's new client-side field level encryption capabilities in version 4.2. It describes how client-side encryption works, including encrypting and decrypting data before it is sent to or retrieved from the database. It also covers key management and the use of JSON schemas to define which fields should be encrypted.

1. @
#MDBlocal
Benjamin Lorenz
New Encryption Capabilities in MongoDB 4.2:
A Deep Dive into Protecting Sensitive Workloads
benjaminlorenz
MUNICH

4. db.coll.insert({
_id: 1,
name: "Doris",
ssn: "457-55-5462"
})

6. doc = db.coll.find_one({
ssn: "457-55-5462"
})

8. print (doc)

9. {
_id: 1
name: "Doris",
ssn: "457-55-5462"
}

12. db.coll.insert({
name: "Doris",
ssn: "457-55-5462"
})
{
insert: "coll",
documents: [{
name: "Doris",
ssn: BinData(6, "a10x…")
}]
}
You see: MongoDB sees:
Encrypt before sending

13. {
_id: 1
name: "Doris",
ssn: BinData(6, "a10x…")
}
Driver receives: You see:
{
_id: 1
name: "Doris",
ssn: "457-55-5462"
}
Decrypt after receiving

14. How does this differ from…?
• … encryption in-transit (TLS)
• … encryption at-rest (encrypted storage engine)

15. Attacker
Query
Client
Disk
insert write
MongoDB
Auth
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })

16. Client
DiskMongoDB
Snoop
TLS
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })
insert write
Attacker

17. Client
DiskMongoDB
Attacker
insert
TLS
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })
write

18. Client
DiskMongoDB
Steal
ESE
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })
insert write
Attacker

19. Client
DiskMongoDB
Login
Client Side Encryption
db.coll.insert({ name: "Doris", ssn: "457-55-5462" })
insert write
Attacker

20. Client
DiskMongoDB
Boundaries of unencrypted data
insert write

21. Client
DiskMongoDB
… with Encrypted Storage Engine
insert write

22. Client
DiskMongoDB
… and TLS
insert write

23. Client
DiskMongoDB
with Client Side Encryption
insert write

25. Client
DiskMongoDB
ssn: BinData(6, "a10x…")
insert write

26. db.coll.update({}, {
$set: { ssn: "457-55-5462" }
})
{
update: "coll",
updates: [{
q:{},
u: {
$set: { ssn: BinData(6, "a10x…") }
}
}]
}
You see: MongoDB sees:
Update that overwrites value

27. db.coll.aggregate([{
$project: { name_ssn: {$concat: [ "$name", " - ", "$ssn" ] } }
}]
Aggregate acting on the data

28. Find with equality query
* For deterministic encryption
db.coll.find({ssn: "457-55-5462" }) {
find: "coll",
filter: { ssn: BinData(6, "a10x…") }
}
You see: MongoDB sees:

29. Find with equality query
* For deterministic encryption
db.test.find(
{
$and: [
{
$or: [
{ ssn : { $in : [ "457-55-5462", "153-96-2097" ]} },
{ ssn: { $exists: false } }
]
},
{ name: "Doris" }
]
}
)
You see:

30. Find with equality query
* For deterministic encryption
MongoDB sees:
{
find: "coll",
filter: {
$and: [
{
$or: [
{ ssn : { $in : [ BinData(6, "a10x…"), BinData(6, "8dk1…") ]} },
{ ssn: { $exists: false } }
]
},
{ name: "Doris" }
]
}
}

31. MongoDB
Attacker
Login

34. Doris
Private stuff in storage

35. PoliceDoris
Private stuff in storage

36. Vault key
Held only by you
Vault

38. Encrypted Data
MongoDB
Encryption Key

40. { _id: 1, ssn: BinData(0, "A81…"), name: "Kevin" }
{ _id: 2, ssn: BinData(0, "017…"), name: "Eric" }
{ _id: 3, ssn: BinData(0, "5E1…"), name: "Albert" }
…

42. Destroy the key
Provably delete all user data.
GDPR "right-to-be-forgotten"

44. client = MongoClient(
auto_encryption_opts=opts)

45. Not sensitive
{

46. One key for all vaults

47. One key per vault

48. {
name: "Doris"
ssn: "457-55-5462",
email: "Doris@gmail.com",
credit_card: "4690-6950-9373-8791",
comments: [ …. ],
avatar: BinData(0, "0fi8…"),
profile: { likes: {…}, dislikes: {…} }
}

51. Describes JSON
{
bsonType: "object",
properties: {
a: {
bsonType: "int"
maximum: 10
}
b: { bsonType: "string" }
},
required: ["a", "b"]
}
{
a: 5,
b: "hi"
}
{
a: 11,
b: false
}
JSON Schema

52. {
bsonType: "object",
properties: {
ssn: {
encrypt: { … }
}
},
required: ["ssn"]
}
JSON Schema "encrypt"

53. encrypt: {
keyId: <UUID[]> or <string>,
algorithm: <string>
bsonType: <string> or <string[]>
}
bsonType indicates the type of underlying data.
algorithm indicates how to encrypt (Random or Deterministic).
keyId indicates the key used to encrypt.

54. opts = AutoEncryptionOptions(
schema_map = { "db.coll": <schema> }
…)

55. Remote Schema Fallback
db.createCollection("coll", { validator: { $jsonSchema: … } } )
Misconfigured
Client
insert "457-55-5462"
error, that should be encrypted
MongoDB

56. What if
… the server lies about the schema?
Misconfigured
Client insert "457-55-5462"
Evil MongoDB
ok :)

57. schema_map
Sub-options

59. Key vault
Key vault key
Held only by you

62. Stores encrypted keys

63. opts = AutoEncryptionOptions(
schema_map = { "db.coll": <schema> },
key_vault_namespace = "db.keyvault"
…)

64. schema_map
Sub-options
key_vault_namespace

65. … attacker drops key vault collection?
What if

66. Keep at home
opts = AutoEncryptionOptions(
schema_map = { "db.coll": <schema> },
key_vault_namespace = "db.keyvault",
key_vault_client = <client>
…)

67. schema_map
Sub-options
key_vault_namespace
key_vault_client

69. (Key Management Service)

70. Protects keys Stores keys
KMS
Key vault key
Key vault
Key vault
collection

71. Decryption requires

72. opts = AutoEncryptionOptions(
schema_map = { "db.coll": <schema> },
key_vault_namespace = "db.keys",
kms_providers = <creds>
…)

73. schema_map
Sub-options
key_vault_namespace
key_vault_client
kms_providers

75. db.coll.insert({
name: "Doris",
ssn: "457-55-5462"
})
Get encrypted key
Decrypt the key with KMSDecrypt the key with KMS
Encrypt 457-55-5462
Send insert
Compare to JSON schema

78. Authenticated Encryption with Associated Data using the
Advanced Encryption Standard (256) with Cipher Block Chaining
and Hashed-based Message Authentication Code using the Secure
Hash Algorithm (512).

79. AEAD_AES_256_CBC_HMAC_SHA_512
Provides confidentiality + integrity

80. AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic
AEAD_AES_256_CBC_HMAC_SHA_512-Random

81. coll.insert({ ssn: "457-55-5462" }) { ssn: BinData(6, "a10x…") }
You see: MongoDB stores:
coll.insert({ ssn: "457-55-5462" }) { ssn: BinData(6, "f991…") }
…Random

82. coll.insert({ ssn: "457-55-5462" }) { ssn: BinData(6, "a10x…") }
You see: MongoDB stores:
coll.insert({ ssn: "457-55-5462" }) { ssn: BinData(6, "a10x…") }
…Deterministic

83. Can be queried
doc = db.coll.find({
ssn: "457-55-5642"
})
{
find: "coll",
filter: { ssn: BinData(0, "a10x…") }
}
Driver sends:
{ ssn: BinData(6, "a10x…") }
MongoDB returns:
…Deterministic

84. Only for binary comparable types.
db.coll.find({ a: { b: 1.0 } })
{ a: { b: NumberInt(1) } }
{ a: { b: 1.0 } }
{ a: { b: NumberLong(1) } }
MongoDB returns:
…Deterministic

85. { a: { b: NumberInt(1) } }
{ a: { b: 1.0 } }
{ a: { b: NumberLong(1) } }
{ a: BinData(6, "19d0…") }
{ a: BinData(6, "b515…") }
{ a: BinData(6, "801f…") }
Encrypted as:

86. db.coll.find({ a: { b: 1.0 } })
{ a: { b: 1.0 } }
MongoDB returns:
"a" encrypted

88. { ssn: BinData(6, "AWNkTYTCw89Ss1DPzV3/2pSRDNGNJ9NB" }
New binary subtype
Older drivers and older MongoDB will treat as a black box.

89. byte algorithm
byte[16] key_id
byte original_bson_type
byte* payload
Ciphertext

90. byte algorithm
byte[16] key_id
byte original_bson_type
byte* payload
key_id + algorithm describes how to decrypt.
No JSON Schema necessary!
Ciphertext

91. byte algorithm
byte[16] key_id
byte original_bson_type
byte* payload
Provides extra server-side validation.
But prohibits single-value types (MinKey, MaxKey, Undefined, Null)
Ciphertext

92. byte algorithm
byte[16] key_id
byte original_bson_type
byte* payload
Payload includes encoded IV and padding block, and HMAC.
Ciphertext adds between 66 to 82 bytes of overhead.
Ciphertext

100. encryption = ClientEncryption(…)
id = encryption.create_data_key(…)
print (id)
Script prints: "609"

101. {
_id: 23
email: "Doris@gmail.com",
pwd: "19dg8%"
following: [ 9, 20, 95 ]
}

102. …
"email": {
"encrypt": {
"keyId": 609,
"algorithm": "…Deterministic"
"bsonType": "string",
}
},
"pwd": {
"encrypt": {
"keyId": 609,
"algorithm": "…Random"
"bsonType": "string"
}
}
…

103. client = MongoClient(
auto_encryption_opts=opts)
def register(db, email, pwd):
db.users.insert_one({ "email": email, "pwd": pwd })
def login(db, email, pwd):
user = db.users.find_one({ "email": email })
if user and matches(user["pwd"], pwd):
return True
else:
return False

104. client = MongoClient()
for doc in client.db.users.find():
print doc
{
_id: 23
email: BinData(6, 810f…"),
pwd: BinData(6, "19A0…")
}

106. {
user_id: 123,
date: Date("6/8/2019"),
title: "My first entry",
body: "Dear diary, … "
}
…
"body": {
"encrypt": {
"keyId": 609,
"algorithm": "…Random",
"bsonType": "string"
}
}
…

107. def create_post(db, user_id, title, body):
db.posts.insert_one({
"user_id": user_id,
"date": datetime.now(),
"title": title,
"body": body
})

109. def delete_user_data(db, user_id):
db.posts.delete_many({ "user_id": user_id })

110. delete key

112. encryption = ClientEncryption(…)
def register(encryption, db, email, pwd):
user_id = db.users.insert_one({
"email": email,
"pwd": pwd
}).inserted_id
opts = DataKeyOpts(keyAltNames=[user_id])
encryption.create_data_key("aws", opts)

113. …
"body": {
"encrypt": {
"keyId": 609,
"algorithm": "…Random",
"bsonType": "string"
}
}
…
…
"body": {
"encrypt": {
"keyId": "/user_id",
"algorithm": "…Random",
"bsonType": "string"
}
}
…
{
_id: 584,
user_id: 23,
date: Date("6/8/2019"),
title: "My first entry",
body: "Dear diary, … "
}

114. def delete_user_data(db, user_id):
db.keyvault.delete_one({ "keyAltNames": user_id })

116. def get_follower_posts(db, user):
cursor = db.posts.find_many(
{ "user_id": { "$in": user["followers"] } },
limit = 20,
sort = { "date": DESCENDING }
)
return list(cursor)

117. def list_follower_posts(db, user):
cursor = db.posts.find_many(
{ "user_id": { "$in": user["followers"] } },
limit = 20,
sort = { "date": DESCENDING },
projection = { "body": False }
)
return list(cursor)

119. (used by journalists and teens)

120. EAST
DICTATORLAND
Users
Global Shards

121. EAST
DICTATORLAND

122. EAST
DICTATORLAND
{ _id: 1, body: BinData(6, "A81…") }
{ _id: 2, body: BinData(6, "017…") }
{ _id: 3, body: BinData(6, "5E1…") }
…

124. THANK YOU