Transparent Data Encryption in PostgreSQL

Copyright©2019 NTT Corp. All Rights Reserved.
Transparent Data Encryption in PostgreSQL
NTT Open Source Software Center
Masahiko Sawada
PGCon 2019

2Copyright©2019 NTT Corp. All Rights Reserved.
• Database servers are often the primary target of the
following attacks
• Privilege abuse
• Database SQL injections attacks
• Storage media theft
• Eavesdropping attacks between client and server
• etc.
Database Security Threats
DB administratorApplications
Database server
Eavesdropping
attacks
SQL
injections
Privilege
abuse
Physical storage
theft

Encryption
Database Server
Application Server

• Protect data from attacks bypassing database access
control layer(ACL)
• Read database file directly
• Taking a backup
• Doesn’t protect from attacks by malicious “privileged”
users
• SELECT SQL command by superuser
• Data is not encrypted while being used
• On shared buffer, on network
• Often implements as transparent data encryption(TDE)
Data at rest Encryption

• Full disk encryption (e.g. dmcrypt) is platform dependent
• Doesn’t protect data from logged-in OS users
How About Full Disk Encryption？

• Provide set of cryptographic functions
• A convenient tool
But,
• Not transparent to users
• Need to modify SQL, application code
• Triggers and views help
• Could be a cause of performance overhead
• Data needs to be decrypted every time it is accessed
How About contrib/pgcrypto?

Transparent Data Encryption in
PostgreSQL

Per tablespace encryption
• CREATE TABLESPACE enctblsp ... WITH (encryption = on);
• Fine grained control
• Specified table and its indexes, TOAST table and WAL are
transparently encrypted
• Also encrypt other objects such as system catalogs and
temporary files
• Under discussion on pgsql-hackers
• [Proposal] Table-level Transparent Data Encryption (TDE) and Key
Management Service (KMS)
Proposal

PostgreSQL I/O Architecture
postgres
Shared Buffer
Disk
postgres postgres
Page Cache (Kernel)
raw block data

postgres
Disk
postgres postgres
Page Cache (Kernel)
raw block data
Shared Buffer
Backend processes
read pages from the
shared buffers and
modify them.

postgres
Disk
postgres postgres
Page Cache (Kernel)
raw block data
Shared Buffer
bgwriter periodically
writes the dirty pages
out to the kernel page
cache.

postgres
Disk
postgres postgres
raw block data
Shared Buffer
Page Cache (Kernel)
Dirty pages are
flushed to the disk by
the checkpointer or
the kernel.

Buffer Level Encryption (our solution)
postgres
Shared Buffer
Disk
Pros:
• Relatively less execution
of encryption and
decryption
• Prevent peeking file on
disk
Cons:
• Possibly repeated
encryption and
decryption of same data
if the database doesn’t fit
in shared buffers
postgres postgres
Page Cache (Kernel)
raw data
encrypted data

Latency (90%tile):
vanilla: 1.98 ms, TDE: 2.01 ms,
pgcrypto: 2.28 ms
Results
6000
6500
7000
7500
8000
8500
20
40
60
80
100
120
140
160
180
200
220
240
260
280
300
TPS
Duraiton(sec)
TPS comparison (R:100,W:3)
vanilla tde pgcrypto
8000
8500
9000
9500
10000
10500
11000
10
30
50
70
90
110
130
150
170
190
210
230
250
270
TPS
Duration (sec)
TPS comparison (R:100)
vanilla tde pgcrypto
Latency (90%tile):
vanilla: 2.32 ms, TDE: 2.45 ms,
pgcrypto: 2.66 ms
DB size < shared buffers DB size > shared buffers

• Advanced Encryption Standard(AES)
• Symmetric key algorithm
• AES-256
• Block cipher
• 16 bytes block size
• Using openssl is preferable (--with-openssl)
• AES-NI
• Block cipher mode of operation
• CBC or XTS
How To Encrypt

• For faster key rotation
• Master key
• Stored outside the database
• Encrypt/Decrypt tablespace keys
• One key per database cluster
• Tablespace Key (= data key)
• Stored inside the database
• Encrypt/Decrypt database objects
• One key per tablespace
2-Tier Key Hierarchy
Master Key
Encrypt/Decrypt
Encrypt/
Decrypt
External Location
Database Server
ENCRYPTED
DATA
Tablespace key

• For faster key rotation
• Master key
• Stored outside the database
• Encrypt/Decrypt tablespace keys
• One key per database cluster
• Tablespace Key (= data key)
• Stored inside the database
• Encrypt/Decrypt database objects
• One key per tablespace
2-Tier Key Hierarchy
Master Key
Encrypt/Decrypt
Encrypt/
Decrypt
External Location
Database Server
ENCRYPTED
DATA
Tablespace key
New Master Key

• Key management is very important
• How can we robustly manage the master key?
• Better leave it to a specialist
• Usually support some kinds of protocols
• KMIP, HTTPS etc
Key Management

• Key manager manages a key management plugin as well as tablespace
keys
• Add generic interface between PostgreSQL and key management
systems (Key management API)
Integration with Key Management Systems
Key management API
get_key(), generate_key(), remove key()
Encrypted file A KMS B KMS
Bufmgr, smgr, encryption etc
File A KMS A KMS
KMIP HTTPSread/write
Key manager (keyring)
Encrypted
Tablespace keys
Shared Memory
master key
Local Memory
Tablespace keys
shared buffer

• PostgreSQL gets the master key from KMS at startup
• Cache the master key on the shared memory
• Risk of key leakage when memory dump
• MADV_DONTDUMP of madvise(2) helps
• Risk of key leakage when swapped out
• mlock(2) helps
• Backend processes get the encrypted tablespace key at startup and
decrypt all of them with the master key
Caching Keys

• WAL Block Encryption
• Encrypt WAL block every commit time
• WAL writer could encrypt
• WAL Record encryption
• Encrypt WAL when inserting to WAL buffer
• Doesn’t encrypt WAL data that is not pertaining to encrypted tables
WAL Encryption
A block on
WAL Buffer
WAL
file
writeencrypt & write
WAL
file
memcpy encrypt & memcpy
1. Encrypt WAL blocks 2. Encrypt WAL records

• It’s more secure if we use the same encryption key for WAL encryption as
that used for table
• Choice #2 would be better approach
WAL Encryption
A block on
WAL Buffer
WAL
file
writeencrypt & write
WAL
file
memcpy encrypt & memcpy
1. Encrypt WAL blocks 2. Encrypt WAL records

Performance Overhead of WAL Encryption
• Compare performance on insert-heavy workload
• Encrypt all WAL blocks/records
• pg_wal directory on tmpfs to avoid disk I/O bottleneck
• Each transaction inserts a few records and commit
• Max 7% degradation
1.00
1.06 1.07 1.05 1.04
0.00
0.20
0.40
0.60
0.80
1.00
1.20
No Encrytpion WAL Block WAL Record WAL Record (1/2) WAL Record (1/5)
INSERT 10M rows (tempfs)

• pg_wal on HDD
• No big performance overhead
Performance Overhead of WAL Encryption
1.00 1.01 1.00
0.00
0.20
0.40
0.60
0.80
1.00
1.20
No Encrytpion WAL Block WAL Record
INSERT 50k rows (HDD)

WAL Record Format
XLogRecord
XLogRecordBlockHeader
(RelfileNode, BlockNumber)
XLogREcordBlockImageHeader
XLogRecordDataHeaderShort
Full page image (w/o hole) for new buffer
xl_heap_header
new tuple
xl_heap_update
xl_heap_header
old tuple
An example of xl_heap_update (wal_level = logical)
Header data
No user data is stored
Block data
FPI and tuples are stored
Main data
Could also contain tuples

WAL Record Encryption
XLogRecord
XLogRecordBlockImageHeader
xl_heap_header
new tuple
xl_heap_update
xl_heap_header
old tuple
Choice #1: Encrypt whole WAL record
• Need another header containing ciphertext
length and tablespace oid (key of encryption
key)
• Need decryption before validation
• Frontend programs(pg_waldump, pg_rewind
etc) need to obtain tablespace keys and master
key
Choice #2: Encrypt only block data + main data
• XLogRecordHeader has a flag saying “hey this record
is encrypted”
• Frontend programs need to obtain tablespace keys
and master key
Choice #3: Move xl_xxx_xxx to just below
header data and #2
• Frontend tools don’t want to see user data don’t need
to decrypt WAL record
• Possible?

XLogRecord (ENCRYPTED!)
xl_heap_header
new tuple
xl_heap_update
xl_heap_header
old tuple
key)
key
is encrypted”
and master key
header data and #2
• Possible?

XLogRecord (ENCRYPTED!)
xl_heap_update
xl_heap_header
new tuple
xl_heap_header
old tuple
key)
key
is encrypted”
and master key
header data and #2
• Possible?

• Temporary files are written bypassing the shared buffers
• base/pgsql_tmp/
• pg_replslots/
• pg_stat_statements
Temporary File Encryption
postgres
Shared Buffer
Disk
temp files

• Temporary files encryption could use “a disposable key”
• Generated randomly by each backend process before use
• lives only during process lifetime
• No other process need to read temporary files
• Interface problem
• Non-uniformed file access interfaces
Disposable Key

CREATE DATABASE ... TABLESPACE enc_tblsp;
• System catalogs could have user sensitive data
• pg_statistics, pg_statistics_ext, pg_proc, pg_class etc
• System catalogs of an encrypted database are encrypted
• Encrypt all system catalogs in database that is created on a
encrypted tablespace
System Catalogs Encryption

• Per tablespace, buffer-level transparent data at rest
encryption
• Less performance overhead
• Encrypt WAL, system catalogs and temporary files as well
• 2-tier key architecture
• Fast key rotation
• Integration with KMSs
• Provide more flexible and robust key management
Conclusion Remarks

Two proposals
• Cluster-wide data at rest encryption is under development
• "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3
• Proposed by Antonin Houska
• Per-Tablespace data at rest encryption
• Table-level Transparent Data Encryption (TDE) and Key Management
Service (KMS)
• Proposed by Moon Insung, Masahiko Sawada
Current Status

• Further discussion on pgsql-hackers
• Submit a draft version patch set for PostgreSQL 13
Future Plans

• Block cipher mode of operation
• https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation
• Disk encryption theory
• https://en.wikipedia.org/wiki/Disk_encryption_theory#XEX-
based_tweaked-codebook_mode_with_ciphertext_stealing_(XTS)
Some References

Thank you

• CTR mode turns a block cipher into a streaming cipher
• Stream cipher: byte-to-byte encryption
• Unlike block mode cipher, random read is available
• Used for stream data such as network packets
CTR (Counter) Mode
https://en.wikipedia.org/wiki/Disk_encryption_theory

• The characteristics of WAL is quite similar to stream data
• Append only
• Data once written is never updated
• Stream cipher doesn’t need padding even for 15 byte or
less data
Why Can CTR Mode be Used for WAL Encryption?

Transparent Data Encryption in PostgreSQL

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Transparent Data Encryption in PostgreSQL

Similar to Transparent Data Encryption in PostgreSQL (20)

More from Masahiko Sawada

More from Masahiko Sawada (20)

Recently uploaded

Recently uploaded (20)

Transparent Data Encryption in PostgreSQL