Transparent Data Encryption in PostgreSQL

2. 2Copyright©2019 NTT Corp. All Rights Reserved. • Database servers are often the primary target of the following attacks • Privilege abuse • Database SQL injections attacks • Storage media theft • Eavesdropping attacks between client and server • etc. Database Security Threats DB administratorApplications Database server Eavesdropping attacks SQL injections Privilege abuse Physical storage theft

4. 4Copyright©2019 NTT Corp. All Rights Reserved. • Protect data from attacks bypassing database access control layer(ACL) • Read database file directly • Taking a backup • Doesn’t protect from attacks by malicious “privileged” users • SELECT SQL command by superuser • Data is not encrypted while being used • On shared buffer, on network • Often implements as transparent data encryption(TDE) Data at rest Encryption

6. 6Copyright©2019 NTT Corp. All Rights Reserved. • Provide set of cryptographic functions • A convenient tool But, • Not transparent to users • Need to modify SQL, application code • Triggers and views help • Could be a cause of performance overhead • Data needs to be decrypted every time it is accessed How About contrib/pgcrypto?

8. 8Copyright©2019 NTT Corp. All Rights Reserved. Per tablespace encryption • CREATE TABLESPACE enctblsp ... WITH (encryption = on); • Fine grained control • Specified table and its indexes, TOAST table and WAL are transparently encrypted • Also encrypt other objects such as system catalogs and temporary files • Under discussion on pgsql-hackers • [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS) Proposal

10. 10Copyright©2019 NTT Corp. All Rights Reserved. PostgreSQL I/O Architecture postgres Disk postgres postgres Page Cache (Kernel) raw block data Shared Buffer Backend processes read pages from the shared buffers and modify them.

11. 11Copyright©2019 NTT Corp. All Rights Reserved. PostgreSQL I/O Architecture postgres Disk postgres postgres Page Cache (Kernel) raw block data Shared Buffer bgwriter periodically writes the dirty pages out to the kernel page cache.

12. 12Copyright©2019 NTT Corp. All Rights Reserved. PostgreSQL I/O Architecture postgres Disk postgres postgres raw block data Shared Buffer Page Cache (Kernel) Dirty pages are flushed to the disk by the checkpointer or the kernel.

13. 13Copyright©2019 NTT Corp. All Rights Reserved. Buffer Level Encryption (our solution) postgres Shared Buffer Disk Pros: • Relatively less execution of encryption and decryption • Prevent peeking file on disk Cons: • Possibly repeated encryption and decryption of same data if the database doesn’t fit in shared buffers postgres postgres Page Cache (Kernel) raw data encrypted data

14. 14Copyright©2019 NTT Corp. All Rights Reserved. Latency (90%tile): vanilla: 1.98 ms, TDE: 2.01 ms, pgcrypto: 2.28 ms Results 6000 6500 7000 7500 8000 8500 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 TPS Duraiton(sec) TPS comparison (R:100,W:3) vanilla tde pgcrypto 8000 8500 9000 9500 10000 10500 11000 10 30 50 70 90 110 130 150 170 190 210 230 250 270 TPS Duration (sec) TPS comparison (R:100) vanilla tde pgcrypto Latency (90%tile): vanilla: 2.32 ms, TDE: 2.45 ms, pgcrypto: 2.66 ms DB size < shared buffers DB size > shared buffers

15. 15Copyright©2019 NTT Corp. All Rights Reserved. • Advanced Encryption Standard(AES) • Symmetric key algorithm • AES-256 • Block cipher • 16 bytes block size • Using openssl is preferable (--with-openssl) • AES-NI • Block cipher mode of operation • CBC or XTS How To Encrypt

16. 16Copyright©2019 NTT Corp. All Rights Reserved. • For faster key rotation • Master key • Stored outside the database • Encrypt/Decrypt tablespace keys • One key per database cluster • Tablespace Key (= data key) • Stored inside the database • Encrypt/Decrypt database objects • One key per tablespace 2-Tier Key Hierarchy Master Key Encrypt/Decrypt Encrypt/ Decrypt External Location Database Server ENCRYPTED DATA Tablespace key

17. 17Copyright©2019 NTT Corp. All Rights Reserved. • For faster key rotation • Master key • Stored outside the database • Encrypt/Decrypt tablespace keys • One key per database cluster • Tablespace Key (= data key) • Stored inside the database • Encrypt/Decrypt database objects • One key per tablespace 2-Tier Key Hierarchy Master Key Encrypt/Decrypt Encrypt/ Decrypt External Location Database Server ENCRYPTED DATA Tablespace key New Master Key

18. 18Copyright©2019 NTT Corp. All Rights Reserved. • Key management is very important • How can we robustly manage the master key? • Better leave it to a specialist • Usually support some kinds of protocols • KMIP, HTTPS etc Key Management

19. 19Copyright©2019 NTT Corp. All Rights Reserved. • Key manager manages a key management plugin as well as tablespace keys • Add generic interface between PostgreSQL and key management systems (Key management API) Integration with Key Management Systems Key management API get_key(), generate_key(), remove key() Encrypted file A KMS B KMS Bufmgr, smgr, encryption etc File A KMS A KMS KMIP HTTPSread/write Key manager (keyring) Encrypted Tablespace keys Shared Memory master key Local Memory Tablespace keys shared buffer

20. 20Copyright©2019 NTT Corp. All Rights Reserved. • PostgreSQL gets the master key from KMS at startup • Cache the master key on the shared memory • Risk of key leakage when memory dump • MADV_DONTDUMP of madvise(2) helps • Risk of key leakage when swapped out • mlock(2) helps • Backend processes get the encrypted tablespace key at startup and decrypt all of them with the master key Caching Keys

21. 21Copyright©2019 NTT Corp. All Rights Reserved. • WAL Block Encryption • Encrypt WAL block every commit time • WAL writer could encrypt • WAL Record encryption • Encrypt WAL when inserting to WAL buffer • Doesn’t encrypt WAL data that is not pertaining to encrypted tables WAL Encryption A block on WAL Buffer WAL file writeencrypt & write WAL file memcpy encrypt & memcpy 1. Encrypt WAL blocks 2. Encrypt WAL records

22. 22Copyright©2019 NTT Corp. All Rights Reserved. • It’s more secure if we use the same encryption key for WAL encryption as that used for table • Choice #2 would be better approach WAL Encryption A block on WAL Buffer WAL file writeencrypt & write WAL file memcpy encrypt & memcpy 1. Encrypt WAL blocks 2. Encrypt WAL records

23. 23Copyright©2019 NTT Corp. All Rights Reserved. Performance Overhead of WAL Encryption • Compare performance on insert-heavy workload • Encrypt all WAL blocks/records • pg_wal directory on tmpfs to avoid disk I/O bottleneck • Each transaction inserts a few records and commit • Max 7% degradation 1.00 1.06 1.07 1.05 1.04 0.00 0.20 0.40 0.60 0.80 1.00 1.20 No Encrytpion WAL Block WAL Record WAL Record (1/2) WAL Record (1/5) INSERT 10M rows (tempfs)

24. 24Copyright©2019 NTT Corp. All Rights Reserved. • pg_wal on HDD • No big performance overhead Performance Overhead of WAL Encryption 1.00 1.01 1.00 0.00 0.20 0.40 0.60 0.80 1.00 1.20 No Encrytpion WAL Block WAL Record INSERT 50k rows (HDD)

25. 25Copyright©2019 NTT Corp. All Rights Reserved. WAL Record Format XLogRecord XLogRecordBlockHeader (RelfileNode, BlockNumber) XLogREcordBlockImageHeader XLogRecordDataHeaderShort Full page image (w/o hole) for new buffer xl_heap_header new tuple xl_heap_update xl_heap_header old tuple An example of xl_heap_update (wal_level = logical) Header data No user data is stored Block data FPI and tuples are stored Main data Could also contain tuples

26. 26Copyright©2019 NTT Corp. All Rights Reserved. WAL Record Encryption XLogRecord XLogRecordBlockHeader (RelfileNode, BlockNumber) XLogRecordBlockImageHeader XLogRecordDataHeaderShort Full page image (w/o hole) for new buffer xl_heap_header new tuple xl_heap_update xl_heap_header old tuple Choice #1: Encrypt whole WAL record • Need another header containing ciphertext length and tablespace oid (key of encryption key) • Need decryption before validation • Frontend programs(pg_waldump, pg_rewind etc) need to obtain tablespace keys and master key Choice #2: Encrypt only block data + main data • XLogRecordHeader has a flag saying “hey this record is encrypted” • Frontend programs need to obtain tablespace keys and master key Choice #3: Move xl_xxx_xxx to just below header data and #2 • Frontend tools don’t want to see user data don’t need to decrypt WAL record • Possible?

27. 27Copyright©2019 NTT Corp. All Rights Reserved. WAL Record Encryption XLogRecord (ENCRYPTED!) XLogRecordBlockHeader (RelfileNode, BlockNumber) XLogRecordBlockImageHeader XLogRecordDataHeaderShort Full page image (w/o hole) for new buffer xl_heap_header new tuple xl_heap_update xl_heap_header old tuple Choice #1: Encrypt whole WAL record • Need another header containing ciphertext length and tablespace oid (key of encryption key) • Need decryption before validation • Frontend programs(pg_waldump, pg_rewind etc) need to obtain tablespace keys and master key Choice #2: Encrypt only block data + main data • XLogRecordHeader has a flag saying “hey this record is encrypted” • Frontend programs need to obtain tablespace keys and master key Choice #3: Move xl_xxx_xxx to just below header data and #2 • Frontend tools don’t want to see user data don’t need to decrypt WAL record • Possible?

28. 28Copyright©2019 NTT Corp. All Rights Reserved. WAL Record Encryption XLogRecord (ENCRYPTED!) XLogRecordBlockHeader (RelfileNode, BlockNumber) XLogRecordBlockImageHeader XLogRecordDataHeaderShort xl_heap_update Full page image (w/o hole) for new buffer xl_heap_header new tuple xl_heap_header old tuple Choice #1: Encrypt whole WAL record • Need another header containing ciphertext length and tablespace oid (key of encryption key) • Need decryption before validation • Frontend programs(pg_waldump, pg_rewind etc) need to obtain tablespace keys and master key Choice #2: Encrypt only block data + main data • XLogRecordHeader has a flag saying “hey this record is encrypted” • Frontend programs need to obtain tablespace keys and master key Choice #3: Move xl_xxx_xxx to just below header data and #2 • Frontend tools don’t want to see user data don’t need to decrypt WAL record • Possible?

29. 29Copyright©2019 NTT Corp. All Rights Reserved. • Temporary files are written bypassing the shared buffers • base/pgsql_tmp/ • pg_replslots/ • pg_stat_statements Temporary File Encryption postgres Shared Buffer Disk temp files

30. 30Copyright©2019 NTT Corp. All Rights Reserved. • Temporary files encryption could use “a disposable key” • Generated randomly by each backend process before use • lives only during process lifetime • No other process need to read temporary files • Interface problem • Non-uniformed file access interfaces Disposable Key

31. 31Copyright©2019 NTT Corp. All Rights Reserved. CREATE DATABASE ... TABLESPACE enc_tblsp; • System catalogs could have user sensitive data • pg_statistics, pg_statistics_ext, pg_proc, pg_class etc • System catalogs of an encrypted database are encrypted • Encrypt all system catalogs in database that is created on a encrypted tablespace System Catalogs Encryption

32. 32Copyright©2019 NTT Corp. All Rights Reserved. • Per tablespace, buffer-level transparent data at rest encryption • Less performance overhead • Encrypt WAL, system catalogs and temporary files as well • 2-tier key architecture • Fast key rotation • Integration with KMSs • Provide more flexible and robust key management Conclusion Remarks

33. 33Copyright©2019 NTT Corp. All Rights Reserved. Two proposals • Cluster-wide data at rest encryption is under development • "WIP: Data at rest encryption" patch and, PostgreSQL 11-beta3 • Proposed by Antonin Houska • Per-Tablespace data at rest encryption • Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS) • Proposed by Moon Insung, Masahiko Sawada Current Status

35. 35Copyright©2019 NTT Corp. All Rights Reserved. • Block cipher mode of operation • https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation • Disk encryption theory • https://en.wikipedia.org/wiki/Disk_encryption_theory#XEX- based_tweaked-codebook_mode_with_ciphertext_stealing_(XTS) Some References

37. 37Copyright©2019 NTT Corp. All Rights Reserved. • CTR mode turns a block cipher into a streaming cipher • Stream cipher: byte-to-byte encryption • Unlike block mode cipher, random read is available • Used for stream data such as network packets CTR (Counter) Mode https://en.wikipedia.org/wiki/Disk_encryption_theory

38. 38Copyright©2019 NTT Corp. All Rights Reserved. • The characteristics of WAL is quite similar to stream data • Append only • Data once written is never updated • Stream cipher doesn’t need padding even for 15 byte or less data Why Can CTR Mode be Used for WAL Encryption?

Transparent Data Encryption in PostgreSQL

Transparent Data Encryption in PostgreSQL

Recommended

More Related Content

What's hot

What's hot(20)

Similar to Transparent Data Encryption in PostgreSQL

Similar to Transparent Data Encryption in PostgreSQL(20)

More from Masahiko Sawada

More from Masahiko Sawada(20)

Recently uploaded

Recently uploaded(20)

Transparent Data Encryption in PostgreSQL