Format
Preserving
Encryption
a lightning intro
Brad Schoening
bschoeni@gmail.com
Encryption
AES Encryption
U2FsdGVkX1/yGbJ4mi2FRoaC
ZihRyVbHFK9lhINEKE6omb40
M+XWp3lmvmU2WLZT
4000 0012 3456 7899
Smith and Brightwell 1997
Using Datatype-Preserving Encryption to
Enhance Data Warehouse Security
Ciphertext bears roughly the same resemblance to plaintext as a hamburger
does to a T-bone steak.
A social security number, encrypted using the DES encryption algorithm, not only
does not resemble a social security number but will likely not contain any
numbers at all.
A database field defined to hold a nine character social security number would
not store the DES-encrypted version of the data. A Visual Basic program would
not read it. A graphical interface would not display it.
There would be nothing that you could do with the encrypted social security
number unless you had made extensive provisions for changes in data format
throughout your application and physical database design.
Format Preserving Encryption (FPE)
Format Preserving Encryption
5107 0855 4069 8446
4000 0012 3456 7899
SSN’
381-09-7847
SSN
078-05-1120
800-38G
March 2016
800-38G
Revision 1
Feb 2019
Code Example
% python3
>>> from ff3 import FF3Cipher
>>> key = "EF4359D8D580AA4F7F036D6F04FC6A94"
>>> tweak = "D8E7920AFA330A73"
>>> c = FF3Cipher(10, key, tweak)
>>> plaintext = "4000001234567899"
>>> ciphertext = c.encrypt(plaintext)
>>> ciphertext
'5107085540698446'
>>> decrypted = c.decrypt(ciphertext)
>>> decrypted
'4000001234567899'
16 digits
https://github.com/bschoening/fpe
March
2016
NIST 800-38G
approved:
FF1 & FF3
February
2019
NIST FPE
Revision 1
Draft*
FF1, FF3-1
April
2017
Breaking FF3
Durak &
Vaudenay
August
2018
The Curse of
Small Domains
Hoang, Tessaro,
Trieu
April
2017
NIST suspends
FF3
2010
FF1, FF2,
FF3 submitted
to NIST
1997
Datatype-
Preserving
Encryption
Brightwell &
Smith
FPE Limitations
• Radix
• digits 0..9  10
• alphabetic A-Z  26
• alphanumeric 0..9, A-Z  36
• Plaintext
• Min domain > 1M
• Max length between 36 and 56
• Key management
• KMS for keys and tweaks
‘Tux’ Linux mascot Encrypted with AES ECB
See https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#ECB
FPE Limitations (cont)
Encrypted with AES CBC
FPE Use Cases
✓ Social Security Number
✓ Credit Card Number
✓ Phone Number
✓ Drivers License
✓ License Plate
❌ DOB
❌ CCV
❌ PIN Codes
Other implementations
• Korean FPE
• FEA-1 and FEA-2 (Lee, Koo, Roh, Kim, Kwon, ICISC 2014)
• Data Type Preserving (DTP) Encryption by Protegrity
• Claimed to be more secure than NIST’s FPE
• “DTP is 10X weaker” (2018 Hoang, Tessaro, Trieu)
Vulnerability
FF1 & FF3 Known plaintext On small domain size
DPT Cyphertext only Any domain size
Open Source NIST 800-38G FPE
github.com/0NG/Format-Preserving-Encryption
github.com/capitalone/fpe
github.com/bschoening/fpe
github.com/zone1511/fpe4j
 NIST Spec defines fifteen test “vectors” or cases
Key
Takeaways
FPE is ready & safe
Unmeasurable risks with proprietary
FPE
Primary key PII = best use
Not a general purpose solution, use
with other PETs
Thank You
Brad Schoening
Privacy Logistics / Schoening Consulting LLC
bschoeni@gmail.com
www.privacylogistics.com
(917) 304-7190

Format preserving encryption (FPE) - A Brief Intro

  • 1.
  • 2.
  • 3.
    Smith and Brightwell1997 Using Datatype-Preserving Encryption to Enhance Data Warehouse Security Ciphertext bears roughly the same resemblance to plaintext as a hamburger does to a T-bone steak. A social security number, encrypted using the DES encryption algorithm, not only does not resemble a social security number but will likely not contain any numbers at all. A database field defined to hold a nine character social security number would not store the DES-encrypted version of the data. A Visual Basic program would not read it. A graphical interface would not display it. There would be nothing that you could do with the encrypted social security number unless you had made extensive provisions for changes in data format throughout your application and physical database design.
  • 4.
    Format Preserving Encryption(FPE) Format Preserving Encryption 5107 0855 4069 8446 4000 0012 3456 7899 SSN’ 381-09-7847 SSN 078-05-1120
  • 5.
  • 6.
    Code Example % python3 >>>from ff3 import FF3Cipher >>> key = "EF4359D8D580AA4F7F036D6F04FC6A94" >>> tweak = "D8E7920AFA330A73" >>> c = FF3Cipher(10, key, tweak) >>> plaintext = "4000001234567899" >>> ciphertext = c.encrypt(plaintext) >>> ciphertext '5107085540698446' >>> decrypted = c.decrypt(ciphertext) >>> decrypted '4000001234567899' 16 digits https://github.com/bschoening/fpe
  • 7.
    March 2016 NIST 800-38G approved: FF1 &FF3 February 2019 NIST FPE Revision 1 Draft* FF1, FF3-1 April 2017 Breaking FF3 Durak & Vaudenay August 2018 The Curse of Small Domains Hoang, Tessaro, Trieu April 2017 NIST suspends FF3 2010 FF1, FF2, FF3 submitted to NIST 1997 Datatype- Preserving Encryption Brightwell & Smith
  • 8.
    FPE Limitations • Radix •digits 0..9  10 • alphabetic A-Z  26 • alphanumeric 0..9, A-Z  36 • Plaintext • Min domain > 1M • Max length between 36 and 56 • Key management • KMS for keys and tweaks
  • 9.
    ‘Tux’ Linux mascotEncrypted with AES ECB See https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#ECB FPE Limitations (cont) Encrypted with AES CBC
  • 10.
    FPE Use Cases ✓Social Security Number ✓ Credit Card Number ✓ Phone Number ✓ Drivers License ✓ License Plate ❌ DOB ❌ CCV ❌ PIN Codes
  • 11.
    Other implementations • KoreanFPE • FEA-1 and FEA-2 (Lee, Koo, Roh, Kim, Kwon, ICISC 2014) • Data Type Preserving (DTP) Encryption by Protegrity • Claimed to be more secure than NIST’s FPE • “DTP is 10X weaker” (2018 Hoang, Tessaro, Trieu) Vulnerability FF1 & FF3 Known plaintext On small domain size DPT Cyphertext only Any domain size
  • 12.
    Open Source NIST800-38G FPE github.com/0NG/Format-Preserving-Encryption github.com/capitalone/fpe github.com/bschoening/fpe github.com/zone1511/fpe4j  NIST Spec defines fifteen test “vectors” or cases
  • 13.
    Key Takeaways FPE is ready& safe Unmeasurable risks with proprietary FPE Primary key PII = best use Not a general purpose solution, use with other PETs
  • 14.
    Thank You Brad Schoening PrivacyLogistics / Schoening Consulting LLC bschoeni@gmail.com www.privacylogistics.com (917) 304-7190

Editor's Notes

  • #3 openssl aes-256-cbc -in /tmp/cc.txt -a     16 vs 65 characters
  • #5 openssl aes-256-cbc -in /tmp/cc.txt -a     16 vs 65 characters
  • #7 Walk you through a python example Key 16 bytes (128 bits) Tweak 8 bytes (64 bits)
  • #8 Using Datatype-Preserving Encryption to Enhance Data Warehouse Security Vaudenay: Swiss Federal Institute of Technology 
  • #9 Mono-case
  • #10 Electronic Code Book (ECB)  Cipher Blocker Chaining (CBC) Limited color palette