Would Your Database Lie to You? Cryptographic Review and Data Integrity, Too!
•Download as PPTX, PDF•
0 likes•153 views
Often, discussions about security in Accumulo focus on access control mechanisms, ignoring data-at-rest confidentiality and data integrity. We have recently modified the implementation of data encryption in Accumulo to improve data protection, including both confidentiality and integrity. Additionally, we are working to design and implement a new cryptography module that will allow us to centralize the code, clean the API, and make configuration easier for system administrators. The new crypto module will provide more flexibility for organizations interested in developing their own approaches to data encryption. In this talk we will discuss the existing cryptography within Accumulo, the motivation behind refactoring the crypto module, and the design of the new module. We will also discuss the implementation progress to date.
Recommended
More Related Content
Similar to Would Your Database Lie to You? Cryptographic Review and Data Integrity, Too!
Similar to Would Your Database Lie to You? Cryptographic Review and Data Integrity, Too! (20)
Recently uploaded
Recently uploaded (20)
Would Your Database Lie to You? Cryptographic Review and Data Integrity, Too!
- 1. Would your database lie to you? Cryptographic review and data integrity, too! Nick Felts Praxis Engineering
- 2. Topics to be addressed • Philosophical – What are we encrypting? – Why are we encrypting it? – How are we encrypting it? – How are we handling keys? • Development – What was the initial plan? – How did that work out? – What is the current progress?
- 3. Quick crypto refresher • Symmetric encryption – Same key used to encrypt and decrypt – Faster than asymmetric • Key terms – Keys: Key-encryption-key (KEK) file-encryption-key (FEK) – Algorithm: The cipher being used (AES) – Mode: Method for encrypting multiple blocks – Initialization Vector (IV): Extra data for some modes – Padding: Adding bits to round out block size
- 4. What and why? • File vs full disk – Protection from exfiltration of files – Segregation of responsibilities – Use both! • Main file types – RFile – WAL file
- 5. RFiles • Relative key files • Sorted, long term storage Data blocks Meta blocks Tail RFile Structure
- 6. WAL Files • Preserves data in case of system failure • Unsorted, short term storage DataHeader WAL Structure
- 7. Initial work • Two file types with two different purposes • Two different approaches to data encryption – WAL: AES/CBC/NoPadding – RFile: AES/GCM/NoPadding • Encrypt file encryption key with AESWrap WAL RFile
- 8. Data integrity via GCM • Authenticated mode – Explicit integrity – Ensure unauthorized changes detected • NIST SP 800-38D[1] – Authenticated file encryption – Requires careful construction of IVs • Java 9+[2][3] – Must support AES/GCM/NoPadding – Provides access to hardware acceleration
- 9. Former crypto implementation What did the crypto implementation look like when we got started?
- 10. Former crypto implementation Accumulo Client Key Encryption Strategy RFile WAL Crypto Module ???Disk File Writer File Stream
- 11. Former crypto implementation Accumulo Client Key Encryption Strategy RFile WAL Crypto Module ???Disk File Writer File Stream
- 12. Former crypto implementation Accumulo Client Key Encryption Strategy RFile WAL Crypto Module ???Disk File Writer File Stream
- 13. Former crypto implementation Accumulo Client Key Encryption Strategy RFile WAL Crypto Module ???Disk File Writer File Stream
- 14. Former crypto implementation Accumulo Client Key Encryption Strategy RFile WAL Crypto Module ???Disk File Writer File Stream
- 15. Former crypto implementation Accumulo Client Key Encryption Strategy RFile WAL Crypto Module ???Disk File Writer File Stream
- 16. Former crypto implementation Accumulo Client Key Encryption Strategy RFile WAL Crypto Module ???Disk File Writer File Stream
- 17. Former crypto implementation Accumulo Client Key Encryption Strategy RFile WAL Crypto Module ???Disk File Writer File Stream
- 18. Former crypto implementation Accumulo Client Key Encryption Strategy RFile WAL Crypto Module ???Disk File Writer File Stream
- 19. Former crypto implementation Accumulo Client Key Encryption Strategy RFile WAL Crypto Module ???Disk File Writer File Stream ??? Incomplete
- 20. Former crypto implementation Accumulo Client Key Encryption Strategy RFile WAL Crypto Module ???Disk File Writer File Stream Two paths to disk
- 21. Crypto redesign goals • Crypto module – Self-contained – Responsible for key management – Agnostic of file formats and vice versa – Aware of calling context – Makes crypto decisions • The interface – Instance-specific crypto not prohibited by design – Flexible and minimal
- 22. Key management redesign • Key management implementation incomplete • Needed a clear method for handling key • Crypto module should store key information • Rekeying the KEK should be possible • Key manager responsible for retrieving from – Key server – File – Card reader
- 23. New design Crypto Service Key Management File I/O Rfile WAL Read Write Read Write Disk
- 24. New design Crypto Service Key Management File I/O Rfile WAL Read Write Read Write Disk File writers request encrypted stream from the crypto service
- 25. New design Crypto Service Key Management File I/O Rfile WAL Read Write Read Write Disk File writers request encrypted stream from the crypto service Crypto service returns relevant parameters for file writers to record
- 26. New design Crypto Service Key Management File I/O Rfile WAL Read Write Read Write Disk Only file writers write to disk
- 27. New design Crypto Service Key Management File I/O Rfile WAL Read Write Read Write Disk Crypto service is self-contained
- 28. New design Crypto Service Key Management File I/O Rfile WAL Read Write Read Write Disk Key management is the responsibility of the crypto module
- 29. New design Crypto Service Key Management File I/O Rfile WAL Read Write Read Write Disk~1600 lines of code removed (PR #560)
- 30. Redesign features • Multiple decrypting KEKs • Instance specific encrypting KEK • Pluggable crypto module • Scalable key management • Simpler configuration
- 31. Accumulo configuration • Requires system administrators to have cryptographic expertise • Overly complicated • Unclear options • Convoluted KEK management
- 32. Accumulo configuration • Easy to understand • Few required values • Simple KEK control Note: Moved to Java properties in a later update
- 33. Redesign goals revisited • Crypto module – Self-contained – Responsible for key management – Agnostic of file formats and vice versa – Aware of calling context – Makes crypto decisions • The interface – Instance-specific crypto not prohibited by design – Flexible • Key manager redesign
- 34. Redesign goals revisited • Crypto module – Self-contained – Responsible for key management – Agnostic of file formats and vice versa – Aware of calling context – Makes crypto decisions • The interface – Instance-specific crypto not prohibited by design – Flexible • Key manager redesign
- 35. Redesign goals revisited • Crypto module – Self-contained – Responsible for key management – Agnostic of file formats and vice versa – Aware of calling context – Makes crypto decisions • The interface – Instance-specific crypto not prohibited by design – Flexible • Key manager redesign
- 36. Redesign goals revisited • Crypto module – Self-contained – Responsible for key management – Agnostic of file formats and vice versa – Aware of calling context – Makes crypto decisions • The interface – Instance-specific crypto not prohibited by design – Flexible • Key manager redesign
- 37. Redesign goals revisited • Crypto module – Self-contained – Responsible for key management – Agnostic of file formats and vice versa – Aware of calling context – Makes crypto decisions • The interface – Instance-specific crypto not prohibited by design – Flexible • Key manager redesign
- 38. Redesign goals revisited • Crypto module – Self-contained – Responsible for key management – Agnostic of file formats and vice versa – Aware of calling context – Makes crypto decisions • The interface – Instance-specific crypto not prohibited by design – Flexible • Key manager redesign
- 39. Redesign goals revisited • Crypto module – Self-contained – Responsible for key management – Agnostic of file formats and vice versa – Aware of calling context – Makes crypto decisions • The interface – Instance-specific crypto not prohibited by design – Flexible • Key manager redesign
- 40. Redesign goals revisited • Crypto module – Self-contained – Responsible for key management – Agnostic of file formats and vice versa – Aware of calling context – Makes crypto decisions • The interface – Instance-specific crypto not prohibited by design – Flexible • Key manager redesign
- 41. Redesign goals revisited • Crypto module – Self-contained – Responsible for key management – Agnostic of file formats and vice versa – Aware of calling context – Makes crypto decisions • The interface – Instance-specific crypto not prohibited by design – Flexible • Key manager redesign
- 42. Timing tests • Performed using uno and Java 9 • Building Accumulo commit: 02f58119a283e2552f0577c732ef8aa0aa5226a1 • Ingest/Writing – Accumulo.test.TestIngest for 3,000,000 rows – ran each service 20 times – NoCryptoService • Fastest ingest: 53.877s • Slowest ingest: 61.888s • Average ingest: 57.234s – AESCryptoService • Fastest ingest: 58.675s • Slowest ingest: 65.017s • Average ingest: 61.390s – Average slowdown: 7.261% • Scanning/Reading – In progress 42
- 43. What might be next? • Intermediate WAL files – Currently MapReduce Map files • More key management options – Key server (e.g. PyKMIP[4]) – Card reader • Explore multiple encrypting KEKs
- 44. Questions? • Answers. • Contact: – Github: PircDef – Email: ngfelts@praxiseng.com
- 45. References [1] https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-38d.pdf [2] https://docs.oracle.com/javase/9/docs/api/javax/crypto/Cipher.html [3] https://docs.oracle.com/javase/9/whatsnew/toc.htm [4] https://github.com/OpenKMIP/PyKMIP