Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Encrypt your volumes with barbican open stack 2018

Presentation from OpenStack 2018 Summit

  • Login to see the comments

Encrypt your volumes with barbican open stack 2018

  1. 1. Encrypt Your Volumes with Barbican
  2. 2. Agenda • Introduction • About OnRamp • Why Encrypt? • Key Managers • How it Works • Our Encryption Story • Gotchas & Limitations • Demo • Q&A
  3. 3. Meet the Speaker • Previously Built and Managed Lab Environments for VMware Certification Training (VCP5/6) • Experience with OpenStack: - Started with Icehouse on Ubuntu (manually) - Juno on Canonical MaaS + Juju - Liberty on CentOS / RDO - Newton on Platform9 • OpenStack Engineer for OnRamp • Deployed Barbican and support for Encrypted Volumes for our OpenStack Based Public Cloud Duncan Wannamaker OpenStack Engineer OnRamp
  4. 4. But, OpenStack is Far From Simple, Right?
  5. 5. Who Is OnRamp? OnRamp is a HITRUST-certified data center services company specializing in high security and compliant hybrid hosting.
  6. 6. OnRamp’s Virtual Private Cloud ■ Built on OpenStack® ■ HITRUST-certified ■ Control costs with capped maximum resource usage ■ Open-source APIs enable simple migrations and eliminate vendor lock-in ■ Hardened Linux and Windows Images ■ Retain full control of your private networks, virtual machines, and storage Offers the ease-of-use of a public cloud, with the security of a private cloud.
  7. 7. Why Encrypt? • Protect Data against Leaks - Personal Health Information (PHI) - Credit Card Payment Data (PCI) - Intellectual Property • Compliance Requirement - HIPAA requires encryption for data at rest and in-transit for PHI - PCI also requires encryption for data at rest and in-transit for cardholder data - In shared hosting environments, each Tenant must only have access to their own stuff  Per-Tenant or Per-Volume encryption keys facilitate this
  8. 8. Why Use a Key Manager? • Security Best Practice - You don’t leave keys in your car - You shouldn’t leave your keys next to your encrypted data • Compliance Requirement - PCI-DSS Requirements:  Store keys in the fewest possible locations  Store secret and private keys used to encrypt/decrypt cardholder data using a key encryption key at least as strong as the encryption key itself - HIPAA- HITRUST Requirements:  Keys shall be stored separately from encrypted data  Key manager systems should be physically protected by fewest number of custodians necessary
  9. 9. What is Barbican? Provides: • ReST API for Secrets Management • Pluggable Backends - Simple Crypto - PKCS#11 and KMIP (HSM) • Integration with Nova, Cinder, and Swift, Neutron, Heat, and many other OS projects • Integration with KeyStone for Auth and RBAC • Built to Scale Does Not Provide: • Graphical User Interface • Key Splitting for Secure Import/Export Plain Text Keys using multiple Key Custodians • Generation of X.509 Certificates (Since Pike) • Volume Encryption
  10. 10. What about Volume Encryption? • LUKS - Linux Unified Key Setup - Allows for multiple user keys or passwords per volume  Master Key always stays the same - Supports CPU Hardware Acceleration (it’s fast!) - Uses CryptSetup and DM-CRYPT  Decrypts full volume to a Local Block Device  Protects iSCSI attached volumes  Can also protect ephemeral storage if using LVM • Queens = QEMU Native LUKS support - QEMU 2.6 and LibVirt 2.2 introduce native LUKS support
  11. 11. Transparent Encryption in Nova and Cinder Thanks to some fantastic work done by Johns Hopkins Applied Physics Laboratory and others… • Nova and Cinder Integration exists with Barbican • Volume Decryption Happens on the Hypervisor instead of within the Guest OS - No Agent Required - Works with Any Operating System - Works with Bootable Volumes - Protects Data at Rest and In-Transit to your hypervisor - Every volume is protected by it’s own unique key
  12. 12. How it Works: Creating an Encrypted Volume 1. User Gets a token from Keystone 2. User Asks Cinder to create volume (using token) 3. Cinder verifies the user token from keystone 4. Cinder then asks Barbican for a key 5. Barbican checks the Cinder Token against keystone 6. Barbican creates the secret and returns secret HREF to Cinder 7. Cinder stores the returned secret HREF into the volume metadata
  13. 13. How it Works: Mounting an Encrypted Volume Nova Keystone Cinder Barbican Fetch Secret HREF from Cinder Validate Request Return Secret HREF and Attach Volume Receive Secret HREF Fetch Secret from Barbican Validate Token Validate Token Receive User Mount Request Validate Request Validate Requestor Return Secret Mount Volume using Secret 1. Nova receives request to mount volume with token 2. Nova Validates User Token 3. Nova Fetches the Secret HREF from Cinder 4. Cinder Validates Nova Token 5. Cinder returns Secret HREF to Nova 6. Nova Fetches Secret from Barbican 7. Barbican Validates Nova Token 8. Barbican Returns Secret 9. Nova Mounts Volume using Secret
  14. 14. OnRamp Encryption Story • Followed Documentation - Wait, these docs are terrible!   Thankfully, Barbican Devs on IRC are very helpful!  • KeyStone: - Added Endpoints to KeyStone - Added Barbican Service User - Added Creator Role • Installed the Barbican-API: - Configured Barbican’s database server - Configured Barbican - Barbican CLI Access Works!
  15. 15. OnRamp Encryption Story • Cinder Integration Issues: - Blank Encrypted Volumes were created successfully - But they could not be created from an image • Nova Integration Issues: - Nova was unable to attach any encrypted volumes - It wasn’t even trying to talk to Barbican
  16. 16. Issue 1: Key orders not getting to Barbican cinder-api Cloud-Hosted cinder-volume On-Premise SSH VPN barbican On-Premise Key Orders LAN Encrypted Volume Cinder DB Secret HREF: null key request for key 0000… href = all zeros! Mount Failed no key 0000… Create Volume Mount Request
  17. 17. Issue 1 Fix: Key orders not getting to Barbican cinder-api cinder-volume forwarder SSH VPN barbican Key Order LAN Encrypted Volume Mount Cinder DB Secret HREF: valid Key Order Cloud-Hosted On-Premise On-Premise Secret HREFSecret HREF Create Volume Mount Request valid href request secretreturn secret
  18. 18. Issue 2: Nova Not Talking to Barbican • Nova error was mentioning a Fixed Key not being defined • ConfKeyManager (Nova Default for Volume Encryption) • Single fixed key for all volumes • Used for testing to substitute in for a real key manager, such as Barbican! Not for production. • Setting the api_class in nova.conf would always use ConfKeyManager and ignore the setting to use barbican • Submitted Bug: • • Fixed in Pike or newer! • Manual Fix for older releases is to comment out lines 27-31 in nova/keymgr/
  19. 19. Some Gotchas using Encrypted Volumes • Live Migration does not work and is dangerous - Volume mounts without decryption on new host which will cause corruption if this is an active file system - Fixed in Queens!  Previous releases use symlinks on hypervisor  New Method uses QEMU Native LUKS support • Barbican Doesn’t Start after Reboot (CentOS Specific) - RDO Packaging Issue - Create /etc/tmpfiles.d/barbican.conf:  d /var/run/barbican 0755 barbican barbican – - Bug report submitted…
  20. 20. Some Limitations using Encrypted Volumes • No mechanism to rotate volume encryption keys - Manual process of creating a new volume and copying over the contents - LUKs supports multiple user keys, so the capability is there • No UI for key management in horizon - for securely exporting and importing split-keys - for managing key ACL’s - for managing key expiration and revocation
  21. 21. Tips for Running in Production • Make sure your key manager and database are secured in a locked cabinet with limited physical access • Use a private barbican instance not accessible to tenants • Automated database backups • Use a highly available database cluster such as Galera • Use multiple barbican-api nodes behind a load balancer • Use SSL to protect key requests in-transit to hypervisors
  22. 22. Demo
  23. 23. Questions?
  24. 24. Barbican PoC Guide and Resources • Step by step instructions in GitHub: • Video walkthrough available: - YouTube Search: “Queens Barbican PoC”  Installing Packstack for Queens (Part 1)  Installing the Barbican-API (Part 2) • IRC Channel: - #openstack-barbican on FreeNode
  25. 25. Thank you! Stay In Touch: @randomhack 888.667.2660