Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deep Dive into Keystone Tokens and Lessons Learned

6,655 views

Published on

Keystone supports four different types of tokens, UUID, PKI, PKIZ, and Fernet. Let’s take a deep dive into:

Understanding token formats
Pros and Cons of each format in Production
Performance across multiple data centers
Token revocation workflow for each of the formats
Horizon usage of the different token types

We previously deployed UUID and PKI in Production and are now moving towards the latest format, Fernet. We would like to share our lessons learned with different formats and help you decide on which format is suitable for your cloud.

Published in: Engineering
  • DOWNLOAD THAT BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download Full EPUB Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download Full doc Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download EPUB Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... Download doc Ebook here { http://bit.ly/2m6jJ5M } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book that can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer that is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story That Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money That the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths that Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • .DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... .DOWNLOAD PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... .DOWNLOAD EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... .DOWNLOAD doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... .DOWNLOAD PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... .DOWNLOAD EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... .DOWNLOAD doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Deep Dive into Keystone Tokens and Lessons Learned

  1. 1. Deep Dive into Keystone Tokens and Lessons Learned Priti Desai & Brad Pokorny
  2. 2. Who are we? Priti Desai Advisory Software Engineer, IBM Brad Pokorny Principal Software Engineer, Symantec Deep Dive into Keystone Tokens and Lessons Learned
  3. 3. What token format should we configure in our OpenStack Deployment?
  4. 4. Token Formats Deep Dive into Keystone Tokens and Lessons Learned UUID PKI PKIZ Fernet
  5. 5. UUID Deep Dive into Keystone Tokens and Lessons Learned
  6. 6. UUID • Simplest and Most Light Weight • Version 4 UUID • Configuration in keystone.conf : [token] provider = keystone.token.providers.uuid.Provider Deep Dive into Keystone Tokens and Lessons Learned
  7. 7. UUID – Token Generation Workflow Keystone KVS Request Token with: • User Name • Password • Project Name Identity Resources Assignment Catalog User Validation Retrieves User ID Token Project Validation Retrieves Project ID and Domain ID Retrieves Roles for this User on the Project or Domain Returns Failure if the User does not have any Role Retrieves Services and Endpoints for all the services Bundles Identity, Resource, Assignment, and Catalog information into Token Payload Creates Token ID : uuid.uuid4().hex Store them in SQL/KVS: • Token ID • Expiration • Valid • User ID • Extra Token Generation Workflow
  8. 8. Sample UUID Token in SQL Backend Deep Dive into Keystone Tokens and Lessons Learned id: f10700e71ff045cbb850072a0bd6a4e6 expires: 2015-10-08 21:18:43 extra: {"token_data": {"token": {"methods": ["password"], "roles": [{"id": "1688449cf1df44839b10a41e3d9b09dd", "name": "admin"}], "expires_at": "2015-10- 08T21:18:43.995255Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "423d45cddec84170be365e0b31a1b15f", "name": "admin"}, "extras": {}, "user": {"domain": {"id": "default", "name": "Default"}, "id": "1334f3ed7eb2483b91b8192ba043b580", "name": "admin"}, "audit_ids": ["bI1EMzqUQM2sqFimOtIPpQ"], "issued_at": "2015-10-08T20:18:43.995284Z"}}, "user": {"domain": {"id": "default", "name": "Default"}, "id": "1334f3ed7eb2483b91b8192ba043b580", "name": "admin"}, "key": "f10700e71ff045cbb850072a0bd6a4e6", "token_version": "v3.0", "tenant": {"domain": {"id": "default", "name": "Default"}, "id": "423d45cddec84170be365e0b31a1b15f", "name": "admin"}, "metadata": {"roles": ["1688449cf1df44839b10a41e3d9b09dd"]}} valid: 1 trust_id: NULL user_id: 1334f3ed7eb2483b91b8192ba043b580
  9. 9. UUID – Keystone Token Validation Workflow Parse Token and Retrieve Metadata Validate Token with: GET v3/auth/tokens • X-Subject-Token • X-Auth-Token Retrieves Token payload from token backend KVS/SQL Read cached token reference and parse: • User ID • Project ID • Audit ID • Token Expiry Token KVS Valid? Current Time < Expiry Time Token Not Found Token Not Found Is Revoked? Token Not Found HTTP/1.1 200 OK Yes No No No Yes Yes Check if a token matches any revocation events Check if a token is expired, current time is calculated in UTC Token Validation Workflow
  10. 10. UUID – Keystone Token Revocation Workflow Revoke by Audit ID Revoke Token with: DELETE v3/auth/tokens • X-Subject-Token • X-Auth-Token Before revoking a token, validate it based on Token Validation Workflow Validate X-Subject-Token Audit ID? Revoke by Token Expiry Create Revoke Event with: Audit ID Revoke At Issued Before Prune Expired Events Set valid to False Yes No Filter existing revocation events based on Revoke At Sample Revocation Event: { "audit_id": "HVvI0d-cTD21yatAfQc4IQ", "issued_before”: "2015-10-24T21:20:45.000000Z" }, Token Revocation Workflow Create Revoke Event with: User ID Project ID Revoke At Issued Before Token Expiry Token KVS
  11. 11. UUID Across Multiple Data Centers Users Groups Domains Projects Roles Catalog Assignments Users Groups Domains Projects Roles Catalog Assignments MySQL Replication (Database is always in sync) LDAP Replication (Directory Tree is always in sync) Tokens KVS UUID Tokens Tokens KVS UUID Tokens Keystone UUID - Multiple Data Centers Nova Keystone Middleware Keystone Nova Keystone Middleware US-EASTUS-WEST Request Token UUID Token Token Validation VM Instance Token Validation Token Not Found Token Found Token Not Found
  12. 12. Pros and cons • Pros – Simplest and Smallest Token Format – Recommended for Simple OpenStack Deployment • Cons – Persistent Token Format – Token validation can only be done by Identity service – Not feasible for multiple OpenStack deployments Deep Dive into Keystone Tokens and Lessons Learned
  13. 13. PKI/PKIZ Deep Dive into Keystone Tokens and Lessons Learned
  14. 14. PKI • Cryptographically Encrypted Signed Document using X509 Standards • CMS • Converted to custom URL- Safe format • Compressed PKI • Prefixed with “PKIZ” Deep Dive into Keystone Tokens and Lessons Learned PKIZ
  15. 15. PKI/PKIZ Configuration - Certificates • Signing Key (signing_key.pem) : • Generate private key in PEM format • Signing Certificate (signing_cert.pem) : • Generate CSR using Signing Key • Submit CSR to CA • Receive Certificate from CA • Certificate Authority Certificate (ca.pem) Deep Dive into Keystone Tokens and Lessons Learned
  16. 16. PKI/PKIZ Configuration • Configuration in keystone.conf : [token] provider = keystone.token.providers.[pki|pkiz].Provider [signing] certfile = /etc/keystone/ssl/certs/signing_cert.pem keyfile = /etc/keystone/ssl/private/signing_key.pem ca_certs = /etc/keystone/ssl/certs/ca.pem Deep Dive into Keystone Tokens and Lessons Learned
  17. 17. PKI/PKIZ – Token Generation Workflow Validate Identity, Resource, and Assignment Request Token with: • User Name • Password • Project Name Token Generation Workflow Create JSON Token Payload Sign JSON Payload with Signing Key and Signing Certificate openssl cms –sign –outform PEM Convert it to UTF-8 Convert CMS Signed Token in PEM format to custom URL Safe format: • “/” replaced with “-” • Deleted: “n”, “----BEGIN CMS----”,“----END CMS---- ” Compress using zlib Convert it to UTF-8 Base64 URL Safe Append Prefix PKIZ PKI PKIZ Store Token into SQL/KVS
  18. 18. Sample PKI Token in SQL Backend Deep Dive into Keystone Tokens and Lessons Learned id: b460fec2efcd0d803e2baf48d3bcd72b expires: 2015-10-09 20:07:36 extra: {"token_data": {"token": {"methods": ["password"], "roles": [{"id": "1688449cf1df44839b10a41e3d9b09dd", "name": "admin"}], "expires_at": "2015-10-09T20:07:36.656431Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "423d45cddec84170be365e0b31a1b15f", "name": "admin"}, "extras": {}, "user": {"domain": {"id": "default", "name": "Default"}, "id": "1334f3ed7eb2483b91b8192ba043b580", "name": "admin"}, "audit_ids": ["8dh07HudSh6rHoU1G9bs-Q"], "issued_at": "2015-10-09T19:07:36.656460Z"}}, "user": {"domain": {"id": "default", "name": "Default"}, "id": "1334f3ed7eb2483b91b8192ba043b580", "name": "admin"}, "key": "MIIDiwYJKoZIhvcNAQcCoIIDfDCCA3gCAQExDTALBglghkgBZQMEAgEwggHZBgkqhkiG9w0BBwGgggHKBIIBxnsidG9rZW4iOnsib WV0aG9kcyI6WyJwYXNzd29yZCJdLCJyb2xlcyI6W3siaWQiOiIxNjg4NDQ5Y2YxZGY0NDgzOWIxMGE0MWUzZDliMDlkZCIsIm5hb WUiOiJhZG1pbiJ9XSwiZXhwaXJlc19hdCI6IjIwMTUtMTAtMDlUMjA6MDc6MzYuNjU2NDMxWiIsInByb2plY3QiOnsiZG9tYWluIjp7I mlkIjoiZGVmYXVsdCIsIm5hbWUiOiJEZWZhdWx0In0sImlkIjoiNDIzZDQ1Y2RkZWM4NDE3MGJlMzY1ZTBiMzFhMWIxNWYiLCJuY W1lIjo…", "token_version": "v3.0", "tenant": {"domain": {"id": "default", "name": "Default"}, "id": "423d45cddec84170be365e0b31a1b15f", "name": "admin"}, "metadata": {"roles": ["1688449cf1df44839b10a41e3d9b09dd"]}} valid: 1 trust_id: NULL user_id: 1334f3ed7eb2483b91b8192ba043b580
  19. 19. Sample PKIZ Token in SQL Backend Deep Dive into Keystone Tokens and Lessons Learned id: c48321ac51a903b07c264ac3e80809c6 expires: 2015-10-12 18:45:23 extra: {"token_data": {"token": {"methods": ["password"], "roles": [{"id": "1688449cf1df44839b10a41e3d9b09dd", "name": "admin"}], "expires_at": "2015-10-12T18:45:23.806229Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "423d45cddec84170be365e0b31a1b15f", "name": "admin"}, "extras": {}, "user": {"domain": {"id": "default", "name": "Default"}, "id": "1334f3ed7eb2483b91b8192ba043b580", "name": "admin"}, "audit_ids": ["kKmQzTuxSnCN9vo3bzxErw"], "issued_at": "2015-10-12T17:45:23.806257Z"}}, "user": {"domain": {"id": "default", "name": "Default"}, "id": "1334f3ed7eb2483b91b8192ba043b580", "name": "admin"}, "key": "PKIZ_eJxtlMtyqzgQhvc8xexTqcPFdsLiLCQEWCSCgAGBdgZscbVxDOHy9CMnc6mpGlWpSmqpW39_Uuv5WTRo2tj9wy CHxiN35dqjqybi9eb6DuE7ZLd7_WxtAd6MtR1wP7PT5PxJE2F7U53WYH5D5qZbc53OSkeWPoo3hdrU7VQwhe5JBReo 71GWv72WT2vLPRk62_XuDmt_T9sZku-veT-xPfUaEk…", "token_version": "v3.0", "tenant": {"domain": {"id": "default", "name": "Default"}, "id": "423d45cddec84170be365e0b31a1b15f", "name": "admin"}, "metadata": {"roles": ["1688449cf1df44839b10a41e3d9b09dd"]}} valid: 1 trust_id: NULL user_id: 1334f3ed7eb2483b91b8192ba043b580
  20. 20. PKI/PKIZ – Token Validation Workflow Parse Token and Retrieve Metadata Validate Token with: GET v3/auth/tokens • X-Subject-Token • X-Auth-Token Retrieves Token reference from token backend KVS/SQL Read cached token reference and parse: • User ID • Project ID • Audit ID • Token Expiry Token KVS Valid? Current Time < Expiry Time Token Not Found Token Not Found Is Revoked? Token Not Found HTTP/1.1 200 OK Yes No No No Yes Yes Check if a token matches any revocation events Check if a token is expired, current time is calculated in UTC Token Validation Workflow Unique ID of X-Subject-Token Hash PKI Token with the pre- configured hashing algorithm
  21. 21. PKI/PKIZ – Keystone Token Revocation Workflow Revoke by Audit ID Revoke Token with: DELETE v3/auth/tokens • X-Subject-Token • X-Auth-Token Before revoking a token, validate it based on Token Validation Workflow Validate X-Subject-Token Audit ID? Revoke by Token Expiry Create Revoke Event with: Audit ID Revoke At Issued Before Prune Expired Events Set valid to False Yes No Filter existing revocation events based on Revoke At Sample Revocation Event: { "audit_id": "HVvI0d-cTD21yatAfQc4IQ", "issued_before”: "2015-10-24T21:20:45.000000Z" }, Token Revocation Workflow Create Revoke Event with: User ID Project ID Revoke At Issued Before Token Expiry Token KVS
  22. 22. PKI/PKIZ - Across Multiple Data Centers Users Groups Domains Projects Roles Catalog Assignments Users Groups Domains Projects Roles Catalog Assignments MySQL Replication (Database is always in sync) LDAP Replication (Directory Tree is always in sync) Tokens KVS PKI/PKIZ Tokens Tokens KVS PKI/PKIZ Tokens Keystone PKI/PKIZ - Multiple Data Centers Nova Keystone Middleware Keystone Nova Keystone Middleware US-EASTUS-WEST Request Token PKI/PKIZ Token Token Validation VM Instance Token Validation VM Instance
  23. 23. Pros and Cons PKI • Pros – Token validation without a request to Keystone • Cons – Larger than standard HTTP Header Size – Complex configuration – base64 –d <pki_token – Not truly feasible for multiple OpenStack Deployments PKIZ • Pros – Token validation without a request to Keystone • Cons – Still Larger than standard HTTP Header Size – Similar to PKI Deep Dive into Keystone Tokens and Lessons Learned
  24. 24. FERNET Deep Dive into Keystone Tokens and Lessons Learned
  25. 25. Fernet • Cryptographic Authentication Method – Fernet • Symmetric Key Encryption • Fernet Keys stored in /etc/keystone/fernet-keys/ – Encrypted with Primary Fernet Key – Decrypted with a list of Fernet Keys Deep Dive into Keystone Tokens and Lessons Learned
  26. 26. • Configuration in keystone.conf : [token] provider = keystone.token.providers.fernet.Provider [fernet_tokens] key_repository = /etc/keystone/fernet-keys/ max_active_keys = <number of keys> # default is 3 Deep Dive into Keystone Tokens and Lessons Learned Fernet Configuration
  27. 27. Fernet Keys • Fernet Key File - 256 bits 83b4sCF0Q4pb3aNWJYtSdtdaH8PMA_5dlN7OswXKbvE= xf3vxf8xb0!tCx8a[xddxa3V%x8bRvxd7Zx1fxc3xccx03xfe]x94xdexcexb3x05xcanxf1 Deep Dive into Keystone Tokens and Lessons Learned SHA256 HMAC Signing Key (128 bits) AES Encrypting Key (128 bits)
  28. 28. Fernet Keys • Fernet Key File Name - Integers starting from 0 • ls /etc/keystone/fernet-keys => 0 1 2 3 4 • Type 1: Primary Key – Encrypt and Decrypt – Key file named with the highest index • Type 2: Secondary Key – Only Decrypt – Lowest Index < Secondary Key File Name < Highest Index • Type 3: Staged Key – Decrypt and Next In Line to become Primary Key – Key file named with lowest index (of 0) Deep Dive into Keystone Tokens and Lessons Learned
  29. 29. Fernet Key Rotation 0 1 Primary KeyStaged Key No Secondary Key 2 Primary Key0Staged Key 1 Secondary Key 3 Primary Key21 Secondary Key 0Staged Key Secondary Key Rotate Rotate
  30. 30. Fernet – Token Generation Workflow Token Generation Workflow HMACFernet Token Version Current Timestamp IV Cipher Text Token Payload: Version User ID Methods Project ID Expiry Time Audit ID Padding Encrypted using Encrypting Key Signed using Signing Key
  31. 31. Sample Fernet Token in SQL Backend Deep Dive into Keystone Tokens and Lessons Learned gAAAAABWLUzy0dxSNo2--K- 3trDutnX7LpUpv3us0crQIl8BDHLLd3lR3F243VwnYpNJHIaUiPEE2roYJJNA- SwBe1swDcr6MYaFR1t9ZYcYF4GRqDm3N9_1EGgXgICbzE_GuUVidG4gky0Cv8 f1nwD7XM26NRh59VEnt2iVTAxlnvAICJDeK5k
  32. 32. Fernet – Keystone Token Validation Workflow Determine the Version from the Token Payload Validate Token with: GET v3/auth/tokens • X-Subject-Token • X-Auth-Token Re-inflate token with “=” and return token with correct padding Version: Fixed Versioning by Keystone: • Unscoped Payload : 0 • Domain Scoped Payload : 1 • Project Scoped Payload : 2 Restore Padding Is Revoked? Token Not Found HTTP/1.1 200 OK No No Yes Yes Check if a token matches any revocation events Check if a token is expired, current time is calculated in UTC Token Validation Workflow Decrypt using Fernet Keys to retrieve Token Payload Disassemble payload to determine validation fields For Project Scoped Token: • User ID Project ID • Methods Token Expiry • Audit ID Current Time < Expiry Time Token Not Found No
  33. 33. Fernet – Keystone Token Revocation Workflow Revoke by Audit ID Revoke Token with: DELETE v3/auth/tokens • X-Subject-Token • X-Auth-Token Before revoking a token, validate it based on Token Validation Workflow Validate X-Subject-Token Audit ID? Revoke by Token Expiry Create Revoke Event with: Audit ID Revoke At Issued Before Prune Expired Events Set valid to False Yes No Filter existing revocation events based on Revoke At Sample Revocation Event: { "audit_id": "HVvI0d-cTD21yatAfQc4IQ", "issued_before”: "2015-10-24T21:20:45.000000Z" }, Token Revocation Workflow Create Revoke Event with: User ID Project ID Revoke At Issued Before Token Expiry Token KVS
  34. 34. Fernet - Across Multiple Data Centers Users Groups Domains Projects Roles Catalog Assignments Users Groups Domains Projects Roles Catalog Assignments MySQL Replication (Database is always in sync) LDAP Replication (Directory Tree is always in sync) Keystone Fernet - Multiple Data Centers Nova Keystone Middleware Keystone Nova Keystone Middleware US-EASTUS-WEST Request Token Fernet Token Token Validation VM Instance Token Validation VM Instance Validate Fernet Token Validate Fernet Token
  35. 35. Pros and cons • Pros – No persistence – Reasonable Token Size – Multiple Data Center • Cons – Token validation impacted by the number of revocation events Deep Dive into Keystone Tokens and Lessons Learned
  36. 36. Fernet Token Validation 11.17 46.406 83.654 124.974 163.529 234.398 376.604 510.058 0 100 200 300 400 500 600 0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 2000 3000 4000 Time(ms) Revocation Events Revocation Events Impact on Validation Time Response Time (ms) Deep Dive into Keystone Tokens and Lessons Learned
  37. 37. Fernet Token Validation 89.46 21.55 11.95 8 5.77 1.96 0 10 20 30 40 50 60 70 80 90 100 0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 2000 3000 4000 ValidationsPerSecond Revocation Events Revocation Events Impact on Validation Requests Token Validation Requests Deep Dive into Keystone Tokens and Lessons Learned
  38. 38. What token format should we configure in our OpenStack Deployment? Fernet for Multiple OpenStack Deployments with minimal Revocation Events Deep Dive into Keystone Tokens and Lessons Learned
  39. 39. HORIZON AND TOKENS Deep Dive into Keystone Tokens and Lessons Learned
  40. 40. How horizon uses tokens Deep Dive into Keystone Tokens and Lessons Learned • Tokens for each logged in user • Unscoped token and project scoped token • Token reuse • Reduced transaction load on Keystone • Stored in the session • Configurable token storage methods • Local memory cache • Cookie backend • Memcache • Database • Cached Database
  41. 41. Cookie backend Deep Dive into Keystone Tokens and Lessons Learned • Currently the devstack default • Token stored in browser cookie • Secure cookies in production, use https • CSRF_COOKIE_SECURE = True • SESSION_COOKIE_SECURE = True • http://docs.openstack.org/developer/horizon/topics/settings.html • http://docs.openstack.org/security-guide/dashboard/cookies.html • Highly scalable • The dreaded boot back to login
  42. 42. Cookie backend Deep Dive into Keystone Tokens and Lessons Learned • The dreaded boot back to login • Now sign in…
  43. 43. Cookie backend Deep Dive into Keystone Tokens and Lessons Learned • And you see… • Cookie overflow!
  44. 44. memcache backend Deep Dive into Keystone Tokens and Lessons Learned • Allows storage of larger token sizes • Tokens stored on server side • Requires memcached • Can be used with backing DB • http://docs.openstack.org/developer/horizon/topics/deployment.html
  45. 45. Token hashing Deep Dive into Keystone Tokens and Lessons Learned • Hashed in Django OpenStack Auth (DOA) • Keeps stored token data small • Currently not working for PKI tokens • New config in Liberty to disable • OPENSTACK_TOKEN_HASH_ENABLED • PKI - Will increase memcache storage requirement
  46. 46. Multiregion and tokens Deep Dive into Keystone Tokens and Lessons Learned • Service regions vs. Authentication regions • Service regions in Keystone catalog • Auth regions specified in AVAILABLE_REGIONS • UUID, PKI, and PKIZ Tokens don’t work across auth regions • Token replication is infeasible • But Fernet tokens work between Authentication regions! Service Region Authentication Region
  47. 47. Horizon and Fernet Deep Dive into Keystone Tokens and Lessons Learned • Yes, Fernet tokens work with Horizon • Liberty and beyond – No patches necessary • Kilo – Needs a patch for DOA • https://review.openstack.org/#/c/169994/
  48. 48. V3 domains Domain Scoped Token Project Scoped Token Deep Dive into Keystone Tokens and Lessons Learned "auth": { "identity": { }, “scope”: { ”domain": { “name”: “Default” } } } "auth": { "identity": { }, “scope”: { “project”: { ”domain": { “name”: “Default” }, “name”: “ProjectA” } } } • Extra token for Horizon
  49. 49. V3 domains Deep Dive into Keystone Tokens and Lessons Learned • Requires changes in Django OpenStack Auth and Horizon • Planned for Mitaka • Info on usage (a bit out of date): • http://www.symantec.com/connect/blogs/how-use-horizon-keystone-v3 • Domains patches: • https://review.openstack.org/#/c/148082/ • https://review.openstack.org/#/c/141153/ • https://review.openstack.org/#/c/196328/
  50. 50. Will fernet tokens solve all our problems? Deep Dive into Keystone Tokens and Lessons Learned • Smaller token size • No persistence for tokens • Seamless authentication across regions • Performance issues with token revocation
  51. 51. Thank You !!! Questions ?
  52. 52. References • Token: https://clubpenguincheatscitya4.files.wordpress.com/2011/08/1_token.jpg • Key to Cloud: https://www.hc1.com/wp- content/uploads/2013/10/14916002_cloud_computing_and_storage_security_concept_blue_glossy_cloud_icon_with_with_golden_key_in_keyhole _.jpg • User Icon: http://findicons.com/files/icons/1075/scrap/300/user_2.png • Password: http://icons.iconarchive.com/icons/sbstnblnd/plateau/512/Apps-password-icon.png • UUID: https://plugins.qgis.org/static/cache/21/c0/21c0d3fedb5bf42ff8a6a11712595124.png • PKI: http://www.zaheerspeaks.com/wp-content/uploads/2009/10/PKI-Certificate.gif • PKIZ: http://i571.photobucket.com/albums/ss153/rijal_abror/pun170-winzip-file-compress-icon59.gif • Identity: https://www.innopay.com/assets/Uploads/icon-digitalidentity-232x232.png Deep Dive into Keystone Tokens and Lessons Learned
  53. 53. EXTRA Deep Dive into Keystone Tokens and Lessons Learned
  54. 54. What is an Openstack Token? Deep Dive into Keystone Tokens and Lessons Learned Key to OpenStack Cloud
  55. 55. How can I generate a token? Deep Dive into Keystone Tokens and Lessons Learned Keystone curl -s POST https://keystone.com/v3/auth/tokens "auth": { "identity": { "methods": [ "password" ], "password": { "user": { ”domain": { ”name": ”MyDomain.com” }, ”name": ”PritiDesai", "password": "secretsecret” }
  56. 56. How can I generate a token? Deep Dive into Keystone Tokens and Lessons Learned Keystone curl -s POST https://keystone.com/v3/auth/tokens "auth": { "identity": { "methods": [ ”token" ], ”token": { ”id": ”e8079ab…” } }
  57. 57. Token generated using password Deep Dive into Keystone Tokens and Lessons Learned Header: X-Subject-Token: a740dcd6f3fc404aaaf556b9cbd2f994 Body: { "token": { "methods": [ "password" ], "expires_at": "2015-10-05T20:25:03.180741Z", "extras": {}, "user": { "domain": { "id": "default", "name": "Default" }, "id": "1334f3ed7eb2483b91b8192ba043b580", "name": ”smith" }, "auth": { "identity": { "methods": [ "password" ], "password": { "user": { ”domain": { ”name": ”Default” }, ”name": ”Smith", "password": "secretsecret” }
  58. 58. Token generated using token Deep Dive into Keystone Tokens and Lessons Learned Header: X-Subject-Token: 3fb7b3b0a0a8489882f07fdb9cd2a990 Body: { "token": { "issued_at": "2015-10-05T19:40:38.943250Z", "audit_ids": [ "4vNgmP5cQk6sMpPiw7EnCg", "HFwMKdDrSCOq-MAtkXKTlw" ], "user": { "name": ”smith", "id": "1334f3ed7eb2483b91b8192ba043b580", "domain": { "name": "Default", "id": "default" } }, "extras": {}, "auth": { "identity": { "methods": [ ”token" ], ”token": { “id”: “a7409b” } } }
  59. 59. Identity Token Path User OpenStack Service Step 1: Obtain unscoped token with credentials POST v3/auth/tokens Keystone Step 2: Discover projects you have access to GET v3/users/<user_id>/projects Step 3: Obtain project scoped token either with your credentials or unscoped token from step 1. Step 4: Invoke the target service by sending requests to endpoints in token from step 3 Step 5: Validate roles and access metadata in token with Keystone service or Keystone Middleware Step 6: Serve API request Step 7: Return response
  60. 60. Token Creation 0 10 20 30 40 50 60 70 80 90 UUID PKI PKIZ Fernet Time Per Request Time Per Request Deep Dive into Keystone Tokens and Lessons Learned 0 5 10 15 20 UUID PKI PKIZ Fernet Requests Per Sec Requests Per Sec
  61. 61. Token Validation 0 2 4 6 8 10 12 14 UUID PKI PKIZ Fernet Time Per Request Time Per Request Deep Dive into Keystone Tokens and Lessons Learned 0 20 40 60 80 100 UUID PKI PKIZ Fernet Requests Per Sec Requests Per Sec
  62. 62. Configurable token hashing https://github.com/openstack/django_openstack_auth/blob/master/openstack_auth/user.py Deep Dive into Keystone Tokens and Lessons Learned

×