Презентация с выступления Артема Гавриченкова, CTO Qrator Labs, на конференции для разработчиков высоконагруженных систем Highload++ 2016

1. Масштабируя TLS
Артём Гавриченков <ximaera@qrator.net>

2. Краткая история нового времени
• 2010: SPDY w/de-facto mandatory* SSL/TLS
• 2014: “HTTPS as a ranking signal” at Google
• 2015: HTTP/2 w/de-facto mandatory* TLS
• 2016: Let’s Encrypt
* – https://forum.nginx.org/read.php?21,236132,236184
*– https://daniel.haxx.se/blog/2015/03/06/tls-in-http2/

5. Краткая история нового времени
• 2010: SPDY w/de-facto mandatory* SSL/TLS
• 2014: “HTTPS as a ranking signal” at Google
• 2015: HTTP/2 w/de-facto mandatory* TLS
• 2016: Let’s Encrypt
* – https://forum.nginx.org/read.php?21,236132,236184
*– https://daniel.haxx.se/blog/2015/03/06/tls-in-http2/

6. Краткая история нового времени
• 2010: SPDY w/de-facto mandatory* SSL/TLS
• 2013: NSA story
• 2014: “HTTPS as a ranking signal” at Google
• 2014:
• 2015: HTTP/2 w/de-facto mandatory* TLS
• 2015:
• 2016: Let’s Encrypt
• 2016:
* – https://forum.nginx.org/read.php?21,236132,236184
*– https://daniel.haxx.se/blog/2015/03/06/tls-in-http2/

7. Краткая история нового времени
• 2010: SPDY w/de-facto mandatory* SSL/TLS
• 2013: NSA story
• 2014: “HTTPS as a ranking signal” at Google
• 2014: Heartbleed, POODLE
• 2015: HTTP/2 w/de-facto mandatory* TLS
• 2015: RFC 7457
• 2016: Let’s Encrypt
* – https://forum.nginx.org/read.php?21,236132,236184
*– https://daniel.haxx.se/blog/2015/03/06/tls-in-http2/

9. Краткая история нового времени
• 2010: SPDY w/de-facto mandatory* SSL/TLS
• 2013: NSA story
• 2014: “HTTPS as a ranking signal” at Google
• 2014: Heartbleed, POODLE
• 2015: HTTP/2 w/de-facto mandatory* TLS
• 2015: RFC 7457
• 2016: Let’s Encrypt
* – https://forum.nginx.org/read.php?21,236132,236184
*– https://daniel.haxx.se/blog/2015/03/06/tls-in-http2/

10. Краткая история нового времени
• 2010: SPDY w/de-facto mandatory* SSL/TLS
• 2013: NSA story
• 2014: “HTTPS as a ranking signal” at Google
• 2014: Heartbleed, POODLE
• 2015: HTTP/2 w/de-facto mandatory* TLS
• 2015: RFC 7457, FREAK, Logjam
• 2016: Let’s Encrypt
• 2016: DROWN
* – https://forum.nginx.org/read.php?21,236132,236184
*– https://daniel.haxx.se/blog/2015/03/06/tls-in-http2/

11. SSL/TLS PKI
• Root certificate authorities, trust chain

12. SSL/TLS PKI
• Root certificate authorities, trust chain
• 92 CAs in Firefox

13. SSL/TLS PKI
• Root certificate authorities, trust chain
• Trusted, because they make it for living
• Independent from large corporations, government, etc.

14. SSL/TLS PKI
• Root certificate authorities, trust chain
• Trusted, because they make it for living
• Independent from large corporations, government, etc.
Except, some of them ARE government

15. SSL/TLS PKI
• Root certificate authorities, trust chain
• Trusted, because they make it for living
• Independent from large corporations, government, etc.
And some of them are large corporations
Except, some of them ARE government

16. SSL/TLS PKI
• Root certificate authorities, trust chain
• Trusted, because they make it for living
• Independent from large corporations, government, etc.
• Pursuing their interests as trusted third parties

17. SSL/TLS PKI
• Root certificate authorities, trust chain
• Trusted, because they make it for living
• Independent from large corporations, government, etc.
• Pursuing their interests as trusted third parties
• Corporations and government always tend to elevate their own interests

18. The story of WoSign
• Trusted since 2009
• Aggressive marketing and free certificates
• Passed audit by Ernst&Young

19. The story of WoSign
https://wiki.mozilla.org/CA:WoSign_Issues
• Issued certificates not requested by domain owner

20. The story of WoSign
https://wiki.mozilla.org/CA:WoSign_Issues
• Issued certificates not requested by domain owner
• Allowed using non-privileged ports (>50,000) to verify domain control

21. The story of WoSign
https://wiki.mozilla.org/CA:WoSign_Issues
• Issued certificates not requested by domain owner
• Allowed using non-privileged ports (>50,000) to verify domain control
• Allowed using subdomains to verify 2nd level domain

22. The story of WoSign
https://wiki.mozilla.org/CA:WoSign_Issues
• Issued certificates not requested by domain owner
• Allowed using non-privileged ports (>50,000) to verify domain control
• Allowed using subdomains to verify 2nd level domain
• Allowed using arbitrary files to verify ownership

23. The story of WoSign
https://wiki.mozilla.org/CA:WoSign_Issues
• Issued certificates not requested by domain owner
• Allowed using non-privileged ports (>50,000) to verify domain control
• Allowed using subdomains to verify 2nd level domain
• Allowed using arbitrary files to verify ownership
• Allowed to issue certificates for arbitrary domains without verification

24. The story of WoSign
https://wiki.mozilla.org/CA:WoSign_Issues
• Issued certificates not requested by domain owner
• Allowed using non-privileged ports (>50,000) to verify domain control
• Allowed using subdomains to verify 2nd level domain
• Allowed using arbitrary files to verify ownership
• Allowed to issue certificates for arbitrary domains without verification
• Issued backdated SHA-1 certificates

25. The story of WoSign
https://wiki.mozilla.org/CA:WoSign_Issues
• Issued certificates not requested by domain owner
• Allowed using non-privileged ports (>50,000) to verify domain control
• Allowed using subdomains to verify 2nd level domain
• Allowed using arbitrary files to verify ownership
• Allowed to issue certificates for arbitrary domains without verification
• Issued backdated SHA-1 certificates
• Used unpatched software (such as dig) on the validation server

26. The story of WoSign
https://wiki.mozilla.org/CA:WoSign_Issues
• Issued certificates not requested by domain owner
• Allowed using non-privileged ports (>50,000) to verify domain control
• Allowed using subdomains to verify 2nd level domain
• Allowed using arbitrary files to verify ownership
• Allowed to issue certificates for arbitrary domains without verification
• Issued backdated SHA-1 certificates
• Used unpatched software (such as dig) on the validation server
• Purchased other CA (StartCom) and attempted to suppress
information about the ownership transfer

27. The story of WoSign
The aftermath?

28. The story of WoSign
The aftermath?
• Banned by Google in Chrome
• Banned by Mozilla for a year

29. The story of WoSign
The aftermath?
• Banned by Google in Chrome
• Banned by Mozilla for a year
• Still trusted by Microsoft
and lots of unpatched equipment

30. Aftermath
• Go and choose the cheapest CA available
• Bonus points if it provides some kind of API

31. Aftermath
• Go and choose the cheapest CA available
• Bonus points if it provides some kind of API
• Pick multiple CAs

32. Aftermath
• Go and choose the cheapest CA available
• Bonus points if it provides some kind of API
• Pick multiple CAs
• “Extended validity” certificates?

33. Aftermath
• Go and choose the cheapest CA available
• Bonus points if it provides some kind of API
• Pick multiple CAs
• “Extended validity” certificates are a security theater
(don’t bother if you are not a bank and auditor doesn’t force you to)

34. Aftermath
• Go and choose the cheapest CA available
• Bonus points if it provides some kind of API
• Pick multiple CAs
• “Extended validity” certificates are a security theater
(don’t bother if you are not a bank and auditor doesn’t force you to)
• Prefer short-lived certificates

35. Long-living certificates?
Pros:
• Discount
• Less pain in the #^$ updating all the certs

36. Long-living certificates?
Pros:
• Discount
• Less pain in the #^$ updating all the certs
Cons:
• Soft-fail CRL and OCSP are not reliable
• Hard-fail CRL and OCSP are never used
(you may do it in your app though)
• Certificate deployment and management must be automated anyway

37. Long-living certificates?
• CRL and OCSP are not reliable
• Certificate deployment and management must be automated
Long-lived cert is a technical debt. It wouldn’t punish you immediately.
It will hurt you eventually.

38. Automated certificate management
• Add, remove, change and revoke your certificates real quick
• Manage certificates properly: short lifetime, multiple keys
• Set up a clientside TLS auth

39. Automated certificate management
• Add, remove, change and revoke your certificates real quick
• Manage certificates properly: short lifetime, multiple keys
• Set up a clientside TLS auth
• Quickly work around obscure issues like “Intermediate CA was
revoked”

40. The story of GlobalSign
• During a planned maintenance, accidentally revoked its own certificate
• Used CDN (Cloudflare) for CRL and OCSP
• Undid revocation, but it’s got cached on CDN

41. The story of GlobalSign
• During a planned maintenance, accidentally revoked its own certificate
• Used CDN (Cloudflare) for CRL and OCSP
• Undid revocation, but it’s got cached on CDN
• Four days before cached response will expire in a browser
• Wikipedia, Dropbox, Spotify, Financial Times affected
• Large sites affected more because CRL got cached everywhere
immediately

42. The story of GlobalSign
• Large sites affected more because CRL got cached everywhere
immediately
• “All is good and yet traffic dropped by 30%”
• Really hard to troubleshoot
• The issue is of distributed nature
• You depend on a vendor

43. The story of GlobalSign
• Large sites affected more because CRL got cached everywhere
immediately
• “All is good and yet traffic dropped by 30%”
• Really hard to troubleshoot
• The issue is of distributed nature
• You depend on a vendor
• Multiple different certs from different vendors helped to track down
• tcpdump also of a great help: sessions got stuck at TLS Server Hello

44. The story of GlobalSign
• Really hard to troubleshoot
• The issue is of distributed nature
• You depend on a vendor
• Multiple different certs from different vendors will help to track down
• tcpdump also of a great help: sessions got stuck at TLS Server Hello
TLS is still bleeding edge of technology.
Unsufficient tools, unsufficient knowledge.

45. The story of GlobalSign
• Really hard to troubleshoot
• So, hours wasted before the root cause is found
• The fix must be immediate => cert management automation!

46. Automated certificate management
• CA with API

47. Automated certificate management
• CA with API
• Let’s Encrypt?

48. Automated certificate management
• CA with API
• Let’s Encrypt?
Very good if you don’t need wildcard certificates.

49. Automated certificate management
• CA with API
• Let’s Encrypt?
Very good if you don’t need wildcard certificates.
• Tools like SSLMate
• In-house plugins for ansible etc.

50. What to set up during the deployment?

51. What to set up during the deployment?
• Strict Transport Security
• “Opportunistic encryption” simply doesn’t work
• Most users won’t notice if HTTPS is absent
• HTTPS only makes sense if it’s enforced

52. What to set up during the deployment?
• Strict Transport Security
• “Opportunistic encryption” simply doesn’t work
• Most users won’t notice if HTTPS is absent
• HTTPS only makes sense if it’s enforced
• Public Key Pinning
• Pin all end-entity public keys
• Create a backup
• Include future leafs
• Rotate often => use automated tools to generate the header

53. What to set up during the deployment?
• Ciphers
• https://wiki.mozilla.org/Security/TLS_Configurations

54. What to set up during the deployment?
• Ciphers
• https://wiki.mozilla.org/Security/TLS_Configurations outdated
• https://mozilla.github.io/server-side-tls/ssl-config-generator/
• Update frequently (automation?)

55. What to set up during the deployment?
• Ciphers
• https://wiki.mozilla.org/Security/TLS_Configurations outdated
• https://mozilla.github.io/server-side-tls/ssl-config-generator/
• Update frequently (automation?)

56. The story of Rijndael

57. The story of Rijndael
(finally it sounds almost like Tolkien)

58. The story of Rijndael/AES
• Ordered by U.S. federal government
• Approved by NSA, 1998-2001
• Adopted by U.S. DoD and Army

59. The story of Rijndael/AES
• Adopted by U.S. DoD and Army
• Military required three distinct security levels,
with less sensitive data to be encrypted using the most weak method
and vice versa

60. The story of Rijndael/AES
• Adopted by U.S. DoD and Army
• Military required three distinct security levels,
with less sensitive data to be encrypted using the most weak method
and vice versa
• Crypto designers implemented three key sizes (128, 192, 256),
with the most weak still unbreakable in foreseeable future
(except quantum computers)

61. The story of Rijndael/AES
• Adopted by U.S. DoD and Army
• Military required three distinct security levels,
with less sensitive data to be encrypted using the most weak method
and vice versa
• Crypto designers implemented three key sizes (128, 192, 256),
with the most weak still unbreakable in foreseeable future
(except quantum computers)
• So, AES-128 is still good enough
• Not that it matters much with modern AES-NI

62. The story of Perfect Forward Secrecy
• Present in ephemeral Diffie-Hellman ciphers

63. The story of Perfect Forward Secrecy
• Present in ephemeral Diffie-Hellman ciphers
• Makes out-of-path analysis impossible
• Makes historic data analysis impossible

64. The story of Perfect Forward Secrecy
• Present in ephemeral Diffie-Hellman ciphers
• Makes out-of-path analysis impossible
• Makes historic data analysis impossible
• Good catch for an out-of-path DPI and/or WAF
70% HTTPS requests come and go without analysis

65. • Present in ephemeral Diffie-Hellman ciphers
• Makes out-of-path analysis impossible
• Makes historic data analysis impossible
• Good catch for an out-of-path DPI and/or WAF
70% HTTP requests go without analysis
The story of Perfect Forward Secrecy
60% legitimate
90% malicious

66. Protocols

67. Protocols
• SSLv2 is dead

68. Protocols
• SSLv2 is dead
• SSLv3 is dead*
• TLSv1.0 is dead
* – if you don’t have to serve content to IE6 or a TV set

69. Protocols
• SSLv2 is dead
• SSLv3 is dead*
• TLSv1.0 is dead
• TLS is alive and growing
* – if you don’t have to serve content to IE6 or a TV set

70. Protocols
• SSLv2 is dead
• SSLv3 is dead*
• TLSv1.0 is dead
• TLS is alive and growing
• Maybe too fast: TLSv1.2 allowed DDoSCoin
* – if you don’t have to serve content to IE6 or a TV set

71. Misc
• OCSP stapling
• Persistent connections (TLS handshake is expensive)
• Fight unencrypted content!

72. Sound Bytes
• Use short-lived certificates!
• Automate!
• Trust Mozilla! :-)

73. Q&A
mailto: ximaera@qrator.net

74. Bonus track
• Client certificates

75. Bonus track
• Client certificates
• May be combined with 2FA

76. Bonus track
• Client certificates
• May be combined with 2FA
• May be integrated into certain applications as well
• Unsupported by some mobile browsers OOTB :-(