Launched in June of 2015, s2n is an AWS open-source implementation of the TLS and SSL network security protocols, which focus on security, simplicity, and performance. With development led by engineers from Amazon EC2, Amazon S3, Amazon CloudFront, and AWS security and cryptography services, s2n is a unique opportunity to observe how we develop and test security and availability for critical software at AWS. Learn how we iterate and code, how we automate software verification beyond the usual code reviews, and how open source works at Amazon.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Colm MacCárthaigh
December 1, 2016
NET405
Amazon s2n: Cryptography
and Open Source at AWS

3. What to expect from the session
Why s2n? A brief look at the OpenSSL Heartbleed bug
Open source at AWS: How we started s2n
How we develop and test s2n
Where s2n is and our future plans

4. OpenSSL

5. Heartbleed
7
April
2014

6. Heartbleed

7. What was the Heartbleed bug? TLS explained
Application protocol (HTTP, SMTP, SQL … )
SSL/TLS
TCP/IP

8. What was the Heartbleed bug? TLS record format
Message
type
Protocol
version
Record
length
Data

9. What was the Heartbleed bug? The Heartbeat
extension
Heartbeat: RFC6520
Message
type
Protocol
version
Record
length
Padding
Heartbeat
message
type
Payload
length Payload

10. What was the Heartbleed bug?

11. What was the Heartbleed bug? Heartbeat
extension format
Message
type
Protocol
version
Record
length
Padding
24 3.2 22
Heartbeat
message
type
Payload
length Payload
Padding1 3 0xBADBAD

12. What was the Heartbleed bug? Heartbeat
extension format
24 3.2 22 Padding1 0xBADBAD65535
Message
type
Protocol
version
Record
length
Padding
Heartbeat
message
type
Payload
length Payload

13. What was the Heartbleed bug?

14. What was the Heartbleed bug? The code

16. Contributing factors
No good test cases
Code is handling raw buffers with no systematic defense
against overflows
A lot of code in one file with a high cognitive load
Needless functionality included by default

17. Two lines of code

18. Heartbleed: The response

19. Heartbleed: The Amazon response
All services upgraded, without additional customer impact,
within 16 hours of the Heartbleed issue
Monitoring and notification process set up to detect
impacted customer instances
Customers do a great job: 80% updated within 5 days, 98%
within one month

20. Heartbleed: The response
The Core Infrastructure Initiative founded
New research into vulnerability detection
LibreSSL project is started; iterates rapidly
BoringSSL project begins

21. The response: s2n

22. The response: s2n
Core of s2n first came together within about 40 hours of
development time over 5 weekends. First working
connection at hour 30.
Ben Laurie: “The really fun part of working on cryptography
is spending hours staring at big opaque numbers and trying
to figure out why they aren’t different big opaque numbers”.

23. The response: s2n
Proposal to make s2n an official AWS project: Start with a
“working backward” document.
Initial feedback from customers: Sounds great! Make sure
it’s open source.
Initial feedback from AWS leadership: Sounds great; need
to define long-term ownership and security process.

24. The response: s2n
Path to open source: working with Amazon Open Source
team, Amazon Legal. Very friendly, extremely helpful, and
smooth process.
Path to ownership: Several teams interested in
participating. Define a “v-team” for s2n. Roadmap, on-call
rotation, long term alignment.
Path to a security process: Work with CISO and CEO.

25. How we develop and test s2n

26. Things to focus on
1. Team culture
2. Simple designs, readable code
3. Quick, easy, habitual testing
4. Systematic defense in depth
5. Verification of everything

27. Team culture

28. Starting s2n
Agree and document goals, priorities, and tenets
Agree and document our coding guidelines
Make testing and safety easy and fast
Think hard about the high-level design

29. s2n’s initial goals
Build and maintain a world-class TLS implementation
Raise the bar for testing, secure coding practices, and
defense in depth
Stay minimal, simple with safe defaults
Write great code but stay humble and cautious

30. s2n’s development principles
https://github.com/awslabs/s2n/blob/master/docs/DEVELOPMENT-GUIDE.md

31. Simple designs, readable code

32. s2n coding guidelines
Focused on clean and consistent, readable code
Include verbose comments to provide context and explain
why we’re doing something; code itself should be self-
documenting about what we’re doing
Automate the code formatting

33. Example code

34. Simple designs, readable code
Readable code > Less code > More code
Avoid branches as much as possible
Small, simple, discrete functions
Split message parsing and flow control

35. Designing for safety: The stuffer
λInput
λ λ Output
Decrypt Parse/
respond
Encrypt

36. Designing for safety: the Stuffer

37. Nested branches

38. Nested branches

39. Straight line

40. Straight line

41. Quick, easy, and habitual
testing

42. Quick, easy, and habitual testing
Make tests easy enough for interns to write in their first
week
Testing is a habit, like brushing your teeth
Make running tests fast
Automate the end-to-end testing

43. Make tests fast

44. Make tests easy to write

45. Example code

46. Systematic defense in depth

47. Designing for safety

48. Designing for safety

49. Stream I/O: The stuffer
Start EndRead cursor Write cursor
data_available() space_remaining()

50. Designing for safety: The stuffer

51. Designing for safety: The stuffer

52. Verification

53. s2n verification
Six formal code audits; government and commercial
Automated formal verification of undefined behavior, HMAC
correctness, RNG correctness, core state machine
Automated fuzz testing with AFL and libFuzzer
In-progress work on automated side-channel verification

54. s2n verification

55. Where is s2n today

56. s2n today
Still focused on the server side; beginning now on client
side
No support for many features, including client certificates
(now in progress)
SSLv3, 3DES disabled in response to published research

57. s2n today
Seven active Amazon contributors; bus-factor: 0
Contributions from 11 non-Amazonians
Used in production
~2.5% performance improvement

58. Key takeaways
Write goals, development principles, code guidelines
Value clear and concise, readable code
Make testing easy, fast, and habitual
Formal verification is getting easier quickly

59. Thank you!
NET405

60. Remember to complete
your evaluations!

61. Related sessions
SEC401 - Automated Formal Reasoning About AWS
Systems
SAC306 - Encryption: It Was the Best of Controls, It
Was the Worst of Controls