AWS Summit London 2014 | Dynamic Content Acceleration (300)

1. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. Dynamic Content Acceleration: Lightning-Fast Web Apps with Amazon CloudFront and Amazon Route 53 Glyn Smith, Business Development Manager April, 2014

2. Fundamental Facts Any web application must have… •  Tight Security •  High Availability •  High Performance

3. Why Does Availability Matter? •  If your application is not available, your revenue loss is 100%. •  Impact to customer loyalty and your brand image.

4. How AWS Helps? ü  Use Amazon Route 53 to health-check your origin webservers, with automatic failover. ü  Use Amazon CloudFront to front your origins to reduce load on your origins. ü  Amazon CloudFront will automatically serve stale content when origin is unavailable. ü  Use Amazon CloudFront to customize your error pages.

5. Why Performance Matters? •  Performance translates to … •  Higher Page Views •  Better Customer Experience •  Higher Conversion Rates

6. Why Performance Matters?

7. Why Performance Matters? Great amount of time and money spent improving back-end infrastructure performance

8. How do we Improve Performance ? A Typical Web Application Has … •  Static or Re-Usable Content •  High TTLs •  Low TTLs (Customized Content) •  Dynamic or Unique Content •  Zero TTL

9. Legacy Architecture

10. Why Not…?

11. Why Don’t Customers Use a CDN for Dynamic Content? I don’t see the value - each request is unique and must go back to the origin web server. AND/OR I see the value, but my current CDN charges premium rates for dynamic content acceleration, with many additional fees. AND/OR Configuring a CDN for dynamic content acceleration requires expensive professional services and is not self-service.

12. How Can Amazon CloudFront Help? ü Caching Static content ü Improving Dynamic – DNS, Keep-Alive Connections, First Byte and Content Download ü SSL Termination close to viewers ü Latency Based Routing ü Design for Failure ü Low prices, same as static content delivery!

13. STATIC or REUSABLE A given content where the state of the content does NOT change for a given period of time t0 t1

14. DYNAMIC OR UNIQUE A given content where the state of the content changes as soon as it gets created t0 t1

15. Example Index.jsp (dynamic) Images (static)

16. Example . sec

17. Page Load Time? . Sec

18. Brief Introduction to Waterfall Graphs

19. Waterfall Graphs •  Most important tool for web-performance measurement •  Most browsers provide waterfall graph plug-ins

20. Understanding Waterfall Graph DNS Lookup TCP Connection Time to First Byte Content Download

21. Understanding Waterfall Graph Index.jsp

22. Optimizing Static Contents Index.jsp Optimize By Caching With Amazon CloudFront

23. Caching Origin Edge Location User Request A

24. Caching Origin Edge Location Get Image User Request A

25. Caching Origin Edge Location Get Image Get Image User Request A

26. Caching Origin Edge Location Get Image Get Image Image User Request A

27. Caching Origin Edge Location Get Image Get Image Image Image User Request A

28. Caching Origin Edge Location Get Image Get Image Image Image User Request A Get ImageImage User Request B

29. Optimizing Static Contents with Caching Brings content closer to the users Improves the experience and performance

30. Optimizing Static Contents with Caching Offloads your infrastructure

31. Before Caching = 1.46sec

32. After Caching = 770ms

33. Cache As Much As You Can

34. STATIC or REUSABLE A given content where the state of the content does NOT change for a given period of time t0 t1

35. Content with Query Strings Reusable? 110 /factor/create_image?name=book1&size=10x10

36. API Calls Reusable? 100 /api/GetBooks?category=fiction

37. API Calls Reusable? 80 /api/GetBooks?top=10

38. Caching for Smaller Time Units •  Imagine your have a read heavy API GETS Hit 1000 RPS •  Offload your web-tier from handling 1000 RPS •  Offload your load balancer: ELB or any other LB •  Provision less capacity and reduce cost 1000 /api/GetBooks?top=10

39. After Caching/Before CloudFront Dynamic Acceleration = 690ms

40. Base Page (First HTML page) Reusable? 220 /index.jsp

41. Can Dynamic Content Be Optimized? User Request Origin Edge Location Poke Poke Ok Ok PokePoke User Request

42. How to Optimize Dynamic Contents? DNS Lookup TCP Connection Time to First Byte Content Download

43. How to Improve DNS Time? DNS Lookup Index.jsp

44. Optimizing DNS Response Time •  Route 53 managed DNS offering •  Designed to be fast •  Low latency DNS resolution •  Global network of DNS servers •  Queries routed to the nearest DNS server Route 53

45. Without Route 53

46. With Route 53

47. How to Improve TCP Connection and First Byte Time? TCP Connection Index.jsp

48. TCP/IP Hand Shake •  HTTP Runs on TCP/IP •  TCP has the concept of TCP handshake •  Every HTTP Connection has to complete TCP Handshake •  TCP/IP Hand Shake Penalizes Dynamic

49. Two Users without CloudFront SYN SYN-ACK ACK GET /index.jsp ACK SYN-ACK GET /index.jsp 2nd User Region SYN 90ms 360ms 360ms

50. SYN SYN-ACK ACK GET /index.jsp GET /index.jsp Keep Alive Connections 2nd Request

51. CloudFront Keep Alive SYN SYN-ACK ACK GET /index.jsp ACK SYN-ACK GET /index.jsp Region SYN 30ms SYN SYN-ACK ACK GET /index.jsp GET /index.jsp 60ms 2nd User 360ms 180ms

52. Without Keep-Alive Connections •  Load on your web server increases the time to first byte Time to First Byte

53. •  Offloads your web tier’s CPU/memory •  More users More TCP connections •  Improves response time 540 msWith CloudFront CloudFront Keep Alive 720 msWithout CloudFront 1 connectionWith CloudFront

54. Test CPU Util. % Without CloudFront 20% With CloudFront 6%

55. Optimize SSL Connections with CloudFront’s SSL Termination TCP Connection Index.jsp

56. SSL Optimization with CloudFront •  CloudFront has the ability to support SSL traffic •  Use CloudFront cert or bring your own •  SSL traffic gets terminated at the closest CloudFront location

57. CloudFront SSL Optimization Benefits •  Taking advantage of keep-alive connections •  SSL introduces additional TCP handshake packets •  Keep alive eliminates additional SSL TCP handshake packets •  Offloading your infrastructure from terminating 1000s of end-users SSL connections

58. How to Improve Content Download Time? Content Download Index.jsp

59. With Amazon CloudFront Slow-Start Optimization

60. TCP Slow Start Packet1 Packet 1 ACK Packet 2 Packet 3 ACK Packet 3 Packet 4 Packet 5 Packet 6 Packet 7

61. •  CloudFront can optimize slow start •  Slow start impacts new connections not the existing ones •  CloudFront uses existing connections so users can skip slow start Slow-Start Optimization with CloudFront

62. Packet1 Packet 1 ACK Packet 2 Packet 3 ACK Packet 3 Packet 4 Packet 5 Packet 6 Packet 7 Packet1 Packet 2 Packet 4 ACK Packet 3 Packet 4 Packet 5 Packet 6 Packet 7 Packet 8 Packet 9 Region

63. Performance Test 1: Without CloudFront Oregon VirginiaRequestorOrigin

64. Performance Test 2: With CloudFront Oregon VirginiaRequestorOrigin

65. Performance Results Test # Of Packets Response Time Per Request Response Time For 200 Requests Without CloudFront 2605 170 ms 33.876 seconds With CloudFront 896 96 ms 19.24 seconds

66. After CloudFront Dynamic Content Optimization = 555 ms

67. Example: somecompany.com . sec

68. How to Improve Content Download Time Even More? Content Download Index.jsp

69. With Amazon Route 53 Latency-based Routing Route 53

70. Latency-based Routing (LBR) •  Run multiple stacks of your application in different EC2 regions around the world •  Create LBR records for each location and tag the location with geo information •  Route 53 will route end users to the endpoint that provides the lowest latency 70

71. LBR Benefits •  Better performance than running in a single region •  Improved reliability relative to running in a single region •  Easier implementation than traditional DNS solutions •  Much lower prices than traditional DNS solutions 71

72. LBR For End-Users

73. LBR For End-Users

74. CloudFront and Route 53 •  Use CloudFront for dynamic content optimization •  Host your origin at multiple AWS locations (or data centers) – US – Europe

75. CloudFront and Route 53 •  Create Origin DNS records in Route 53 for each location •  Route 53 measures the latency between CloudFront and all configured origins •  Route 53 resolves origin’s hostname to the closest location •  Reduce content download time

76. Lower Latency with CloudFront and Route 53

77. Lower Latency with CloudFront and Route 53

78. Design for Failure with CloudFront and Route 53 Route 53

79. Design for Failure Normal interaction: 1.  Users connect to CloudFront 2.  CloudFront connects to Origin Region CloudFront

80. Design for Failure How does caching help, if the origin is down and fails to respond to CloudFront? Region CloudFront

81. Design for Failure: Serve Cached Content Origin Edge Location User Request A

82. Design for Failure: Serve Cached Content Origin Edge Location Get Image User Request A

83. Design for Failure: Serve Cached Content Origin Edge Location Get Image Get Image User Request A

84. Design for Failure: Serve Cached Content Origin Edge Location Get Image Get Image Image User Request A

85. Design for Failure: Serve Cached Content Origin Edge Location Get Image Get Image Image Image User Request A

86. Design for Failure: Serve Cached Content Origin Edge Location Image User Request B

87. Design for Failure: Serve Cached Content Origin Edge Location Get Image Get Image User Request B

88. Design for Failure: Serve Cached Content Origin Edge Location Get Image Get Image User Request B

89. Design for Failure: Serve Cached Content Origin Edge Location Get Image Get ImageImage User Request B

90. Design for Failure: Caching •  Caching improves performance, but ……. •  Can also improve availability •  If your infrastructure is experiencing failure, CloudFront can serve cached content instead of 5xx,4xx and etc

91. Design for Failure •  Health-checking your origin with Amazon Route 53 Region Route53 Health Check Health Check

92. Design for Failure •  Failures can be detected by Route 53 health checks Region Route53 Health Check Health Check CloudFront

93. Design for Failure •  The traffic shifts to the healthy instances or load balancers instead Region Route53 Health Check Health Check CloudFront

94. Region 2 Route53 Health Check Health Check Region 1 Origin DNS Resolution Based on Latency And Health Checks Can mix health check and latency-based routing Can apply the same logic to multiregion deployments

95. Region 2 Route53 Health Check Health Check Region 1 Origin DNS Resolution Based on Latency And Health Checks Users get connected to the closest region if both regions are healthy

96. Region 2 Route53 Health Check Health Check Region 1 Origin DNS Resolution Based on Latency And Health Checks Route 53 detects failures via health checks

97. Region 2 Route53 Health Check Health Check Region 1 Origin DNS Resolution Based on Latency And Health Checks Users get connected to the healthy regions if the closest region is not passing the health check

98. Summary •  Accelerate all your content with CloudFront •  Use CloudFront with Route 53 latency-based routing to improve your performance •  Design for failure with CloudFront and Amazon Route 53

99. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. Dynamic Content Acceleration: Lightning-Fast Web Apps with Amazon CloudFront and Amazon Route 53 Glyn Smith, Business Development Manager April, 2014 Thank you!

100. © 2014 Amazon.com, Inc. and its aﬃliates. All rights reserved. May not be copied, modiﬁed, or distributed in whole or in part without the express consent of Amazon.com, Inc. Lightning Fast WebApps Matthew Painter, CTO, import.io 30th April 2014

101. What is import.io? http://go.import.io/aws-about

102. Use cases http://go.import.io/aws-oxfam

103. Measuring CDN performance •  How do we know our CDN is effective? –  Significant engineering work –  Simple tools •  http://go.import.io/aws-test –  Academic research •  http://go.import.io/aws-measure [PDF]

104. CloudFront use cases •  Distributing large files •  Serving static WebApps •  Serving branded content

105. Distributing binaries •  Application installation binaries –  50 to 100MB each •  CloudFront with S3 Origin –  1 hour TTL (set on CDN) •  Extremely quick to set up (30 mins)

106. Serving static WebApps •  API / Client separation Browser Elastic Load Balancer Elastic Load Balancers import.io api.import.io AutoScaling Group (API servers) AutoScaling Group (nginx reverse proxies) S3 Upstream CloudFront Distribution xyz.cloudfront.ne t version = ? version = 𝑥 version = 𝑥 version = 𝑥

107. Serving static WebApps •  Versioning static files Git repos Build server proj1_branch2_183 proj1_branch3_184 proj2_branch0_185 S3 /build /build/183 /build/184 /build/185 CloudFront Distribution Clone Upload

108. Serving static WebApps •  Quick to set up •  Change configuration when WebApp is built •  JavaScript requires CORS set on S3 –  Cross Origin Resource Sharing: http://go.import.io/aws-cors –  Compatibility recently improved –  http://go.import.io/aws-s3cors •  1 year (or more) TTL, set using S3 headers –  http://go.import.io/aws-expiration

109. Serving static WebApps •  Initial load: 11% better, same spread •  Start render: 45% better, 63% lower spread •  Full load: 13% better, 33% lower spread •  Repeat load: 25% better, 36% lower spread •  Caveats: networks, sample sizes, traffic, etc.

110. Serving branded content •  cdn.import.io –  Alias record in Route53 –  http://go.import.io/aws-alias –  Domain needs to be setup on both Route53 and CloudFront •  SSL with custom domain (https://cdn.import.io) –  SNI SSL certificate –  http://go.import.io/aws-sni

111. Route53 Failover •  Provides a website in case of disaster •  Important to inform users and reduce support calls Route53 (import.io) Elastic Load Balancer AutoScaling Group Health checks S3 static content (import.io bucket)

112. POC: Latency-based Routing •  Trials of serving frontend content globally Browser Elastic Load Balancer AutoScaling Group Elastic Load Balancer AutoScaling Group Route53 No shared state

113. POC: Latency-based Routing •  Route53 and CloudFront Browser Elastic Load Balancer AutoScaling Group Elastic Load Balancer AutoScaling Group Route53 No shared state CloudFront

114. POC: Dynamic content optimisation •  Trials of dynamic content behind CloudFront Browser Elastic Load Balancer Elastic Load Balancers import.io api.import.io AutoScaling Group (API servers) AutoScaling Group (nginx reverse proxies) S3 Upstream CloudFront Distribution xyz.cloudfront.ne t version = ? version = 𝑥 version = 𝑥 version = 𝑥 CloudFront Distribution

115. Thank you We use: Java: Hazelcast, Guava, CometD, Spring, Jetty, OSGi We do: Devops, AWS, testing, agile We have: Cool office, diverse team, well funded matthew.painter@import.io

116. Please rate this session using the AWS Summits App and help us build better events

117. #AWSSummit @AWScloud @AWS_UKI