AWS Pop-up Loft Berlin: Cache is King - Running Lean Architectures: Optimizing for Cost Efficiency

1. Cache is King Scale Your Application while Improving Performance and Lowering Costs Constantin Gonzalez Principal Solutions Architect Amazon Web Services Germany GmbH glez@amazon.de | @zalez

2. Cash is KingCash is King

3. Cache is Cash

4. Httparchive

5. Content Breakdown 87% of Your Site Consists Of Static/Re-usable Content

6. 52% of The Internet’s Content is NOT Cached Today But Wait…

7. Cache is King Source: httparchive.org300K Tracked Websites Oct 2014-Oct 2015 ~52% {

8. CACHE EVERYTHING

9. Browser Cache Edge Cache Web Cache Application Cache Database Cache Image Source: cakeormistake.wordpress.com

11. Speed of Light ~300,000 Km/s Image Source: wordlesstech.com

12. ~200,000 Km/s through fiber Image Source: www2.wiringandcabling.com

13. Download Performance Round Trip: 13,026 km

14. Download Performance Round Trip: 13,026 km Theoretical: 65 ms

15. Download Performance Round Trip: 13,026 km Theoretical: 65 ms Measured: 68 ms

16. Time to First Byte 68 ms Client Server

17. Time to First Byte 68 ms 68 ms Client Server SYN SYN/ACK

18. Time to First Byte 68 ms 68 ms 68 ms Client Server 136 ms SYN SYN/ACK ACK, GET /image.jpg HTTP/1.1

19. Browser Cache

20. Browser Caching Set max-age or expiry date in the headers. HTML5 Application Cache. Helps eliminate network latency. But… Browser cache size is limited. (e.g. IE is 8-50M, Chrome is < 80M, Firefox is 50MB, etc.)

21. Browser Cache Edge Cache

22. Time to First Byte 68 ms 68 ms 68 ms Client Server 136 ms SYN SYN/ACK ACK, GET /image.jpg HTTP/1.1

23. Bring the Bytes Closer to Your Users 68 ms Client Origin SYN 10 ms SYN/ACK GET /image.jpg HTTP/1.1ACK, GET /image.jpg HTTP/1.1 CloudFront Saves 68 ms RTT SYN SYN/ACK Time to first byte: 20 ms vs. 136 ms

24. Amazon CloudFront: 53 Global Locations

25. Edge Cache: Amazon CloudFront Amazon S3 Elastic Load Balancing Mobile Clients Browsers/Desktop Clients Tablets/Devices Amazon CloudFront Edge Location

26. Edge Cache How do you decide what to cache? • Static or Re-Usable Content • Customized Content • On-Demand and Live Video • Dynamic or Unique Content

27. Cache Static or Re-Usable Content HTTP/1.0 200 OK Date: Mon, 19 Mar 201 12:51:28 GMT Server: Apache Last-Modified: Mon, 19 Mar 2012 07:15:25 GMT Accept-Ranges: bytes Content-Length: 1918 Cache-Control: max-age=86400 Content-Type: image/jpeg Vary: Accept-Encoding Age: 16 X-Cache: Hit from cloudfront Connection: keep-alive

29. Cache Customized Content Customized Content

31. Cache Video Content

32. Caching Your Live Stream? Source HTTP Cache Edge Paris New York Encoder HLS, HDS, Smooth Streaming Hong Kong

35. Dynamic Content? Zero TTL – cannot be cached! CloudFront can still help…

36. Dynamic Content? Zero TTL – cannot be cached! CloudFront can still help… TCP/IP optimizations for the network path Keep-Alive Connections to reduce RTT SSL Termination close to viewers POST/PUT upload optimizations Latency Based Routing Low prices, same as static content delivery!

37. AZAZ CloudFront EU APUS ELB US

38. Browser Cache Edge Cache Web Cache

39. You can’t have dessert, until you have your dinner! -An Experiment. https://secure.flickr.com/photos/stephen-oung/6319155216/

40. Experiment Premise • Start with basic infrastructure. – Single m1.large EC2 instance running Amazon Linux – Single non-Multi-AZ m1.xlarge RDS MySQL 5.6 instance – Apache httpd-2.2.25 – PHP 5.3.27 – Drupal Commerce Kickstart 7.2 – EIP on Instance • Throw a ton of traffic at it. – 8,000,000 queries, 40 per second, from 4 other instances. • Profile it. – New Relic PHP agent – webpagetest.org

41. Experiment Infrastructure Web/App Instance Elastic IP Amazon RDS DB Instance Clients

42. Results? (not pretty) 14.7 second responses Mostly Static Data Low RPM

43. Web Cache Webserver or proxy caches would live between your CDN/Users and your web tier and can offer up increased cost performance via reducing internal application and database load. Can also offer up increased edge to origin speed for lots of content.

44. Web Cache Popular solutions: – Varnish – Nginx – Apache with mod_cache/mod_proxy – Squid – Perlbal – Language/framework caches (i.e., APC, Zend)

45. Web Cache Availability Zone VPC Subnet VPC Subnet Elastic Load Balancing Cache Instances Web/App Instances Elastic Load Balancing Availability Zone VPC Subnet VPC Subnet Elastic Load Balancing Cache Instances Web/App Instances Elastic Load Balancing Internet Gateway Amazon Route 53 Amazon CloudFront Customer Traffic

63. Web Cache • Opt for in-memory caching when possible. • Pay attention to your cache hit/miss ratios. It could be a sign that you need to re-size the instances or re-size the number of nodes in your cache pool. • Set smart TTLs so that you don’t affect new deploys or cache content for too long. • Be smart about what cookies can burst cache and what cookies can’t. Don’t serve up other people’s content or stale dynamic pages.

64. Web Cache How do you decide what to cache? – All logged out user pages – Any completely static pages – Traffic/log analysis • Look at your web logs/CDN logs • Find heavily hit pages • Figure out how often they actually change • Apply a TTL to that page to be cached – Even 60 second TTLs could help drastically!

65. Browser Cache Edge Cache Web Cache Application Cache

66. Application Cache Application level caches for information such as session data, temporary application data such as cart information, and live aggregation of data feeds.

67. Application Cache Popular solutions: – Memcached – Redis – Cassandra – Amazon DynamoDB

68. Application Cache Availability Zone VPC Subnet VPC SubnetVPC Subnet Elastic Load Balancing Cache Instances Web/App Instances Elastic Load Balancing ElastiCache Cache Node Availability Zone VPC Subnet VPC SubnetVPC Subnet Elastic Load Balancing Cache Instances Web/App Instances Elastic Load Balancing ElastiCache Cache Node Internet Gateway

69. Application Cache Availability Zone VPC Subnet VPC SubnetVPC Subnet Elastic Load Balancing Cache Instances Web/App Instances Elastic Load Balancing ElastiCache Cache Node Availability Zone VPC Subnet VPC SubnetVPC Subnet Elastic Load Balancing Cache Instances Web/App Instances Elastic Load Balancing ElastiCache Cache Node Internet Gateway

70. Amazon DynamoDB Session Handler for PHP https://aws.amazon.com/sdkforphp/

71. Application Cache Internet Gateway Availability Zone VPC Subnet VPC Subnet Elastic Load Balancing Cache Instances Web/App Instances Elastic Load Balancing Squid Proxy Instances Amazon DynamoDB Availability Zone VPC Subnet VPC Subnet Elastic Load Balancing Cache Instances Web/App Instances Elastic Load Balancing Squid Proxy Instances

72. Application Cache Internet Gateway Availability Zone VPC Subnet VPC Subnet Elastic Load Balancing Cache Instances Web/App Instances Elastic Load Balancing Squid Proxy Instances Amazon DynamoDB Availability Zone VPC Subnet VPC Subnet Elastic Load Balancing Cache Instances Web/App Instances Elastic Load Balancing Squid Proxy Instances

73. Application Cache Use Cases: – Session information – Temporary data • Cart info, metadata – Rate limiting • Fight abuse of APIs, spamming, functionality abuse – Counters • Views, Scores, Leader Boards

74. Application Cache How do you decide what to cache? – Ideally, you have to treat data you cache at this tier as loss tolerant if working with in-memory caches. – Session information that wouldn’t make sense in cookies or in a true DB. – Look at the kind of data your application is generating and storing in a DB that it might not need to.

75. Browser Cache Edge Cache Web Cache Application Cache Database Cache

76. Database Cache Reduce workload on database servers by caching commonly requested information, or any information that might not change frequently (i.e., user info, listing info, product info).

77. Database Cache Popular solutions: – In-engine query caches – Memcached • On dedicated host • On DB host (built in w/ MySQL 5.6) – Redis • On dedicated host

78. Database Cache A word of caution: In-engine DB caches are often not recommended for many use cases, as they can significantly impact the performance of many databases. Depending on the workload and dataset you have, an in-engine query cache might not be a good idea for you. We recommend off DB caches where possible.

79. Database Cache Availability Zone VPC Subnet VPC SubnetVPC Subnet Elastic Load Balancing Cache Instances Web/App Instances Elastic Load Balancing ElastiCache Cache Node Availability Zone VPC Subnet VPC SubnetVPC Subnet Elastic Load Balancing Cache Instances Web/App Instances Elastic Load Balancing ElastiCache Cache Node Internet Gateway RDS Instance Primary (M-AZ) RDS Instance Standby (MAZ)

80. Database Cache Availability Zone VPC Subnet VPC SubnetVPC Subnet Elastic Load Balancing Cache Instances Web/App Instances Elastic Load Balancing ElastiCache Cache Node Availability Zone VPC Subnet VPC SubnetVPC Subnet Elastic Load Balancing Cache Instances Web/App Instances Elastic Load Balancing ElastiCache Cache Node Internet Gateway RDS Instance Primary (M-AZ) RDS Instance Standby (MAZ)

81. Memcached – Code Sample function retrieveValue($query) { $queryId = md5($query); if ($myValue = $memcache->get($queryId)) { return $myValue; } else { $myValue = dbfetch($query); $memcache->set($queryId, $myValue); return $myValue; } }

82. MySQL 5.6 + Memcached RDS MySQL supports version 5.6 with integrated Memcached on the instance: – Part of the InnoDB engine – Memcached running as part of MySQL talks directly to data in InnoDB tables, essentially turning MySQL into a fast “key-value store” – From the opposite view point, adds persistence to Memcached – Same Memcached API as standalone https://dev.mysql.com/doc/refman/5.6/en/innodb-memcached-intro.html

83. Cache everything MongoDB DynamoDB Amazon RDS MongoDB DynamoDB Amazon RDS ElastiCache

84. MongoDB Queries • Shut down 8 machines Caching saves money DynamoDB Reads • Saved 3k reads per second • (>20k reads per second in total)

85. Without negative caching Cache really everything! With negative caching • Cache hit ratio 25-30% • Cache hit ratio 89-95% Hit Miss

86. AWS Marketplace & Partners Can Help • Customers can find, research, buy software. • Simple pricing, aligns with Amazon EC2 usage model. • Launch in minutes! • Marketplace billing integrated into your AWS account. • 1100+ products across 24+ categories. Learn more at: aws.amazon.com/marketplace

87. Back To Our Experiment! https://secure.flickr.com/photos/toyochin/1382531438/

88. Experiment Infrastructure (original) Web/App Instance Elastic IP Amazon RDS DB Instance Clients

89. Results? (not pretty) 14.7 second responses Mostly Static Data Low RPM

90. Experiment Infrastructure (with cake) • Added in ElastiCache Memcached cache.m1.large • Added in Amazon CloudFront for static content – Tuned expires and cache headers in Apache • Added in APC for PHP caching – Increased memory to 128Mb, no other changes • Did nothing to the DB • Drupal memcached & CDN modules

91. CloudFront Experiment Infrastructure (with cake) Web/App Instance Elastic IP Amazon RDS DB Instance Clients ElastiCache Cache Node

92. Results? (AWESOME!) response time ~1/2 ~2x RPM Repeat Page view from 15.8sec to 2.75sec

93. Easy as Pie! https://secure.flickr.com/photos/linecon0/2654865842/

94. Thank you! Constantin Gonzalez glez@amazon.de @zalez

AWS Pop-up Loft Berlin: Cache is King - Running Lean Architectures: Optimizing for Cost Efficiency

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (15)

Similar to AWS Pop-up Loft Berlin: Cache is King - Running Lean Architectures: Optimizing for Cost Efficiency

Similar to AWS Pop-up Loft Berlin: Cache is King - Running Lean Architectures: Optimizing for Cost Efficiency (20)

More from AWS Germany

More from AWS Germany (20)

Recently uploaded

Recently uploaded (20)

AWS Pop-up Loft Berlin: Cache is King - Running Lean Architectures: Optimizing for Cost Efficiency