Your SlideShare is downloading. ×
0
Cache is King:
Scale Your Application while Improving
Performance and Lowering Costs

November 15, 2013

© 2013 Amazon.com...
Cash is King
Cache is Cash
Httparchive
Content Breakdown
87% of Your Site Consists Of Static/Re-usable Content
But Wait…
56% of The Internet’s Content is NOT Cached Today
Cache is King

~56%

300K Tracked Websites Oct 15 2012-Oct 1 2013

{
Source: httparchive.org
CACHE EVERYTHING
Browser Cache
Edge Cache
Web Cache

Application Cache
Database Cache
Image Source: cakeormistake.wordpress.com
Speed of Light ~300,000 Km/s
Image Source: wordlesstech.com
~200,000 Km/s through fiber
Image Source: www2.wiringandcabling.com
Download Performance

Round Trip: 13,026 km
Download Performance

Round Trip: 13,026 km
Theoretical: 65 ms
Download Performance

Round Trip: 13,026 km
Theoretical: 65 ms
Measured: 68 ms
Time to First Byte
68 ms

Client

Server
Time to First Byte
68 ms
SYN

68 ms

SYN/ACK

Client

Server
Time to First Byte
68 ms
SYN

68 ms
68 ms
136 ms

SYN/ACK
ACK,
GET /image.jpg HTTP/1.1

Client

Server
Browser Cache
Browser Caching
Set max-age or expiry date in the headers.

HTML5 Application Cache.
Helps eliminate network latency.
But…...
Browser Cache
Edge Cache
Time to First Byte
68 ms
SYN

68 ms
68 ms
136 ms

SYN/ACK
ACK,
GET /image.jpg HTTP/1.1

Client

Server
Bring the Bytes Closer to Your Users
10 ms

Saves 68 ms
RTT

68 ms

SYN

SYN

SYN/ACK

SYN/ACK

ACK,

GET /image.jpg HTTP/...
Amazon CloudFront: 46 Global Locations

=
Edge Cache: Amazon CloudFront
Mobile Clients

Amazon S3

Amazon CloudFront
Edge Location

Browsers/Desktop
Clients

Elasti...
Edge Cache
How do you decide what to cache?
• Static or Re-Usable Content
• Customized Content

• On-Demand and Live Video...
Cache Static or Re-Usable Content
HTTP/1.0 200 OK
Date: Mon, 19 Mar 201 12:51:28 GMT
Server: Apache
Last-Modified: Mon, 19...
Cache Customized Content

Customized Content
Cache Video Content
Caching Your Live Stream?
Edge
HLS, HDS,
Smooth Streaming

HTTP
Cache
Hong Kong

Source

Encoder

Paris

New York
Dynamic Content?
Zero TTL – cannot be cached! CloudFront can still help…
Dynamic Content?
Zero TTL – cannot be cached! CloudFront can still help…
TCP/IP optimizations for the network path

Keep...
“We are excited to use CloudFront's new
POST, PUT, PATCH, and DELETE capabilities
to accelerate our RESTful APIs on Amazon...
Browser Cache
Edge Cache
Web Cache
You can’t have dessert,
until you have your dinner!

-An Experiment.
https://secure.flickr.com/photos/stephen-oung/6319155...
Experiment Premise
• Start with basic infrastructure.
–
–
–
–
–
–

Single m1.large EC2 instance running Amazon Linux
Singl...
Experiment Infrastructure

Elastic IP
Amazon RDS
DB Instance

Web/App
Instance

Clients
Results?

(not pretty)

14.7 second responses

Low RPM

Mostly Static Data
Web Cache
Webserver or proxy caches would live between
your CDN/Users and your web tier and can offer
up increased cost pe...
Web Cache
Popular solutions:
–
–
–
–
–
–

Varnish
Nginx
Apache with mod_cache/mod_proxy
Squid
Perlbal
Language/framework c...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
Customer
Traffic

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing
VPC Subnet
...
Web Cache
• Opt for in-memory caching when possible.

• Pay attention to your cache hit/miss ratios. It could be a sign th...
Web Cache
How do you decide what to cache?
– All logged out user pages
– Any completely static pages
– Traffic/log analysi...
Browser Cache
Edge Cache
Web Cache

Application Cache
Application Cache
Application level caches for information such
as session data, temporary application data
such as cart i...
Application Cache
Popular solutions:
–
–
–
–

Memcached
Redis
Cassandra
Amazon DynamoDB
Application Cache

ElastiCache
Cache Node

Web/App
Instances

VPC Subnet

Elastic Load
Balancing

Cache
Instances

Elastic...
Application Cache

ElastiCache
Cache Node

Web/App
Instances

VPC Subnet

Elastic Load
Balancing

Cache
Instances

Elastic...
Amazon DynamoDB Session Handler for PHP
https://aws.amazon.com/sdkforphp/
Application Cache

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing

Squid Proxy
Instanc...
Application Cache

Web/App
Instances

Elastic Load
Balancing

Cache
Instances

Elastic Load
Balancing

Squid Proxy
Instanc...
Application Cache
Use Cases:
– Session information
– Temporary data
• Cart info, metadata

– Rate limiting
• Fight abuse o...
Application Cache
How do you decide what to cache?
– Ideally, you have to treat data you cache at this tier as
loss tolera...
Browser Cache
Edge Cache
Web Cache

Application Cache
Database Cache
Database Cache
Reduce workload on database servers by
caching commonly requested information, or
any information that migh...
Database Cache
Popular solutions:
– In-engine query caches
– Memcached
• On dedicated host
• On DB host (built in w/ MySQL...
Database Cache
A word of caution:
In-engine DB caches are often not recommended for
many use cases, as they can significan...
Database Cache

RDS Instance ElastiCache
Primary (M-AZ) Cache Node

Web/App
Instances

VPC Subnet

Elastic Load
Balancing
...
Database Cache

RDS Instance ElastiCache
Primary (M-AZ) Cache Node

Web/App
Instances

VPC Subnet

Elastic Load
Balancing
...
Memcached – Code Sample
function retrieveValue($query)
{
$queryId = md5($query);
if ($myValue = $memcache->get($queryId))
...
MySQL 5.6 + Memcached
RDS MySQL supports version
5.6 with integrated Memcached
on the instance:
– Part of the InnoDB engin...
AWS Marketplace & Partners Can Help
•

Customers can find, research, buy
software.

•

Simple pricing, aligns with Amazon
...
Back To Our
Experiment!
https://secure.flickr.com/photos/toyochin/1382531438/
Experiment Infrastructure (original)

Elastic IP
Amazon RDS
DB Instance

Web/App
Instance

Clients
Results?

(not pretty)

14.7 second responses

Low RPM

Mostly Static Data
Experiment Infrastructure (with cake)
• Added in ElastiCache Memcached
cache.m1.large
• Added in Amazon CloudFront for sta...
Experiment Infrastructure (with cake)
CloudFront

Elastic IP
Amazon RDS
DB Instance

Web/App
Instance

Clients
ElastiCache...
Results?

(AWESOME!)

response time ~1/2

~2x RPM
Repeat Page view from 15.8sec to 2.75sec
Easy as Pie!

https://secure.flickr.com/photos/linecon0/2654865842/
Toronto Star
•
•
•
•

Canada’s largest daily newspaper
Focused on metro Toronto
3.3 million monthly unique visitors
Small ...
mystar
Wheels.ca
Toronto.com
Thegridto.com
Why Cache?
•
•
•
•

Performance
Scalability
Reliability
Cost
Breaking News
Breaking News: Client View
Breaking News: Server View
The Caching Onion
DB
App

Reverse
proxy
Edge
Edge – The Onion Skin
• Examples: Akamai,
CloudFront
• Static Asset caching
• Full site caching
• Origins
• Behaviors
• th...
Edge – The Onion Skin
Cloudfront
Elastic Load Balancer
Production - US East Region

Thestar.com
architecture

VPC

Dispatc...
Cloudfront
Elastic Load Balancer
Production - US East Region

VPC

Dispatcher/Apache
Amazon Linux 64 Bit
C1 Medium

Publis...
Whole Site Delivery
•
•

•
•

•

Cache everything possible
No server side cookies written,
only select pages pass query
st...
Reverse Proxy / Web Accelerator
• Examples: Varnish,
Nginx, Apache
• Serves “static” content
• Reduce load on app
server
•...
Reverse Proxy / Web Accelerator
mystar
Reverse Proxy / Web Accelerator
mystar
Reverse Proxy / Web Accelerator
Wheels.ca
App-level caching
• Examples:
Memcached, Reddis
– ElastiCache now supports
both

• Session caching
• View caching
• mystar...
App-level caching
Database caching
• Example:
Memcached
• Cache frequent SQL
queries
• Reduce DB server
load, response time
• mystar, wheels...
Database Caching
Wheels.ca
Database Caching
Wheels.ca
Thanks!
Ping me
sevans@thestar.ca
@stephenaevans
Please give us your feedback on this
presentation

SVC203
As a thank you, we will select prize
winners daily for completed...
APPENDIX
Reverse Proxy / Web Accelerator

Varnish Cache is a web application accelerator also
known as a caching HTTP reverse proxy...
Reverse Proxy / Web Accelerator
Nginx (pronounced "engine x") is an open
source reverse proxy server for HTTP,
HTTPS, SMTP...
App-level caching
Memcached
Free & open source, high-performance, distributed
memory object caching system, generic in nat...
App-level caching
Redis
Redis is an open source, BSD licensed, advanced
key-value store. It is often referred to as a data...
Edge – The Onion Skin
“A content delivery network is a large distributed
system of servers deployed in multiple data
cente...
Reverse Proxy / Web Accelerator
“A web accelerator is a proxy server that
reduces web site access times. They can be
a sel...
Scale Your Application while Improving Performance and Lowering Costs (SVC203) | AWS re:Invent 2013
Scale Your Application while Improving Performance and Lowering Costs (SVC203) | AWS re:Invent 2013
Scale Your Application while Improving Performance and Lowering Costs (SVC203) | AWS re:Invent 2013
Scale Your Application while Improving Performance and Lowering Costs (SVC203) | AWS re:Invent 2013
Scale Your Application while Improving Performance and Lowering Costs (SVC203) | AWS re:Invent 2013
Scale Your Application while Improving Performance and Lowering Costs (SVC203) | AWS re:Invent 2013
Scale Your Application while Improving Performance and Lowering Costs (SVC203) | AWS re:Invent 2013
Scale Your Application while Improving Performance and Lowering Costs (SVC203) | AWS re:Invent 2013
Scale Your Application while Improving Performance and Lowering Costs (SVC203) | AWS re:Invent 2013
Scale Your Application while Improving Performance and Lowering Costs (SVC203) | AWS re:Invent 2013
Scale Your Application while Improving Performance and Lowering Costs (SVC203) | AWS re:Invent 2013
Scale Your Application while Improving Performance and Lowering Costs (SVC203) | AWS re:Invent 2013
Upcoming SlideShare
Loading in...5
×

Scale Your Application while Improving Performance and Lowering Costs (SVC203) | AWS re:Invent 2013

3,730

Published on

Scaling your application as you grow should not mean slow to load and expensive to run. Learn how you can use different AWS building blocks such as Amazon ElastiCache and Amazon CloudFront to “cache everything possible” and increase the performance of your application by caching your frequently-accessed content. This means caching at different layers of the stack: from HTML pages to long-running database queries and search results, from static media content to application objects. And how can caching more actually cost less? Attend this session to find out!

Published in: Technology

Transcript of "Scale Your Application while Improving Performance and Lowering Costs (SVC203) | AWS re:Invent 2013"

  1. 1. Cache is King: Scale Your Application while Improving Performance and Lowering Costs November 15, 2013 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
  2. 2. Cash is King
  3. 3. Cache is Cash
  4. 4. Httparchive
  5. 5. Content Breakdown 87% of Your Site Consists Of Static/Re-usable Content
  6. 6. But Wait… 56% of The Internet’s Content is NOT Cached Today
  7. 7. Cache is King ~56% 300K Tracked Websites Oct 15 2012-Oct 1 2013 { Source: httparchive.org
  8. 8. CACHE EVERYTHING
  9. 9. Browser Cache Edge Cache Web Cache Application Cache Database Cache Image Source: cakeormistake.wordpress.com
  10. 10. Speed of Light ~300,000 Km/s Image Source: wordlesstech.com
  11. 11. ~200,000 Km/s through fiber Image Source: www2.wiringandcabling.com
  12. 12. Download Performance Round Trip: 13,026 km
  13. 13. Download Performance Round Trip: 13,026 km Theoretical: 65 ms
  14. 14. Download Performance Round Trip: 13,026 km Theoretical: 65 ms Measured: 68 ms
  15. 15. Time to First Byte 68 ms Client Server
  16. 16. Time to First Byte 68 ms SYN 68 ms SYN/ACK Client Server
  17. 17. Time to First Byte 68 ms SYN 68 ms 68 ms 136 ms SYN/ACK ACK, GET /image.jpg HTTP/1.1 Client Server
  18. 18. Browser Cache
  19. 19. Browser Caching Set max-age or expiry date in the headers. HTML5 Application Cache. Helps eliminate network latency. But… Browser cache size is limited. (e.g. IE is 8-50M, Chrome is < 80M, Firefox is 50MB, etc.)
  20. 20. Browser Cache Edge Cache
  21. 21. Time to First Byte 68 ms SYN 68 ms 68 ms 136 ms SYN/ACK ACK, GET /image.jpg HTTP/1.1 Client Server
  22. 22. Bring the Bytes Closer to Your Users 10 ms Saves 68 ms RTT 68 ms SYN SYN SYN/ACK SYN/ACK ACK, GET /image.jpg HTTP/1.1 GET /image.jpg HTTP/1.1 Client CloudFront Origin Time to first byte: 20 ms vs. 136 ms
  23. 23. Amazon CloudFront: 46 Global Locations =
  24. 24. Edge Cache: Amazon CloudFront Mobile Clients Amazon S3 Amazon CloudFront Edge Location Browsers/Desktop Clients Elastic Load Balancing Tablets/Devices
  25. 25. Edge Cache How do you decide what to cache? • Static or Re-Usable Content • Customized Content • On-Demand and Live Video • Dynamic or Unique Content
  26. 26. Cache Static or Re-Usable Content HTTP/1.0 200 OK Date: Mon, 19 Mar 201 12:51:28 GMT Server: Apache Last-Modified: Mon, 19 Mar 2012 07:15:25 GMT Accept-Ranges: bytes Content-Length: 1918 Cache-Control: max-age=86400 Content-Type: image/jpeg Vary: Accept-Encoding Age: 16 X-Cache: Hit from cloudfront Connection: keep-alive
  27. 27. Cache Customized Content Customized Content
  28. 28. Cache Video Content
  29. 29. Caching Your Live Stream? Edge HLS, HDS, Smooth Streaming HTTP Cache Hong Kong Source Encoder Paris New York
  30. 30. Dynamic Content? Zero TTL – cannot be cached! CloudFront can still help…
  31. 31. Dynamic Content? Zero TTL – cannot be cached! CloudFront can still help… TCP/IP optimizations for the network path Keep-Alive Connections to reduce RTT SSL Termination close to viewers POST/PUT upload optimizations Latency Based Routing Low prices, same as static content delivery!
  32. 32. “We are excited to use CloudFront's new POST, PUT, PATCH, and DELETE capabilities to accelerate our RESTful APIs on Amazon EC2. With these new HTTP methods we can now take advantage of CloudFront’s global footprint and optimized connections back to our origin servers in AWS. Routing our customers’ API requests via a CloudFront edge location near them will help improve their experience by minimizing packet loss and upload latency. This will help provide a streamlined experience for our customers.” - Ilan Rabinovitch, Tech Lead, Site Reliability Engineering
  33. 33. Browser Cache Edge Cache Web Cache
  34. 34. You can’t have dessert, until you have your dinner! -An Experiment. https://secure.flickr.com/photos/stephen-oung/6319155216/
  35. 35. Experiment Premise • Start with basic infrastructure. – – – – – – Single m1.large EC2 instance running Amazon Linux Single non-Multi-AZ m1.xlarge RDS MySQL 5.6 instance Apache httpd-2.2.25 PHP 5.3.27 Drupal Commerce Kickstart 7.2 EIP on Instance • Throw a ton of traffic at it. – 8,000,000 queries, 40 per second, from 4 other instances. • Profile it. – New Relic PHP agent – webpagetest.org
  36. 36. Experiment Infrastructure Elastic IP Amazon RDS DB Instance Web/App Instance Clients
  37. 37. Results? (not pretty) 14.7 second responses Low RPM Mostly Static Data
  38. 38. Web Cache Webserver or proxy caches would live between your CDN/Users and your web tier and can offer up increased cost performance via reducing internal application and database load. Can also offer up increased edge to origin speed for lots of content.
  39. 39. Web Cache Popular solutions: – – – – – – Varnish Nginx Apache with mod_cache/mod_proxy Squid Perlbal Language/framework caches (i.e., APC, Zend)
  40. 40. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  41. 41. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  42. 42. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  43. 43. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  44. 44. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  45. 45. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  46. 46. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  47. 47. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  48. 48. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  49. 49. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  50. 50. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  51. 51. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  52. 52. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  53. 53. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  54. 54. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  55. 55. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  56. 56. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  57. 57. Web Cache Customer Traffic Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Amazon Route 53 Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances VPC Subnet Availability Zone Elastic Load Balancing VPC Subnet Amazon CloudFront
  58. 58. Web Cache • Opt for in-memory caching when possible. • Pay attention to your cache hit/miss ratios. It could be a sign that you need to re-size the instances or re-size the number of nodes in your cache pool. • Set smart TTLs so that you don’t affect new deploys or cache content for too long. • Be smart about what cookies can burst cache and what cookies can’t. Don’t serve up other people’s content or stale dynamic pages.
  59. 59. Web Cache How do you decide what to cache? – All logged out user pages – Any completely static pages – Traffic/log analysis • • • • Look at your web logs/CDN logs Find heavily hit pages Figure out how often they actually change Apply a TTL to that page to be cached – Even 60 second TTLs could help drastically!
  60. 60. Browser Cache Edge Cache Web Cache Application Cache
  61. 61. Application Cache Application level caches for information such as session data, temporary application data such as cart information, and live aggregation of data feeds.
  62. 62. Application Cache Popular solutions: – – – – Memcached Redis Cassandra Amazon DynamoDB
  63. 63. Application Cache ElastiCache Cache Node Web/App Instances VPC Subnet Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Internet Gateway ElastiCache Cache Node VPC Subnet Web/App Instances Elastic Load Balancing VPC Subnet Availability Zone Cache Instances Elastic Load Balancing VPC Subnet
  64. 64. Application Cache ElastiCache Cache Node Web/App Instances VPC Subnet Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Internet Gateway ElastiCache Cache Node VPC Subnet Web/App Instances Elastic Load Balancing VPC Subnet Availability Zone Cache Instances Elastic Load Balancing VPC Subnet
  65. 65. Amazon DynamoDB Session Handler for PHP https://aws.amazon.com/sdkforphp/
  66. 66. Application Cache Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing Squid Proxy Instances VPC Subnet VPC Subnet Amazon DynamoDB Availability Zone Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Squid Proxy Instances
  67. 67. Application Cache Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing Squid Proxy Instances VPC Subnet VPC Subnet Amazon DynamoDB Availability Zone Internet Gateway Web/App Instances Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Squid Proxy Instances
  68. 68. Application Cache Use Cases: – Session information – Temporary data • Cart info, metadata – Rate limiting • Fight abuse of APIs, spamming, functionality abuse – Counters • Views, Scores, Leader Boards
  69. 69. Application Cache How do you decide what to cache? – Ideally, you have to treat data you cache at this tier as loss tolerant if working with in-memory caches. – Session information that wouldn’t make sense in cookies or in a true DB. – Look at the kind of data your application is generating and storing in a DB that it might not need to.
  70. 70. Browser Cache Edge Cache Web Cache Application Cache Database Cache
  71. 71. Database Cache Reduce workload on database servers by caching commonly requested information, or any information that might not change frequently (i.e., user info, listing info, product info).
  72. 72. Database Cache Popular solutions: – In-engine query caches – Memcached • On dedicated host • On DB host (built in w/ MySQL 5.6) – Redis • On dedicated host
  73. 73. Database Cache A word of caution: In-engine DB caches are often not recommended for many use cases, as they can significantly impact the performance of many databases. Depending on the workload and dataset you have, an in-engine query cache might not be a good idea for you. We recommend off DB caches where possible.
  74. 74. Database Cache RDS Instance ElastiCache Primary (M-AZ) Cache Node Web/App Instances VPC Subnet Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Internet Gateway RDS Instance ElastiCache Standby (MAZ) Cache Node VPC Subnet Web/App Instances Elastic Load Balancing VPC Subnet Availability Zone Cache Instances Elastic Load Balancing VPC Subnet
  75. 75. Database Cache RDS Instance ElastiCache Primary (M-AZ) Cache Node Web/App Instances VPC Subnet Elastic Load Balancing Cache Instances Elastic Load Balancing VPC Subnet VPC Subnet Availability Zone Internet Gateway RDS Instance ElastiCache Standby (MAZ) Cache Node VPC Subnet Web/App Instances Elastic Load Balancing VPC Subnet Availability Zone Cache Instances Elastic Load Balancing VPC Subnet
  76. 76. Memcached – Code Sample function retrieveValue($query) { $queryId = md5($query); if ($myValue = $memcache->get($queryId)) { return $myValue; } else { $myValue = dbfetch($query); $memcache->set($queryId, $myValue); return $myValue; } }
  77. 77. MySQL 5.6 + Memcached RDS MySQL supports version 5.6 with integrated Memcached on the instance: – Part of the InnoDB engine – Memcached running as part of MySQL talks directly to data in InnoDB tables, essentially turning MySQL into a fast “key-value store” – From the opposite view point, adds persistence to Memcached – Same Memcached API as standalone https://dev.mysql.com/doc/refman/5.6/en/innodb-memcached-intro.html
  78. 78. AWS Marketplace & Partners Can Help • Customers can find, research, buy software. • Simple pricing, aligns with Amazon EC2 usage model. • Launch in minutes! • Marketplace billing integrated into your AWS account. • 1100+ products across 24+ categories. Learn more at: aws.amazon.com/marketplace
  79. 79. Back To Our Experiment! https://secure.flickr.com/photos/toyochin/1382531438/
  80. 80. Experiment Infrastructure (original) Elastic IP Amazon RDS DB Instance Web/App Instance Clients
  81. 81. Results? (not pretty) 14.7 second responses Low RPM Mostly Static Data
  82. 82. Experiment Infrastructure (with cake) • Added in ElastiCache Memcached cache.m1.large • Added in Amazon CloudFront for static content – Tuned expires and cache headers in Apache • Added in APC for PHP caching – Increased memory to 128Mb, no other changes • Did nothing to the DB • Drupal memcached & CDN modules
  83. 83. Experiment Infrastructure (with cake) CloudFront Elastic IP Amazon RDS DB Instance Web/App Instance Clients ElastiCache Cache Node
  84. 84. Results? (AWESOME!) response time ~1/2 ~2x RPM Repeat Page view from 15.8sec to 2.75sec
  85. 85. Easy as Pie! https://secure.flickr.com/photos/linecon0/2654865842/
  86. 86. Toronto Star • • • • Canada’s largest daily newspaper Focused on metro Toronto 3.3 million monthly unique visitors Small in-house digital group, supported by vendors & corporate IT • Digital group run as “startup” within corporate structure • Tech stack includes Java, PHP, Ruby, Python
  87. 87. mystar
  88. 88. Wheels.ca
  89. 89. Toronto.com
  90. 90. Thegridto.com
  91. 91. Why Cache? • • • • Performance Scalability Reliability Cost
  92. 92. Breaking News
  93. 93. Breaking News: Client View
  94. 94. Breaking News: Server View
  95. 95. The Caching Onion
  96. 96. DB App Reverse proxy Edge
  97. 97. Edge – The Onion Skin • Examples: Akamai, CloudFront • Static Asset caching • Full site caching • Origins • Behaviors • thestar.com, wheels.ca, toronto.com, thegridto.com Edge
  98. 98. Edge – The Onion Skin Cloudfront Elastic Load Balancer Production - US East Region Thestar.com architecture VPC Dispatcher/Apache Amazon Linux 64 Bit C1 Medium Publish CQ/CRX Publish CQ/CRX Amazon Linux 64 Bit Amazon Linux 64 Bit C1-Xlarge C1-Xlarge Dispatcher/Apache Amazon Linux 64 Bit C1 Medium Publish CQ/CRX Amazon Linux 64 Bit C1-Xlarge Publish CQ/CRX Amazon Linux 64 Bit C1-Xlarge Dispatcher/Apache Amazon Linux 64 Bit C1 Medium Publish CQ/CRX Publish CQ/CRX Amazon Linux 64 Bit Amazon Linux 64 Bit C1-Xlarge C1-Xlarge Standby Master Author CQ/CRX Amazon Linux 64 Bit C1-Xlarge Availability Zone A Author CQ/CRX Amazon Linux 64 Bit C1-Xlarge Availability Zone B Availability Zone D
  99. 99. Cloudfront Elastic Load Balancer Production - US East Region VPC Dispatcher/Apache Amazon Linux 64 Bit C1 Medium Publish CQ/CRX Publish CQ/CRX Amazon Linux 64 Bit Amazon Linux 64 Bit C1-Xlarge C1-Xlarge Dispatcher/Apache Amazon Linux 64 Bit C1 Medium Publish CQ/CRX Amazon Linux 64 Bit C1-Xlarge Standby Publish CQ/CRX Amazon Linux 64 Bit C1-Xlarge Dispatcher/Apache Amazon Linux 64 Bit C1 Medium Publish CQ/CRX Publish CQ/CRX Amazon Linux 64 Bit Amazon Linux 64 Bit C1-Xlarge C1-Xlarge
  100. 100. Whole Site Delivery • • • • • Cache everything possible No server side cookies written, only select pages pass query strings Control caching granularly using 19 different rules We use a single origin, but Elastic Load Balancing and multi-tiered multi-AZ configuration on backend Planning a multi-region DR architecture that will also leverage Amazon CloudFront
  101. 101. Reverse Proxy / Web Accelerator • Examples: Varnish, Nginx, Apache • Serves “static” content • Reduce load on app server • wheels.ca, mystar, thestar.com, thegridto.com Reverse proxy
  102. 102. Reverse Proxy / Web Accelerator mystar
  103. 103. Reverse Proxy / Web Accelerator mystar
  104. 104. Reverse Proxy / Web Accelerator Wheels.ca
  105. 105. App-level caching • Examples: Memcached, Reddis – ElastiCache now supports both • Session caching • View caching • mystar App
  106. 106. App-level caching
  107. 107. Database caching • Example: Memcached • Cache frequent SQL queries • Reduce DB server load, response time • mystar, wheels.ca DB
  108. 108. Database Caching Wheels.ca
  109. 109. Database Caching Wheels.ca
  110. 110. Thanks! Ping me sevans@thestar.ca @stephenaevans
  111. 111. Please give us your feedback on this presentation SVC203 As a thank you, we will select prize winners daily for completed surveys!
  112. 112. APPENDIX
  113. 113. Reverse Proxy / Web Accelerator Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. You install it in front of any server that speaks HTTP and configure it to cache the contents. Varnish Cache is really, really fast. It typically speeds up delivery with a factor of 300 - 1000x, depending on your architecture. - varnish-cache.org
  114. 114. Reverse Proxy / Web Accelerator Nginx (pronounced "engine x") is an open source reverse proxy server for HTTP, HTTPS, SMTP, POP3, and IMAP protocols, as well as a load balancer, HTTP cache, and a web server (origin server). The nginx project started with a strong focus on high concurrency, high performance and low memory usage. - Wikipedia
  115. 115. App-level caching Memcached Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load. Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering
  116. 116. App-level caching Redis Redis is an open source, BSD licensed, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. – Redis.io
  117. 117. Edge – The Onion Skin “A content delivery network is a large distributed system of servers deployed in multiple data centers across the Internet. The goal of a CDN is to serve content to end-users with high availability and high performance.” - Wikipedia
  118. 118. Reverse Proxy / Web Accelerator “A web accelerator is a proxy server that reduces web site access times. They can be a self-contained hardware appliance or installable software.” - Wikipedia
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×