Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Extending your applications to the edge with CDNs

3,065 views

Published on

CDNs have traditionally been considered "black box" services with lack of control and visibility. Modern applications and DevOps culture require more flexibility, customization, and insight into your infrastructure, and how content is being served. We've grown accustomed to using services through programmable interfaces with instant feedback and full visibility, and CDNs should not be exempt from these requirements. In this talk, we'll explore how you can use content delivery networks as an extension of your applications, with full control, flexibility, and transparency at the network edge.

Hooman Beheshti is VP of Technology at Fastly, where he develops web performance services for the world's smartest CDN platform. A pioneer in the application acceleration space, Hooman helped design one of the original load balancers while at Radware and has held senior technology positions with Strangeloop Networks and Crescendo Networks. He has worked on the core technologies that make the Internet work faster for nearly 20 years and is an expert and frequent speaker on the subjects of load balancing, application performance, and content delivery networks.

Published in: Internet
  • Be the first to comment

Extending your applications to the edge with CDNs

  1. 1. Extending  Applica/ons  To  The  Edge   With  CDNs   Hooman  Behesh/,  VP  Technology   Webperf  meetup  NYC  –  May  12  2015  
  2. 2. What  is  a  CDN   •  Global  network  of  HTTP  caches   – Get  content  closer  to  users   •  Performance  benefits   •  offload  
  3. 3. In  the  past…  
  4. 4. Classic  CDNs   •  Black  box  service   •  Making  changes  sucked!   •  LiRle  to  no  real-­‐/me  interfaces   – APIs  or  configura/on  management   •  Lots  of  professional  services   •  Decent  caching,  terrible  uncaching  
  5. 5. Cacheable   Uncacheable   Sta3c  Dynamic   How  classic  CDNs  look  at  your  content  
  6. 6. Cache-Control: max-age=31536000
  7. 7. Classic  CDNs:  dynamic  content   •  Focused  on  delivery:  transport  op/miza/on  
  8. 8. But  some3mes…  
  9. 9. I  know  you’re  calling  it   dynamic,  but  it’s  kind   of  sta3c  some3mes  
  10. 10. I  know  you’re  calling  it   dynamic,  but  it’s  kind   of  sta3c  some3mes   That’s  cool,  how  long   is  it  sta3c  for?  
  11. 11. That’s  cool,  how  long   is  it  sta3c  for?  Some3mes  a  liFle  bit,   some3mes  a  lot  
  12. 12. Some3mes  a  liFle  bit,   some3mes  a  lot   Neat!  
  13. 13. Some3mes  a  liFle  bit,   some3mes  a  lot   You  look  great  today!  
  14. 14. You  look  great  today!   Don’t  change  the   subject!    How  do  I   cache  it?  
  15. 15. Don’t  change  the   subject!    How  do  I   cache  it?   Cache  it  for  half  an   hour?  
  16. 16. Cache  it  for  half  an   hour?  Some3mes  it  changes   much  faster  
  17. 17. Some3mes  it  changes   much  faster   Cache  it  for  10   minutes?  
  18. 18. Cache  it  for  10   minutes?  Are  you  just  guessing   right  now?  
  19. 19. Are  you  just  guessing   right  now?   Just  trying  to  figure   out  your  comfort   level  with  mistakes!  
  20. 20. Expiry  guessing!  
  21. 21. Terrible  Caching  
  22. 22. Terrible  Caching   Terrible  Anxiety   or   (ex:  news  story)  
  23. 23. Things  are  different  today  
  24. 24. Today…   •  Influenced  by  clouds   •  Devops  requires  programmability  and  control   •  CDNs  should  be  part  of  the  applica/on   •  Real-­‐/me  integra/on   –  Changes  and  APIs   •  Unpredictable  content  expiry   •  Your  content  à  you  need  control    
  25. 25. What  does  this  all  mean?  
  26. 26. Control   Visibility   Cacheability  
  27. 27. Control   Visibility   Cacheability  
  28. 28. Cacheability   •  Granularity  in  caching  rules  (i.e.  cache  the  way   you  want)   –  Customizable  cache  keys   –  Ex:  geo-­‐ip  caching   –  Ex:  normalizing  user  agents   •  Cache  everything   –  Expect  high  cache-­‐hit  ra/os   –  No  cache-­‐control  anxiety!  
  29. 29. Cacheability   •  Granularity  in  caching  rules  (i.e.  cache  the  way   you  want)   –  Customizable  cache  keys   –  Ex:  geo-­‐ip  caching   –  Ex:  normalizing  user  agents   •  Cache  everything   –  Expect  high  cache-­‐hit  ra/os   –  No  cache-­‐control  anxiety!  
  30. 30. Cache  hit  ra3os  
  31. 31. Cache  Hit  Ra3o:  Tradi3onal  calcula3on   1  -­‐     Requests  to  Origin     Total  Requests  
  32. 32. Origin  
  33. 33. Origin   Cache          
  34. 34. TCP   Origin   Cache          
  35. 35. HTTP   Origin   Cache          
  36. 36. Origin   Cache           HTTP  
  37. 37. Origin   Cache           HTTP  
  38. 38. Origin   Cache           HTTP  
  39. 39. Origin   Cache           HOT   COLD  
  40. 40. Origin   Cache           cache  “hit”  
  41. 41. Cache  Hit  Ra3o:  Tradi3onal  calcula3on   1  -­‐     Requests  to  Origin     Total  Requests  
  42. 42. Isn’t  this  beFer?   Hits   Total  Requests   @edge  
  43. 43. Isn’t  this  beFer?   Hits   Hits  +  Misses   @edge  
  44. 44. Cache  Hit  Ra3o   vs.  1  -­‐    Requests  to  Origin     Total  Requests   Hits   Hits  +  Misses   @edge  
  45. 45. Cache  Hit  Ra3o   vs.  1  -­‐    Requests  to  Origin     Total  Requests   Hits   Hits  +  Misses   @edge   Offload  
  46. 46. Cache  Hit  Ra3o   vs.  1  -­‐    Requests  to  Origin     Total  Requests   Hits   Hits  +  Misses   @edge   Offload   Performance  
  47. 47. Effect  on  long  tail  content  
  48. 48. Long  Tail:  Cacheable  but  seldom  fetched  
  49. 49. Popular   Medium  Tail  (1hr)   Long  tail    (6hr)                  
  50. 50. Popular   Medium  Tail  (1hr)   Long  tail    (6hr)               Connect  (median)   Popular   14msec   1hr  Tail   15msec   6hr  Tail   16msec      
  51. 51. Popular   Medium  Tail  (1hr)   Long  tail    (6hr)   Connect  (median)   Popular   14msec   1hr  Tail   15msec   6hr  Tail   16msec  
  52. 52. Popular   Medium  Tail  (1hr)   Long  tail    (6hr)   Connect  (median)   Wait  (median)   Popular   14msec   19msec   1hr  Tail   15msec   26msec   6hr  Tail   16msec   32msec  
  53. 53. Popular   Medium  Tail  (1hr)   Long  tail    (6hr)   Connect  (median)   Wait  (median)   Popular   14msec   19msec   1hr  Tail   15msec   26msec   6hr  Tail   16msec   32msec   6,400+  measurements   77,000+  measurements   38,000+  measurements  
  54. 54. Popular   Medium  Tail  (1hr)   Long  tail    (6hr)   Isn’t  this  beFer?  
  55. 55. “Dynamic”  Content  
  56. 56. We’ve  been  told:   Sta3c  =  Cacheable   Dynamic  =  Uncacheable  
  57. 57. Cacheable   Uncacheable   Sta3c   Dynamic   How  our  sites  used  to  look  
  58. 58. Cacheable   Uncacheable   Sta3c  Dynamic   How  we’re  told  our  sites  look  
  59. 59. Cacheable   Uncacheable   Sta3c   Dynamic   How  our  sites  actually  look   Event   Driven  
  60. 60. Event-­‐Driven  content:   Sta3c  for  unpredictable  periods  of  3me    
  61. 61. Three  types  of  content   •  Sta/c   –  Infrequent  changes,  can  cache  for  a  long  /me   –  Cache-­‐control  headers  are  good  enough   –  Examples:  Images,  javascript  ,  css,  etc   •  Dynamic   –  Can’t  cache  at  all    -­‐  must  go  to  origin   –  DSA  and  transport  is  key  for  performance   –  Examples:  Logins,  credit  card  transac/ons,  Ajax,  etc   •  Event-­‐driven   –  Sta/c,  but  unpredictably   –  Cache-­‐control  headers  are  not  good  enough   –  Examples:  That  news  story,  wiki  pages,  user  comments,  sports  scores,  etc  
  62. 62. Three  types  of  content   •  Sta/c   –  Infrequent  changes,  can  cache  for  a  long  /me   –  Cache-­‐control  headers  are  good  enough   –  Examples:  Images,  javascript  ,  css,  etc   •  Dynamic   –  Can’t  cache  at  all    -­‐  must  go  to  origin   –  DSA  and  transport  is  key  for  performance   –  Examples:  Logins,  credit  card  transac/ons,  Ajax,  etc   •  Event-­‐driven   –  Sta/c,  but  unpredictably   –  Cache-­‐control  headers  are  not  good  enough   –  Examples:  That  news  story,  wiki  pages,  user  comments,  sports  scores,  etc  
  63. 63. Three  types  of  content   •  Sta/c   –  Infrequent  changes,  can  cache  for  a  long  /me   –  Cache-­‐control  headers  are  good  enough   –  Examples:  Images,  javascript  ,  css,  etc   •  Dynamic   –  Can’t  cache  at  all    -­‐  must  go  to  origin   –  DSA  and  transport  is  key  for  performance   –  Examples:  Logins,  credit  card  transac/ons,  Ajax,  etc   •  Event-­‐driven   –  Sta/c,  but  unpredictably   –  Cache-­‐control  headers  are  not  good  enough   –  Examples:  That  news  story,  wiki  pages,  user  comments,  sports  scores,  etc  
  64. 64. Caching  event-­‐driven  content   •  Not  easy   – Classic  CDNs  con/nue  to  consider  it  the  same  as   dynamic  (Bad  caching  or  panic!)   – Need  the  right  tools  to  cache   •  You  may  not  know  expiry  ahead  of  /me,  but   your  app  knows  when  things  change  
  65. 65. Cache  normally  and  then      “uncache”  when  it  changes  
  66. 66. Uncaching!   •  Purging   – Accessible  through  API   – Granular   •  Dependencies  through  labels,  tags,  or  surrogate  keys   – Must  be  instantaneous   •  Ex:  news  story,  part  2    
  67. 67. Instant  Purge   •  Enables  caching  of  event-­‐driven  content   –  Can  deal  with  unpredictable  invalida/on   –  As  pages  update/publish,  purge  command  is  sent  to  CDN   •  Instant?   –  Instant  ≠    12  minutes!   –  Instant  ≠    quick  acknowledgement!   –  <1sec   –  Determinis/c  and  predictable  
  68. 68. Example:   CMS  +  Purge  
  69. 69. WordPress:  Before   CDN  Node  
  70. 70. WordPress:  Before   CDN  Node  
  71. 71. WordPress:  Before   CDN  Node  
  72. 72. WordPress:  Before   CDN  Node  
  73. 73. WordPress:  Before   CDN  Node   Cache  
  74. 74. WordPress:  Ajer   CDN  Node  
  75. 75. WordPress:  Ajer   CDN  Node   HTTP/1.1 200 OK Content-Type: text/html Content-Length: 55,666 Cache-Control: Totally Long Time!
  76. 76. WordPress:  Ajer   CDN  Node   HTTP/1.1 200 OK Content-Type: text/html Content-Length: 55,666 Cache-Control: Totally Long Time!
  77. 77. WordPress:  Ajer   CDN  Node  
  78. 78. WordPress:  Ajer   CDN  Node  
  79. 79. WordPress:  Ajer   CDN  Node   PURGE  
  80. 80. WordPress:  Ajer   CDN  Node   PURGE  
  81. 81. WordPress:  Ajer   CDN  Node   (Has  to  be  instantaneous!)   PURGE  
  82. 82. WordPress:  Ajer   CDN  Node   HTTP/1.1 200 OK Content-Type: text/html Content-Length: 55,666 Cache-Control: Totally Long Time!
  83. 83. WordPress:  Ajer   CDN  Node  
  84. 84. Example:   customer1.js  
  85. 85. Before   CDN  Node   Origin   http://www.3rdparty.com/customer1.js (Referer: www.customer1.com)
  86. 86. Before   CDN  Node   Origin   http://www.3rdparty.com/customer1.js
  87. 87. Before   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60 Last-Modified: Tue, 12 May 2015 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js
  88. 88. Before   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60 Last-Modified: Tue, 12 May 2015 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js
  89. 89. Before   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60 Last-Modified: Tue, 12 May 2015 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js
  90. 90. Before   CDN  Node   Origin   http://www.3rdparty.com/customer1.js (Ajer  1  min)  
  91. 91. Before   CDN  Node   Origin   http://www.3rdparty.com/customer1.js VALIDATION If-Modified-Since: Tue, 12 May 2015 19:51:30 GMT (Ajer  1  min)  
  92. 92. Before   CDN  Node   Origin   http://www.3rdparty.com/customer1.js 304 Not Modified (Ajer  1  min)  
  93. 93. Before   CDN  Node   Origin   http://www.3rdparty.com/customer1.js 304 Not Modified (Ajer  1  min)  
  94. 94. Ajer   CDN  Node   Origin   http://www.3rdparty.com/customer1.js
  95. 95. Ajer   CDN  Node   Origin   http://www.3rdparty.com/customer1.js
  96. 96. Ajer   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Tue, 12 May 2015 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js
  97. 97. Ajer   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Tue, 12 May 2015 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js
  98. 98. Ajer   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Tue, 12 May 2015 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js
  99. 99. Ajer   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Tue, 12 May 2015 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js
  100. 100. Ajer   CDN  Node   Origin   http://www.3rdparty.com/customer1.js (Ajer  1  min)  
  101. 101. Ajer   CDN  Node   Origin   http://www.3rdparty.com/customer1.js (Ajer  1  min)  
  102. 102. This  happens  many  many  3mes!   (many  many  happy  visitors!)  
  103. 103. Ajer   CDN  Node   Origin   Customer1   changes  config   (Ajer  1  min)  
  104. 104. Ajer   CDN  Node   Origin   PURGE  customer1.js   Customer1   changes  config   (Ajer  1  min)  
  105. 105. Ajer   CDN  Node   Origin   PURGE  customer1.js   Customer1   changes  config   (Ajer  1  min)  
  106. 106. Ajer   CDN  Node   Origin   http://www.3rdparty.com/customer1.js (Ajer  1  min)  
  107. 107. Ajer   CDN  Node   Origin   http://www.3rdparty.com/customer1.js HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 (Ajer  1  min)  
  108. 108. Ajer   CDN  Node   Origin   HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js (Ajer  1  min)  
  109. 109. Control   Visibility   Cacheability  
  110. 110. Programmability   •  Granular  Invalida/on  API   •  Configura/on  API   •  Logic  at  the  Edge   •  All  must  be  instantaneous  and  real  /me  
  111. 111. Control  at  the  Edge   •  Moving  app  logic  to  the  edge   •  Ex:  VCL  (Varnish  Configura/on  Language)   – Script-­‐like  configura/on  for  func/onality  at  the   edge   •  Any  mechanism  offered  by  CDN  to  let  you  do   stuff  at  the  edge  is  good  
  112. 112. Logic  at  the  Edge:  Examples     •  Generate  content  at  the  edge   •  HTTP  header  manipula/on   •  Origin  selec/on   •  Caching  rules   •  Geo-­‐IP  rules   •  Forcing  SSL   •  Serving  stale  content     –  stale-­‐while-­‐revalidate,  stale-­‐while-­‐error   •  Etc…  
  113. 113. Example:   Edge-­‐generated  Content  
  114. 114. JSON  Data  Center  ID   CDN  Node   Origin   http://www.site.com/which_datacenter.js
  115. 115. JSON  Data  Center  ID   CDN  Node   Origin   { ‘datacenter’ : ‘LAX’ } http://www.site.com/which_datacenter.js
  116. 116. VCL  Snippet  
  117. 117. JSONP  Geo  IP   CDN  Node   Origin   myFunction( { “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237” }); http://www.site.com/geo_ip.js?jsonp=myFunction
  118. 118. JSONP  Geo  IP   CDN  Node   Origin   myFunction( { “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237” }); http://www.site.com/geo_ip.js?jsonp=myFunction
  119. 119. JSONP  Geo  IP   CDN  Node   Origin   somethingElse( { “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237” }); http://www.site.com/geo_ip.js?jsonp=somethingElse
  120. 120. Control   Visibility   Cacheability  
  121. 121. Real-­‐3me  Analy3cs   •  Stats  API   •  Network  stats   •  HTTP  stats  (status  codes,  etc)   •  Caching  stats  (hits,  misses,  errors,  etc)   •  All  have  to  be  real-­‐/me   –  Historic  data  is  also  a  must  for  long  term  stats  analysis  
  122. 122. Logging   •  Real  /me!   •  Log  streaming   – To  any  logging  des/na/on  endpoint   – Ex:  S3,  FTP,  syslog,  etc  
  123. 123. Example:   Beacon  Termina3on  at  the  Edge  
  124. 124. Before   CDN  Node   Origin   Log  Analysis   http://collector.site.com/beacon.img?a=1&b=2&c=3
  125. 125. Before   CDN  Node   Origin   Log  Analysis   http://collector.site.com/beacon.img?a=1&b=2&c=3
  126. 126. Before   CDN  Node   Origin   Log  Analysis   HTTP/1.1 200 OK Pragma: no-cache Expires: Wed, 19 Apr 2000 11:43:00 GMT Cache-Control: no-cache, no-store Last-Modified: Wed, 21 Jan 2004 19:51:30 GMT Content-Type: image/gif Date: Fri, 20 Jun 2014 12:22:20 GMT Server: Apache Content-Length: 35 http://collector.site.com/beacon.img?a=1&b=2&c=3
  127. 127. Before   CDN  Node   Origin   Log  Analysis   HTTP/1.1 200 OK Pragma: no-cache Expires: Wed, 19 Apr 2000 11:43:00 GMT Cache-Control: no-cache, no-store Last-Modified: Wed, 21 Jan 2004 19:51:30 GMT Content-Type: image/gif Date: Fri, 20 Jun 2014 12:22:20 GMT Server: Apache Content-Length: 35 http://collector.site.com/beacon.img?a=1&b=2&c=3
  128. 128. Ajer   CDN  Node   Origin   http://collector.site.com/beacon.img?a=1&b=2&c=3
  129. 129. Ajer   CDN  Node   Origin   HTTP/1.1 200 OK Pragma: no-cache Expires: Wed, 19 Apr 2000 11:43:00 GMT Cache-Control: no-cache, no-store Last-Modified: Wed, 21 Jan 2004 19:51:30 GMT Content-Type: image/gif Date: Fri, 20 Jun 2014 12:22:20 GMT Server: Apache Content-Length: 35 http://collector.site.com/beacon.img?a=1&b=2&c=3
  130. 130. Ajer   CDN  Node   Origin   HTTP/1.1 204 No Content Date: Sat, 21 Jun 2014 23:21:12 GMT Server: Awesome Server Content-Length: 0 http://collector.site.com/beacon.img?a=1&b=2&c=3
  131. 131. Ajer   CDN  Node   Origin   Syslog  /  S3  /  FTP/etc   http://collector.site.com/beacon.img?a=1&b=2&c=3
  132. 132. Summary   •  CDNs  should  be  an  extension  of  your   applica/ons:   –  Flexible  caching,  predictable  edge  caching,  and  instant   purging  to  un-­‐cache!   –  Control  at  the  edge   –  Real-­‐/me  visibility  and  analy/cs   •  Full  real-­‐/me  programma/c  interface  is  a  must    
  133. 133. Thank  you!   hooman@fastly.com  

×