Extending your applications to the edge with CDNs

1. Extending Applica/ons To The Edge With CDNs Hooman Behesh/, VP Technology Webperf meetup NYC – May 12 2015

2. What is a CDN •  Global network of HTTP caches – Get content closer to users •  Performance beneﬁts •  oﬄoad

3. In the past…

4. Classic CDNs •  Black box service •  Making changes sucked! •  LiRle to no real-‐/me interfaces – APIs or conﬁgura/on management •  Lots of professional services •  Decent caching, terrible uncaching

5. Cacheable Uncacheable Sta3c Dynamic How classic CDNs look at your content

6. Cache-Control: max-age=31536000

7. Classic CDNs: dynamic content •  Focused on delivery: transport op/miza/on

8. But some3mes…

9. I know you’re calling it dynamic, but it’s kind of sta3c some3mes

10. I know you’re calling it dynamic, but it’s kind of sta3c some3mes That’s cool, how long is it sta3c for?

11. That’s cool, how long is it sta3c for? Some3mes a liFle bit, some3mes a lot

12. Some3mes a liFle bit, some3mes a lot Neat!

13. Some3mes a liFle bit, some3mes a lot You look great today!

14. You look great today! Don’t change the subject! How do I cache it?

15. Don’t change the subject! How do I cache it? Cache it for half an hour?

16. Cache it for half an hour? Some3mes it changes much faster

17. Some3mes it changes much faster Cache it for 10 minutes?

18. Cache it for 10 minutes? Are you just guessing right now?

19. Are you just guessing right now? Just trying to ﬁgure out your comfort level with mistakes!

20. Expiry guessing!

21. Terrible Caching

22. Terrible Caching Terrible Anxiety or (ex: news story)

23. Things are diﬀerent today

24. Today… •  Inﬂuenced by clouds •  Devops requires programmability and control •  CDNs should be part of the applica/on •  Real-‐/me integra/on –  Changes and APIs •  Unpredictable content expiry •  Your content à you need control

25. What does this all mean?

26. Control Visibility Cacheability

28. Cacheability •  Granularity in caching rules (i.e. cache the way you want) –  Customizable cache keys –  Ex: geo-‐ip caching –  Ex: normalizing user agents •  Cache everything –  Expect high cache-‐hit ra/os –  No cache-‐control anxiety!

29. Cacheability •  Granularity in caching rules (i.e. cache the way you want) –  Customizable cache keys –  Ex: geo-‐ip caching –  Ex: normalizing user agents •  Cache everything –  Expect high cache-‐hit ra/os –  No cache-‐control anxiety!

30. Cache hit ra3os

31. Cache Hit Ra3o: Tradi3onal calcula3on 1 -‐ Requests to Origin Total Requests

32. Origin

33. Origin Cache

34. TCP Origin Cache

35. HTTP Origin Cache

36. Origin Cache HTTP

39. Origin Cache HOT COLD

40. Origin Cache cache “hit”

41. Cache Hit Ra3o: Tradi3onal calcula3on 1 -‐ Requests to Origin Total Requests

42. Isn’t this beFer? Hits Total Requests @edge

43. Isn’t this beFer? Hits Hits + Misses @edge

44. Cache Hit Ra3o vs. 1 -‐ Requests to Origin Total Requests Hits Hits + Misses @edge

45. Cache Hit Ra3o vs. 1 -‐ Requests to Origin Total Requests Hits Hits + Misses @edge Oﬄoad

46. Cache Hit Ra3o vs. 1 -‐ Requests to Origin Total Requests Hits Hits + Misses @edge Oﬄoad Performance

47. Eﬀect on long tail content

48. Long Tail: Cacheable but seldom fetched

49. Popular Medium Tail (1hr) Long tail (6hr)

50. Popular Medium Tail (1hr) Long tail (6hr) Connect (median) Popular 14msec 1hr Tail 15msec 6hr Tail 16msec

51. Popular Medium Tail (1hr) Long tail (6hr) Connect (median) Popular 14msec 1hr Tail 15msec 6hr Tail 16msec

52. Popular Medium Tail (1hr) Long tail (6hr) Connect (median) Wait (median) Popular 14msec 19msec 1hr Tail 15msec 26msec 6hr Tail 16msec 32msec

53. Popular Medium Tail (1hr) Long tail (6hr) Connect (median) Wait (median) Popular 14msec 19msec 1hr Tail 15msec 26msec 6hr Tail 16msec 32msec 6,400+ measurements 77,000+ measurements 38,000+ measurements

54. Popular Medium Tail (1hr) Long tail (6hr) Isn’t this beFer?

55. “Dynamic” Content

56. We’ve been told: Sta3c = Cacheable Dynamic = Uncacheable

57. Cacheable Uncacheable Sta3c Dynamic How our sites used to look

58. Cacheable Uncacheable Sta3c Dynamic How we’re told our sites look

59. Cacheable Uncacheable Sta3c Dynamic How our sites actually look Event Driven

60. Event-‐Driven content: Sta3c for unpredictable periods of 3me

61. Three types of content •  Sta/c –  Infrequent changes, can cache for a long /me –  Cache-‐control headers are good enough –  Examples: Images, javascript , css, etc •  Dynamic –  Can’t cache at all -‐ must go to origin –  DSA and transport is key for performance –  Examples: Logins, credit card transac/ons, Ajax, etc •  Event-‐driven –  Sta/c, but unpredictably –  Cache-‐control headers are not good enough –  Examples: That news story, wiki pages, user comments, sports scores, etc

64. Caching event-‐driven content •  Not easy – Classic CDNs con/nue to consider it the same as dynamic (Bad caching or panic!) – Need the right tools to cache •  You may not know expiry ahead of /me, but your app knows when things change

65. Cache normally and then “uncache” when it changes

66. Uncaching! •  Purging – Accessible through API – Granular •  Dependencies through labels, tags, or surrogate keys – Must be instantaneous •  Ex: news story, part 2

67. Instant Purge •  Enables caching of event-‐driven content –  Can deal with unpredictable invalida/on –  As pages update/publish, purge command is sent to CDN •  Instant? –  Instant ≠ 12 minutes! –  Instant ≠ quick acknowledgement! –  <1sec –  Determinis/c and predictable

68. Example: CMS + Purge

69. WordPress: Before CDN Node

73. WordPress: Before CDN Node Cache

74. WordPress: Ajer CDN Node

75. WordPress: Ajer CDN Node HTTP/1.1 200 OK Content-Type: text/html Content-Length: 55,666 Cache-Control: Totally Long Time!

79. WordPress: Ajer CDN Node PURGE

80. WordPress: Ajer CDN Node PURGE

81. WordPress: Ajer CDN Node (Has to be instantaneous!) PURGE

84. Example: customer1.js

85. Before CDN Node Origin http://www.3rdparty.com/customer1.js (Referer: www.customer1.com)

86. Before CDN Node Origin http://www.3rdparty.com/customer1.js

87. Before CDN Node Origin HTTP/1.1 200 OK Cache-Control: max-age=60 Last-Modified: Tue, 12 May 2015 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js

90. Before CDN Node Origin http://www.3rdparty.com/customer1.js (Ajer 1 min)

91. Before CDN Node Origin http://www.3rdparty.com/customer1.js VALIDATION If-Modified-Since: Tue, 12 May 2015 19:51:30 GMT (Ajer 1 min)

92. Before CDN Node Origin http://www.3rdparty.com/customer1.js 304 Not Modified (Ajer 1 min)

93. Before CDN Node Origin http://www.3rdparty.com/customer1.js 304 Not Modified (Ajer 1 min)

94. Ajer CDN Node Origin http://www.3rdparty.com/customer1.js

95. Ajer CDN Node Origin http://www.3rdparty.com/customer1.js

96. Ajer CDN Node Origin HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Tue, 12 May 2015 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js

100. Ajer CDN Node Origin http://www.3rdparty.com/customer1.js (Ajer 1 min)

102. This happens many many 3mes! (many many happy visitors!)

103. Ajer CDN Node Origin Customer1 changes conﬁg (Ajer 1 min)

104. Ajer CDN Node Origin PURGE customer1.js Customer1 changes conﬁg (Ajer 1 min)

105. Ajer CDN Node Origin PURGE customer1.js Customer1 changes conﬁg (Ajer 1 min)

107. Ajer CDN Node Origin http://www.3rdparty.com/customer1.js HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 (Ajer 1 min)

108. Ajer CDN Node Origin HTTP/1.1 200 OK Cache-Control: max-age=60, s-maxage=2592000 Last-Modified: Wed, 24 Sep 2014 19:51:30 GMT Content-Type: application/javascript Date: Thu, 25 Sep 2014 12:22:20 GMT Server: Apache Content-Length: 7835 http://www.3rdparty.com/customer1.js (Ajer 1 min)

110. Programmability •  Granular Invalida/on API •  Conﬁgura/on API •  Logic at the Edge •  All must be instantaneous and real /me

111. Control at the Edge •  Moving app logic to the edge •  Ex: VCL (Varnish Configura/on Language) – Script-‐like configura/on for func/onality at the edge •  Any mechanism offered by CDN to let you do stuff at the edge is good

112. Logic at the Edge: Examples •  Generate content at the edge •  HTTP header manipula/on •  Origin selec/on •  Caching rules •  Geo-‐IP rules •  Forcing SSL •  Serving stale content –  stale-‐while-‐revalidate, stale-‐while-‐error •  Etc…

113. Example: Edge-‐generated Content

114. JSON Data Center ID CDN Node Origin http://www.site.com/which_datacenter.js

115. JSON Data Center ID CDN Node Origin { ‘datacenter’ : ‘LAX’ } http://www.site.com/which_datacenter.js

116. VCL Snippet

117. JSONP Geo IP CDN Node Origin myFunction( { “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237” }); http://www.site.com/geo_ip.js?jsonp=myFunction

118. JSONP Geo IP CDN Node Origin myFunction( { “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237” }); http://www.site.com/geo_ip.js?jsonp=myFunction

119. JSONP Geo IP CDN Node Origin somethingElse( { “city” : “New York”, “state”: “New York”, “country”: “United States”, “ip”: “173.18.14.237” }); http://www.site.com/geo_ip.js?jsonp=somethingElse

121. Real-‐3me Analy3cs •  Stats API •  Network stats •  HTTP stats (status codes, etc) •  Caching stats (hits, misses, errors, etc) •  All have to be real-‐/me –  Historic data is also a must for long term stats analysis

122. Logging •  Real /me! •  Log streaming – To any logging des/na/on endpoint – Ex: S3, FTP, syslog, etc

123. Example: Beacon Termina3on at the Edge

124. Before CDN Node Origin Log Analysis http://collector.site.com/beacon.img?a=1&b=2&c=3

125. Before CDN Node Origin Log Analysis http://collector.site.com/beacon.img?a=1&b=2&c=3

126. Before CDN Node Origin Log Analysis HTTP/1.1 200 OK Pragma: no-cache Expires: Wed, 19 Apr 2000 11:43:00 GMT Cache-Control: no-cache, no-store Last-Modified: Wed, 21 Jan 2004 19:51:30 GMT Content-Type: image/gif Date: Fri, 20 Jun 2014 12:22:20 GMT Server: Apache Content-Length: 35 http://collector.site.com/beacon.img?a=1&b=2&c=3

127. Before CDN Node Origin Log Analysis HTTP/1.1 200 OK Pragma: no-cache Expires: Wed, 19 Apr 2000 11:43:00 GMT Cache-Control: no-cache, no-store Last-Modified: Wed, 21 Jan 2004 19:51:30 GMT Content-Type: image/gif Date: Fri, 20 Jun 2014 12:22:20 GMT Server: Apache Content-Length: 35 http://collector.site.com/beacon.img?a=1&b=2&c=3

128. Ajer CDN Node Origin http://collector.site.com/beacon.img?a=1&b=2&c=3

129. Ajer CDN Node Origin HTTP/1.1 200 OK Pragma: no-cache Expires: Wed, 19 Apr 2000 11:43:00 GMT Cache-Control: no-cache, no-store Last-Modified: Wed, 21 Jan 2004 19:51:30 GMT Content-Type: image/gif Date: Fri, 20 Jun 2014 12:22:20 GMT Server: Apache Content-Length: 35 http://collector.site.com/beacon.img?a=1&b=2&c=3

130. Ajer CDN Node Origin HTTP/1.1 204 No Content Date: Sat, 21 Jun 2014 23:21:12 GMT Server: Awesome Server Content-Length: 0 http://collector.site.com/beacon.img?a=1&b=2&c=3

131. Ajer CDN Node Origin Syslog / S3 / FTP/etc http://collector.site.com/beacon.img?a=1&b=2&c=3

132. Summary •  CDNs should be an extension of your applica/ons: –  Flexible caching, predictable edge caching, and instant purging to un-‐cache! –  Control at the edge –  Real-‐/me visibility and analy/cs •  Full real-‐/me programma/c interface is a must

133. Thank you! hooman@fastly.com

Extending your applications to the edge with CDNs

SergeyChernyshev

Extending your applications to the edge with CDNs