Reorganizing Website Architecture for HTTP/2 and Beyond

16,181 views

Published on

presentation slides used at Velocity in Amsterdam 2016

Published in: Technology

Reorganizing Website Architecture for HTTP/2 and Beyond

  1. 1. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Reorganizing Website Architecture for HTTP/2 and Beyond DeNA Co., Ltd. Kazuho Oku
  2. 2. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Who am I n  lead developer of H2O HTTP/2 server ⁃  one of the most sophisticated HTTP/2 impl. ⁃  initial public release: 2014/10 (license: MITL) ⁃  used by Fastly, etc. n  author of HTTP performance-related I-Ds ⁃  Cache Digests for HTTP/2 ⁃  103 Early Hints n  lives in Japan, works for DeNA Co., Ltd. Reorganizing Website Architecture for HTTP/2 and Beyond
  3. 3. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Current State of HTTP Reorganizing Website Architecture for HTTP/2 and Beyond
  4. 4. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Why use HTTP/2? n  latency has become the bottleneck of the Web n  HTTP/2 to conceal latency by raising concurrency ⁃  6 concurrent requests in HTTP/1 ⁃  ~100 in HTTP/2 Reorganizing Website Architecture for HTTP/2 and Beyond
  5. 5. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Current state of HTTP n  HTTP/2 (RFC 7540) released on May 2015 1: https://github.com/HTTPWorkshop/workshop2016/blob/master/talks/http2-review-data.pdf Reorganizing Website Architecture for HTTP/2 and Beyond 45 28 37 41 18 31 0% 20% 40% 60% 80% 100% 2015/7 2016/7 # of transac+ons by Firefox1 HTTP HTTPS (H1) HTTPS (H2)
  6. 6. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Achievements n  header compression (HPACK) ⁃  compression ratio: 90% (at median) according to Mozilla n  multiplexing ⁃  the tiled benchmark Reorganizing Website Architecture for HTTP/2 and Beyond
  7. 7. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. So is everything optimal now? Not quite. Reorganizing Website Architecture for HTTP/2 and Beyond
  8. 8. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Issues n  errors in prioritization n  TCP head-of-line blocking n  inefficiency of HTTP/2 push n  compression ratio of small responses Reorganizing Website Architecture for HTTP/2 and Beyond
  9. 9. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Prioritization Reorganizing Website Architecture for HTTP/2 and Beyond
  10. 10. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピュー ターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。 イメージを表示できません。メモリ不足のためにイメージを開 くことができないか、イメージが破損している可能性がありま す。コンピューターを再起動して再度ファイルを開いてくださ い。それでも赤い x が表示される場合は、イメージを削除して 挿入してください。 イメージを表示できません。メモリ不足のためにイメージを 開くことができないか、イメージが破損している可能性があ ります。コンピューターを再起動して再度ファイルを開いて ください。それでも赤い x が表示される場合は、イメージを イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピュー ターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。 イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示さ れる場合は、イメージを削除して挿入してください。 The prioritization tree Reorganizing Website Architecture for HTTP/2 and Beyond Root Leader G Follower G weight: 1 HTML weight: 32 Image weight: 22 Image weight: 22 Image weight: 22 CSS weight: 32 CSS weight: 32 n  hybrid approach using weights and chaining ⁃  servers are expected to obey to the priority specified by the clients n  Firefoxʼs prioritization tree is shown below JS weight: 32 JS weight: 32 イメージを表示できません。メモリ不足のためにイメージを 開くことができないか、イメージが破損している可能性があ ります。コンピューターを再起動して再度ファイルを開いて ください。それでも赤い x が表示される場合は、イメージを 削除して挿入してください。 イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピュータ ーを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。
  11. 11. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. The prioritization tree n  some web browsers fail to specify priority ⁃  Safari, Blink ⁃  older versions of Chrome also had issues ⁃  server-side countermeasures required Reorganizing Website Architecture for HTTP/2 and Beyond イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再 度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。 イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して 再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。 イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イ メージが破損している可能性があります。コンピューターを再起動して再度ファイルを 開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してく ださい。 イメージを表示で きません。メモリ 不足のためにイメ ージを開くことが イメージを表示できませ ん。メモリ不足のために イメージを開くことがで きないか、イメージが破 イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージ が破損している可能性があります。コンピューターを再起動して再度ファイルを開いてくださ い。それでも赤い x が表示される場合は、イメージを削除して挿入してください。 イメージを表示できません。メモリ不足のためにイメージを開くことができないか ァイルを開いてください。それでも赤い x が表示される場合は、イメージを削除し Root HTML weight: 16 CSS weight: 16 JS weight: 16 Image weight: 16 Image weight: 16 Image weight: 16
  12. 12. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. HTTP/2 prioritization: the solution n  bandwidth distribution on server-side: ⁃  use Weighted Fair Queuing (WFQ) or Deficit Round Robin (DRR) ⁃  some servers do it right: •  nghttp2 (and Apache) implements WFQ in O(log N) •  H2O approximates WFQ in O(1) n  detect dumb clients and fallback to server-driven prioritization ⁃  H2O reprioritizes CSS, JS for clients that do not use priority chains Reorganizing Website Architecture for HTTP/2 and Beyond
  13. 13. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. HTTP/2 prioritization: benchmark n  differences between the times spent until first-paint (red bar) Reorganizing Website Architecture for HTTP/2 and Beyond
  14. 14. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. TCP head-of-line blocking Reorganizing Website Architecture for HTTP/2 and Beyond
  15. 15. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Typical sequence of HTTP/2 Reorganizing Website Architecture for HTTP/2 and Beyond HTTP/2 200 OK <!DOCTYPE HTML> ... <LINK HREF="style.css"... client server GET / GET /style.css need to switch from sending HTML to sending CSS at this very moment, or from sending images to sending CSS in case the CSS in discovered late (i.e. hidden assets) 1 RTT *
  16. 16. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. TCP head-of-line blocking n  head-of-line blocking: ⁃  high-priority data blocked by preceding data in flight n  TCP head-of-line blocking: ⁃  data in TCP send buffer blocks following data of higher priority Reorganizing Website Architecture for HTTP/2 and Beyond
  17. 17. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. TCP head-of-line blocking Reorganizing Website Architecture for HTTP/2 and Beyond n  typical H2 server writes much more than that can be sent immediately ⁃  unsent data in TCP send buffer (and TLS buffer) head-of-line blocks following data TCP send buffer CWND unacked poll threshold TLS buf. TLS Records sent immediately not immediately sent HTTP/2 frames
  18. 18. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. TCP head-of-line blocking: the solution n  write only what can be sent immediately ⁃  obtain CWND and unacked size using TCP_INFO n  adjust poll threshold to delay write notification until TCP becomes ready to send some data immediately Reorganizing Website Architecture for HTTP/2 and Beyond CWND unacked poll threshold TLS Records sent immediately not immediately sent HTTP/2 frames TCP send buffer
  19. 19. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. TCP head-of-line blocking: benchmark 1 Reorganizing Website Architecture for HTTP/2 and Beyond n  conditions: ⁃  server in Ireland, client in Tokyo (RTT 250ms) ⁃  load tiny js at the top of a large HTML n  result: delay decreased from 511ms to 250ms ⁃  i.e. JS fetch latency was 2 RTT, became 1 RTT •  similar results in other environments
  20. 20. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. TCP head-of-line blocking: benchmark 2 n  using same data as previous n  server and client both in Japan (25ms RTT) Reorganizing Website Architecture for HTTP/2 and Beyond 0 50 100 150 200 250 300 HTML JS milliseconds downloading HTML (and JS within) RTT ~25ms master latopt
  21. 21. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. TCP buffers exist everywhere n  HTTP/2 serverʼs TCP send buffer ⁃  the negative impact can be avoided by a clever HTTP/2 server (as shown by H2O) n  TLS terminator ⁃  better to avoid using, if performance matters n  mobile networks ⁃  cannot be controlled by a web-site admin ⁃  QUIC would be the final solution Reorganizing Website Architecture for HTTP/2 and Beyond
  22. 22. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Push Reorganizing Website Architecture for HTTP/2 and Beyond
  23. 23. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Push n  positive reports: ⁃  “20-30% speedup on page render time”2 n  negative comments: ⁃  many unnecessary pushes (47% are reset2) ⁃  increased render time in anti-patterns3 ⁃  “consider preload instead of push”3 n  what is the cause of these negative views? 2: https://github.com/HTTPWorkshop/workshop2016/blob/master/talks/server-push.pdf 3: https://docs.google.com/document/d/1K0NykTXBbbbTlv60t5MyJvXjqKGsCVNYHyLEXIxYMv0/edit Reorganizing Website Architecture for HTTP/2 and Beyond
  24. 24. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Itʼs the cached resources n  server shouldnʼt push resources already cached ⁃  but HTTP/2 doesnʼt provide a way to determine ⁃  pushing same resources multiple times is waste of bandwidth n  solution: cache-digests ⁃  now an IETF HTTP WG draft ⁃  https://tools.ietf.org/html/draft-ietf-httpbis- cache-digest-01 Reorganizing Website Architecture for HTTP/2 and Beyond
  25. 25. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Cache-digests n  client sends a digest of cached responses along with an HTTP request n  server examines the digest, pushes only the resources that arenʼt cached n  digest: list of SHA(URL)s stored in Golomb-coded sets (a space- efficient variant of bloom filter) n  size of digest: ~ N*log2(1/P) bits ⁃  N: # of resources ⁃  P: false-positive-rate Reorganizing Website Architecture for HTTP/2 and Beyond
  26. 26. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Example n  set false positive rate to: P=1/27 n  calculate SHA256 of two URLs (N=2) ⁃  https://example.com/style.css (SHA256: baf9…) ⁃  https://example.com/script.js (SHA256: 1174…) n  truncate to log2(N*1/P) bits and sort ⁃  0x11, 0xba n  emit log2(N), log2(1/P) in 5 bits, then the deltas of sorted list using Golomb coding (subtracted by 1) ⁃  00001 00111 1 0010001 01 010 1000 N 1/P 0x11 0xa8 (0xba-0x11-1) Reorganizing Website Architecture for HTTP/2 and Beyond
  27. 27. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Overhead of sending a Cache Digest n  # of fresh responses in browser cache per domain is expected to be <~1K ⁃  looking at about:cache •  *.facebook.com: 790 •  *.google.com: 373 n  HTTP request + cache-digest can be packed into a single TCP packet ⁃  when setting P to 1/128 or 1/256 •  or, P can be set to a bigger value to conserve space, or cache-digest could be split into multiple packets following the HTTP request Reorganizing Website Architecture for HTTP/2 and Beyond
  28. 28. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Implementation Status n  three ways to send a cache-digest ⁃  I-D proposes the use of HTTP/2 frame ⁃  header-based scheme is TBD •  so that websites can implement cache-digest using ServiceWorker (e.g. cache-digest.js) 1: the listed softwares may have interoperability issues due to implementing different versions of the draft 2: approximating the cache state using cookies has accuracy issues Reorganizing Website Architecture for HTTP/2 and Beyond Conveyer Server Implementa+on Client Implementa+on HTTP/2 frame nghYp2 N/A HTTP header1 Apache, H2O cache-digest.js HTTP cookie2 H2O not needed
  29. 29. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. but even after solving the never-push-cached- resources issue, push doesnʼt necessarily improve performance Reorganizing Website Architecture for HTTP/2 and Beyond
  30. 30. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Benchmark scores n  first-paint does not change ⁃  since the browser needs both HTML and CSS ⁃  push only changes the order between the two n  less time spent showing a blank page ⁃  since unload happens when HTML arrives Reorganizing Website Architecture for HTTP/2 and Beyond
  31. 31. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. so is push an useless technology? no Reorganizing Website Architecture for HTTP/2 and Beyond
  32. 32. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Three use-cases of H2 push n  prioritization n  push while processing request n  push from edge Reorganizing Website Architecture for HTTP/2 and Beyond
  33. 33. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Pushing for prioritization Reorganizing Website Architecture for HTTP/2 and Beyond client server GET / GET /style.css HTTP/2 200 OK <html> <link rel=style.css ... HTTP/2 200 OK body: ... #title: ... 1.  send CSS, JS first 2.  then send HTML (can be rendered progressively) without push client server GET / GET /style.css HTTP/2 HTTP/2 200 OK body: ... #title: ... with push HTTP/2 200 OK <html>
 <link rel=style.css ...
  34. 34. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Push while processing request n  web applications involving DB access, etc. Reorganizing Website Architecture for HTTP/2 and Beyond req. process request push-asset HTML push-asset push-asset push-asset req. process request asset HTML asset asset asset req. 450ms (5 RTT + processing =me) 250ms (1 RTT + processing =me) without push with push
  35. 35. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Push from edge n  CDNsʼ use-case ⁃  utilize the conn. while waiting for app. response Reorganizing Website Architecture for HTTP/2 and Beyond req. push-asset HTML push-asset push-asset push-asset client edge server (CDN) app. server req. HTML
  36. 36. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Impact of push n  prioritization doesnʼt have much benefit ⁃  as discussed n  push-while-processing, and push-from-edge are the real use-cases ⁃  but how to use? Reorganizing Website Architecture for HTTP/2 and Beyond
  37. 37. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Push while processing Reorganizing Website Architecture for HTTP/2 and Beyond req. process request push-asset HTML push-asset push-asset push-asset req. process request asset HTML asset asset asset req. 450ms (5 RTT + processing =me) 250ms (1 RTT + processing =me) without push with push n  One way is to configure the H2 server mruby.handler: | Proc.new do |env| push_paths = [] if /(/|.html)$/.match(env["PATH_INFO"]) push_paths << "/style.css” ... end [399, push_paths.empty? ? {} : { "link" => push_paths.map{|p| "<#{p}>; rel=preload” }.join("n") }, []] end fastcgi.connect: ...
  38. 38. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Push while processing Reorganizing Website Architecture for HTTP/2 and Beyond n  or use 103 Early Hints ⁃  application sends 103 then processes the request GET / HTTP/1.1 Host: example.com HTTP/1.1 103 Early Hints Link: </style.css>; rel=preload HTTP/1.1 200 OK Content-Type: text/html Link: </style.css>; rel=preload <!DOCTYPE HTML> ... HTTP/2 server app. server GET / 103 Early Hints Link: … 200 OK process request Web Browser GET / 200 OK push
  39. 39. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. 103 Early Hints n  a new I-D (published last week) ⁃  https://tools.ietf.org/html/draft-kazuho-early- hints-status-code-00 n  the status code indicates that the server is likely to send a final request with the headers included in the informational response ⁃  can be used to hint preloading 3rd party assets (e.g. ads) as well n  already supported by: nghttp2, H2O ⁃  other server / web-application framework developers have shown interest Reorganizing Website Architecture for HTTP/2 and Beyond
  40. 40. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Push from edge n  at least some CDNs are likely to provide configuration directives at the edge-server level Reorganizing Website Architecture for HTTP/2 and Beyond req. push-asset HTML push-asset push-asset push-asset client edge server (CDN) app. server req. HTML
  41. 41. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. compression ratio of small responses Reorganizing Website Architecture for HTTP/2 and Beyond
  42. 42. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. File size vs. compression ratio n  claim: no need to concatenate, since H2 improves request concurrency n  reality: small files suffer from low compression ratio ⁃  problem for API as well note: first N bytes of jquery-3.1.1.js compressed with best quality settings Reorganizing Website Architecture for HTTP/2 and Beyond 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 256 1,024 4,096 16,384 65,536 86,709 file size Compression ra/o of a JavaScript file gzip brotli
  43. 43. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Two proposals n  Shared Dictionary Compression over HTTP (SDCH) ⁃  uses a pre-shared compression dictionary ⁃  https://tools.ietf.org/html/draft-lee-sdch-spec-00 n  Compression Dictionaries for HTTP/2 ⁃  re-uses the compression context of a previously transmitted response ⁃  https://tools.ietf.org/html/draft-vkrasnov-h2- compression-dictionaries-01 Reorganizing Website Architecture for HTTP/2 and Beyond
  44. 44. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Differences between the approaches n  pre-shared dictionary: ⁃  achieves high compression ratio when the dictionary is cached by the client (i.e. when revisiting) ⁃  cannot be used at first visit n  re-using compression context of another response: ⁃  works effectively at first visit ⁃  easier to use, since no configuration needed ⁃  may have security issues •  i.e. CRIME attack Reorganizing Website Architecture for HTTP/2 and Beyond
  45. 45. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Compression ratio of small responses n  beneficial to concatenate small files (to ~32KB or larger) for the time being Reorganizing Website Architecture for HTTP/2 and Beyond
  46. 46. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. Summary Reorganizing Website Architecture for HTTP/2 and Beyond
  47. 47. Copyright (C) 2016 DeNA Co.,Ltd. All Rights Reserved. HTTP/2 is not perfect (yet) n  choice of servers / CDNs causes difference ⁃  be careful when looking at benchmarks n  avoid using: ⁃  TLS terminators ⁃  hidden assets (or un-hide them w. preload links) n  use push for either of: ⁃  push-while-processing (103 Early Hints) ⁃  push-from-edge (configure your CDN) n  continue to concatenate files, at least the tiny ones Reorganizing Website Architecture for HTTP/2 and Beyond

×