Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

H2O - the optimized HTTP server

67,135 views

Published on

Introduction of H2O, an optimized HTTP server / library implementation with support for HTTP/1, HTTP/2, websocket

Published in: Technology
  • Nice to see some relevant discussions here. Yes I have been trying to provide some useful tools for the web users and few months back I could sucessfully launched a really easy to use and free web tool on HTTP2 . HTTP2 test allows you to validate immediately if your server is ready for HTTP2. If the test detects it is not we will show you in our tutorials how to get your site ready for HTTP2. In order to test if your site is ready for HTTP2 just open the link - https://http2.co/ in a browser and enter your website URL. Our test tool will show you which version of the HTTP protocol your server is using.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • How to Get Automated winning picks for NFL, NCAA and MLB? ▲▲▲ http://scamcb.com/zcodesys/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

H2O - the optimized HTTP server

  1. 1. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. H2O the optimized HTTP server DeNA Co., Ltd. Kazuho Oku 1
  2. 2. Who am I? n long experience in network-‐‑‒related / high-‐‑‒ performance programming n works in the field: ⁃ Palmscape / Xiino • world's first web browser for Palm OS, bundled by Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Sony, IBM, NTT DoCoMo ⁃ MySQL extensions: Q4M, mycached, … • MySQL Conference Community Awards (as DeNA) ⁃ JSX • altJS with an optimizing compiler H2O -‐‑‒ the optimized HTTP server2
  3. 3. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Agenda n Introduction of H2O n The motives behind n Writing a fast server n Writing H2O modules n Current status the future n Questions regarding HTTP/2 H2O -‐‑‒ the optimized HTTP server3
  4. 4. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Introducing H2O H2O -‐‑‒ the optimized HTTP server4
  5. 5. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. H2O – the umbrella project n h2o – the standalone HTTP server ⁃ libh2o – can be used as a library as well n picohttpparser – the HTTP/1 parser n picotest – TAP-‐‑‒compatible testing library n qrintf – C preprocessor for optimizing s(n)printf n yoml – DOM-‐‑‒like wrapper for libyaml github.com/h2o H2O -‐‑‒ the optimized HTTP server5
  6. 6. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. h2o n the standalone HTTP server n protocols: ⁃ HTTP/1.x ⁃ HTTP/2 • via Upgrade, NPN, ALPN, direct ⁃ WebSocket (uses wslay) ⁃ with SSL support (uses OpenSSL) n modules: ⁃ file (static files), reverse-‐‑‒proxy, reproxy, deflate n configuration using yaml H2O -‐‑‒ the optimized HTTP server6
  7. 7. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. libh2o n h2o is also available as a library n event loop can be selected ⁃ libuv ⁃ h2o's embedded event loop n configurable via API and/or yaml ⁃ dependency to libyaml is optional H2O -‐‑‒ the optimized HTTP server7
  8. 8. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Modular design n library layer: ⁃ memory, string, socket, timeout, event-‐‑‒loop, http1client, … n protocol layer: ⁃ http1, http2, websocket, loopback n handlers: ⁃ file, reverse-‐‑‒proxy n output filters: ⁃ chunked-‐‑‒encoder, deflate, reproxy n loggers: ⁃ access-‐‑‒log H2O -‐‑‒ the optimized HTTP server8
  9. 9. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Testing n two levels of testing for better quality ⁃ essential for keeping the protocol implementations and module-‐‑‒level API apart n unit-‐‑‒testing ⁃ every module has (can have) it's own unit-‐‑‒test ⁃ tests run using the loopback protocol handler • module-‐‑‒level unit-‐‑‒tests do not depend on the protocol n end-‐‑‒to-‐‑‒end testing ⁃ spawns the server and connect via network ⁃ uses nghttp2 H2O -‐‑‒ the optimized HTTP server9
  10. 10. Internals n uses h2o_buf_t (pair of [char*, size_̲t]) is used to represent data ⁃ common header names are interned into tokens • those defined in HPACK static_̲table + α n mostly zero-‐‑‒copy n incoming data allocated using: malloc, realloc, mmap Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. ⁃ requires 64-‐‑‒bit arch for heavy use n uses writev for sending data H2O -‐‑‒ the optimized HTTP server10
  11. 11. 6 bytes 1,024 bytes 10,240 bytes 6 bytes 1,024 bytes 10,240 bytes 6 bytes 1,024 bytes 10,240 bytes 6 bytes 1,024 bytes 10,240 bytes HTTP/1 (local; osx) HTTP/1 (local; linux) HTTP/1 (remote; linux) HTTPS/1 (remote; linux) Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Fast 120,000 100,000 80,000 60,000 40,000 20,000 0 Requests / second.core nginx-­‐1.7.7 h2o Note: used MacBook Pro Early 2014 (Core i7@2.4GHz), Amazon EC2 cc2.8xlarge, no logging H2O -‐‑‒ the optimized HTTP server11
  12. 12. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Why is it fast? Why should it be fast? H2O -‐‑‒ the optimized HTTP server12
  13. 13. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. It all started with PSGI/Plack n PSGI/Plack is the WSGI/Rack for Perl n on Sep 7th 2010: ⁃ first commit to github.com/plack/Plack ⁃ I asked: why ever use FastCGI? • at the time, HTTP was believed to be slow, and FastCGI is necessary ⁃ the other choice was to use Apache+mod_̲perl ⁃ I proposed: • write a fast HTTP parser in C, and use it from Perl • get rid of specialized protocols / tightly-‐‑‒coupled legacy servers ⁃ for ease of dev., H2O -‐‑‒ the optimized HTTP server deploy., admin.13
  14. 14. So I wrote HTTP::Parser::XS and picohttpparser. H2O -‐‑‒ the optimized HTTP server14 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
  15. 15. How fast is picohttpparser? n 10x faster than http-‐‑‒parser according to 3p bench. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. ⁃ github.com/fukamachi/fast-‐‑‒http HTTP Parser Performance Comparison 329,033 3,162,745 3,500,000 3,000,000 2,500,000 2,000,000 1,500,000 1,000,000 500,000 0 hYp-­‐parser@5fd51fd picohYpparser@56975cd requests / second H2O -‐‑‒ the optimized HTTP server15
  16. 16. HTTP::Parser::XS n the de-‐‑‒facto HTTP parser used by PSGI/Plack Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. ⁃ PSGI/Plack is the WSGI/Rack for Perl n modern Perl-‐‑‒based services rarely use FastCGI or mod_̲perl n the application servers used (Starlet, Starman, etc.) speak HTTP using HTTP::Parser::XS ⁃ application servers can be and in fact are written in Perl, since the slow part is handled by HTTP::Parser::XS n picohttpparser is the C-‐‑‒based backend of HTTP::Parser::XS H2O -‐‑‒ the optimized HTTP server16
  17. 17. The lessons learned n using one protocol (HTTP) everywhere reduces the TCO ⁃ easier to develop, debug, test, monitor, administer ⁃ popular protocols tend to be better designed implemented thanks to the competition Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. n similar transition happens everywhere ⁃ WAP has been driven out by HTTP HTML ⁃ we rarely use FTP these days H2O -‐‑‒ the optimized HTTP server17
  18. 18. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. but HTTP is not yet used everywhere n web browser ⁃ HTTP/1 is used now, transiting to HTTP/2 n SOA / microservices ⁃ HTTP/1 is used now • harder to transit to HTTP/2 since many proglangs use blocking I/O ⁃ other protocols coexist: RDBMS, memcached, … • are they the next target of HTTP (like FastCGI?) n IoT • MQTT is emerging H2O -‐‑‒ the optimized HTTP server18
  19. 19. So I decided to write H2O n in July 2014 n life of the developers becomes easier if all the services use HTTP n but for the purpose, it seems like we need to raise the bar (of performance) ⁃ or other protocols may emerge / continue to be used n now (at the time of transition to HTTP/2) might be a good moment to start a performance race between HTTP implementers H2O -‐‑‒ the optimized HTTP server19 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
  20. 20. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Writing a fast server H2O -‐‑‒ the optimized HTTP server20
  21. 21. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Two things to be aware of n characteristics of a fast program 1. executes less instructions • speed is a result of simplicity, not complexity 2. causes less pipeline hazards • minimum number of conditional branches / indirect calls • use branch-‐‑‒predictor-‐‑‒friendly logic ⁃ e.g. conditional branch exists, but it is taken 95% H2O -‐‑‒ the optimized HTTP server21
  22. 22. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. H2O -‐‑‒ design principles n do it right ⁃ local bottlenecks can be fixed afterwards ⁃ large-‐‑‒scale design issues are hard to notice / fix n do it simple ⁃ as explained ⁃ provide / use hooks only at high-‐‑‒level • hooks exist for: protocol, generator, filter, logger H2O -‐‑‒ the optimized HTTP server22
  23. 23. The performance pitfalls n many server implementations spend CPU cycles in the following areas: Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. ⁃ memory allocation ⁃ parsing input ⁃ stringifying output and logs ⁃ timeout handling H2O -‐‑‒ the optimized HTTP server23
  24. 24. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Memory allocation H2O -‐‑‒ the optimized HTTP server24
  25. 25. Memory allocation in H2O n uses region-‐‑‒based memory management Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. ⁃ memory pool of Apache n strategy: ⁃ memory block is assigned to the Request object ⁃ small allocations returns portions of the block ⁃ memory is never returned to the block ⁃ The entire memory block gets freed when the Request object is destroyed H2O -‐‑‒ the optimized HTTP server25
  26. 26. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Memory allocation in H2O (cont'd) n malloc (of small chunks) void *h2o_mempool_alloc(h2o_mempool_t *pool, size_t sz)! {! (snip)! void *ret = pool-chunks-bytes + pool-chunks-offset;! pool-chunks-offset += sz;! return ret;! } ! n free ⁃ no code (as explained) H2O -‐‑‒ the optimized HTTP server26
  27. 27. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Parsing input H2O -‐‑‒ the optimized HTTP server27
  28. 28. Parsing input n HTTP/1 request parser may or may not be a bottleneck, depending on its performance ⁃ if the parser is capable of handling 1M reqs/sec, then it will spend 10% of time if the server handles 100K reqs/sec. 3,500,000 3,000,000 2,500,000 2,000,000 1,500,000 1,000,000 500,000 H2O -‐‑‒ the optimized HTTP server28 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. 329,033 3,162,745 0 hYp-­‐parser@5fd51fd picohYpparser@56975cd requests / second HTTP/1 Parser Performance Comparison
  29. 29. Parsing input (cont'd) n it's good to know the logical upper-‐‑‒bound ⁃ or we might try to optimize something that can no more be faster Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. n Q. How fast could a text parser be? H2O -‐‑‒ the optimized HTTP server29
  30. 30. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Q. How fast could a text server be? Answer: around 1GB/sec. is a good target ⁃ since any parser needs to read every byte and execute a conditional branch depending on the value • # of instructions: 1 load + 1 inc + 1 test + 1 conditional branch • would likely take several CPU cycles (even if superscalar) • unless we use SIMD instructions H2O -‐‑‒ the optimized HTTP server30
  31. 31. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Parsing input n What's wrong with this parser? for (; s != end; ++s) {! int ch = *s;! switch (ctx.state) {! case AAA:! if (ch == ' ')! ctx.state = BBB;! break;! case BBB:! ...! }! H2O -‐‑‒ the optimized HTTP server31
  32. 32. Parsing input (cont'd) n never write a character-‐‑‒level state machine if performance matters Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. for (; s != end; ++s) {! int ch = *s;! switch (ctx.state) { // ß executed for every char! case AAA:! if (ch == ' ')! ctx.state = BBB;! break;! case BBB:! ...! }! H2O -‐‑‒ the optimized HTTP server32
  33. 33. Parsing input fast n each state should consume a sequence of bytes Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. while (s != end) {! switch (ctx.state) {! case AAA:! do {! if (*s++ == ' ') {! ctx.state = BBB;! break;! }! } while (s != end);! break;! case BBB:! ... H2O -‐‑‒ the optimized HTTP server33
  34. 34. Stateless parsing n stateless in the sense that no state value exists ⁃ stateless parsers are generally faster than stateful parsers, since it does not have state -‐‑‒ a variable used for a conditional branch n HTTP/1 parsing can be stateless since the request-‐‑‒ line and the headers arrive in a single packet (in most cases) ⁃ and even if they did not, it is easy to check if the end-‐‑‒of-‐‑‒headers has arrived (by looking for CR-‐‑‒ LF-‐‑‒CR-‐‑‒LF) and then parse the input • this countermeasure is essential to handle the Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Slowloris attack H2O -‐‑‒ the optimized HTTP server34
  35. 35. picohttpparser is stateless n states are the execution contexts (instead of being a variable) const char* parse_request(const char* buf, const char* buf_end, …)! {! /* parse request line */! ADVANCE_TOKEN(*method, *method_len);! ++buf;! ADVANCE_TOKEN(*path, *path_len);! ++buf;! if ((buf = parse_http_version(buf, buf_end, minor_version, ret)) == NULL)! return NULL;! EXPECT_CHAR('015');! EXPECT_CHAR('012');! return parse_headers(buf, buf_end, headers, num_headers, max_headers, …);! }! H2O -‐‑‒ the optimized HTTP server35 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
  36. 36. loop exists within a function (≒state) n the code looks for the end of the header value #define IS_PRINTABLE(c) ((unsigned char)(c) - 040u 0137u)! ! static const char* get_token_to_eol(const char* buf, const char* buf_end, …! {! Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. while (likely(buf_end - buf = 8)) {! #define DOIT() if (unlikely(! IS_PRINTABLE(*buf))) goto NonPrintable; ++buf! DOIT(); DOIT(); DOIT(); DOIT();! DOIT(); DOIT(); DOIT(); DOIT();! #undef DOIT! continue;! NonPrintable:! if ((likely((uchar)*buf '040') likely(*buf != '011'))! || unlikely(*buf == '177'))! goto FOUND_CTL;! } H2O -‐‑‒ the optimized HTTP server36
  37. 37. The hottest loop of picohttpparser (cont'd) n after compilation, uses 4 instructions per char movzbl (%r9), %r11d! movl %r11d, %eax! addl $-32, %eax! cmpl $94, %eax! ja LBB5_5! movzbl 1(%r9), %r11d // load char! leal -32(%r11), %eax // subtract! cmpl $94, %eax // and check if is printable! ja LBB5_4 // if not, break! movzbl 2(%r9), %r11d // load next char! leal -32(%r11), %eax // subtract! cmpl $94, %eax // and check if is printable! ja LBB5_15 // if not, break! movzbl 3(%r9), %r11d // load next char! … H2O -‐‑‒ the optimized HTTP server37 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
  38. 38. strlen vs. picoh?pparser strlen (simple) picohYpparser@56975cd Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. strlen vs. picohttparser n not as fast as strlen, but close size_t strlen(const char *s) {! const char *p = s;! for (; *p != '0'; ++p)! ;! return p - s;! }! n ! not much room ! left for further optimization (wo. using SIMD insns.)! ! 0.90 0.80 0.70 0.60 0.50 0.40 0.30 0.20 0.10 0.00 bytes / clock H2O -‐‑‒ the optimized HTTP server38
  39. 39. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. picohttpparser is small and simple $ wc picohttpparser.?! 376 1376 10900 picohttpparser.c! 62 333 2225 picohttpparser.h! 438 1709 13125 total! $ ! ! n good example of do-‐‑‒it-‐‑‒simple-‐‑‒for-‐‑‒speed approach ⁃ H2O (incl. the HTTP/2 parser) is designed using the approach H2O -‐‑‒ the optimized HTTP server39
  40. 40. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Stringification H2O -‐‑‒ the optimized HTTP server40
  41. 41. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Stringification n HTTP/1 responses are in strings sprintf(buf, HTTP/1.%d %d %srn, …)! n s(n)printf is known to be slow ⁃ but the interface is great ⁃ it's tiresome to write like: p = strappend_s(p, HTTP/1.);! p = strappend_n(p, minor_version);! *p++ = ' ';! P = strappend_n(p, status);! *p++ = ' ';! p = strappend_s(p, reason);! p = strappend_s(p, rn); H2O -‐‑‒ the optimized HTTP server41
  42. 42. Stringification (cont'd) n stringification is important for HTTP/2 servers too ⁃ many elements still need to be stringified • headers (status, date, last-‐‑‒modified, etag, …) • access log (IP address, date, # of bytes, …) H2O -‐‑‒ the optimized HTTP server42 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
  43. 43. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Why is s(n)printf slow? n it's a state machine ⁃ interprets the format string (e.g. hello: %s) at runtime n it uses the locale ⁃ not for all types of variables, but… n it uses varargs n it's complicated ⁃ sprintf may parse a number when used for stringifying a number sprintf(buf, %11d, status)! H2O -‐‑‒ the optimized HTTP server43
  44. 44. How should we optimize s(n)printf? n by compiling the format string at compile-‐‑‒time ⁃ instead of interpreting it at runtime ⁃ possible since the supplied format string is almost always a string literal Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. n and that's qrintf H2O -‐‑‒ the optimized HTTP server44
  45. 45. qrintf n qrintf is a preprocessor that rewrites s(n)printf invocations to set of functions calls specialized to each format string n qrintf-‐‑‒gcc is a wrapper of GCC that Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. ⁃ first applies the GCC preprocessor ⁃ then applies the qrintf preprocessor ⁃ then calls the GCC compiler n similar wrapper could be implemented for Clang ⁃ but it's a bit harder ⁃ help wanted! H2O -‐‑‒ the optimized HTTP server45
  46. 46. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Example // original code (248 nanoseconds)! snprintf(buf, sizeof(buf), %u.%u.%u.%u, ! (addr 24) 0xff, (addr 16) 0xff, (addr 8) 0xff, addr 0xff);! ! // after preprocessed by qrintf (21.5 nanoseconds)! _qrintf_chk_finalize(! _qrintf_chk_u(_qrintf_chk_c(! _qrintf_chk_u(_qrintf_chk_c(! _qrintf_chk_u(_qrintf_chk_c(! _qrintf_chk_u(! _qrintf_chk_init(buf, sizeof(buf)), (addr 24) 0xff),! '.'), (addr 16) 0xff),! '.'), (addr 8) 0xff),! '.'), addr 0xff)); H2O -‐‑‒ the optimized HTTP server46
  47. 47. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Performance impact on H2O n 20% performance gain ⁃ gcc: 82,900 reqs/sec ⁃ qrintf-‐‑‒gcc: 99,200 reqs/sec. n benchmark condition: ⁃ 6-‐‑‒byte file GET over HTTP/1.1 ⁃ access logging to /dev/null H2O -‐‑‒ the optimized HTTP server47
  48. 48. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Timeout handling H2O -‐‑‒ the optimized HTTP server48
  49. 49. Timeout handling by the event loops n most event loops use balanced trees to handle timeouts ⁃ so that timeout events can be triggered fast ⁃ cons. is that it takes time to set the timeouts n in case of HTTP, timeout should be set at least once per request ⁃ otherwise the server cannot close a stale connection H2O -‐‑‒ the optimized HTTP server49 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
  50. 50. Timeout requirements of a HTTP server n much more set than triggered ⁃ is set more than once per request ⁃ most requests succeed before timeout Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. n the timeout values are uniform ⁃ e.g. request timeout for every connection would be the same (or i/o timeout or whatever) n balanced-‐‑‒tree does not seem like a good approach ⁃ any other choice? H2O -‐‑‒ the optimized HTTP server50
  51. 51. Use pre-‐‑‒sorted link-‐‑‒list n H2O maintains a linked-‐‑‒list for each timeout configuration ⁃ request timeout has its own linked-‐‑‒list, i/o timeout has its own, … Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. n how to set the timeout: ⁃ timeout entry is inserted at the end of the linked-‐‑‒ list • thus the list is naturally sorted n how the timeouts get triggered: ⁃ H2O iterates from the start of each linked-‐‑‒list, and triggers those that have timed-‐‑‒out H2O -‐‑‒ the optimized HTTP server51
  52. 52. note: N: number of ]meout entries, M: number of ]meout configura]ons, trigger performance of list of linked-­‐list can Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Comparison Chart OperaAon (frequency in HTTPD)Balanced-­‐treeList of linked-­‐list set (high)O(log N)O(1) clear (high)O(log N)O(1) trigger (low)O(1)O(M) be reduced to O(1) H2O -‐‑‒ the optimized HTTP server52
  53. 53. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Miscellaneous H2O -‐‑‒ the optimized HTTP server53
  54. 54. Miscellaneous n the entire stack of H2O is carefully designed (for simplicity and for performance) ⁃ for example, the built-‐‑‒in event loop of H2O (which is the default for h2o), is faster than libuv 0 10,000 20,000 30,000 40,000 50,000 60,000 70,000 80,000 H2O -‐‑‒ the optimized HTTP server54 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. 6 bytes 4,096 bytes requests / sec.core size of content Benchmark: libuv vs. internal libuv-­‐network-­‐and-­‐file@7876f53 libuv-­‐network-­‐only@da85742 internal (master@a5d1105)
  55. 55. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Writing H2O modules H2O -‐‑‒ the optimized HTTP server55
  56. 56. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Module types of H2O n handler ⁃ generates the contents • e.g. file handler, proxy handler n filter ⁃ modifies the content • e.g. chunked encoder, deflate ⁃ can be chained n logger H2O -‐‑‒ the optimized HTTP server56
  57. 57. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Writing a hello world handler static int on_req(h2o_handler_t *self, h2o_req_t *req) {! static h2o_generator_t generator = {};! static h2o_buf_t body = H2O_STRLIT(hello worldn);! if (! h2o_memis(req-method.base, req-method.len, H2O_STRLIT(GET)))! return -1;! req-res.status = 200;! req-res.reason = OK;! h2o_add_header(req-pool, req-res.headers, H2O_TOKEN_CONTENT_TYPE,! H2O_STRLIT(text/plain));! h2o_start_response(req, generator);! h2o_send(req, body, 1, 1);! return 0;! }! ! h2o_handler_t *handler = h2o_create_handler( host_config, sizeof(*handler));! handler-on_req = on_req; H2O -‐‑‒ the optimized HTTP server57
  58. 58. The handler API /**! * called by handlers to set the generator! * @param req the request! * @param generator the generator! */! void h2o_start_response(h2o_req_t *req, h2o_generator_t *generator);! /**! * called by the generators to send output! * note: generator should close the resources opened by itself after sending the final chunk (i.e. calling the function with is_final set to true)! * @param req the request! * @param bufs an array of buffers! * @param bufcnt length of the buffers array! * @param is_final if the output is final! */! void h2o_send(h2o_req_t *req, h2o_buf_t *bufs, size_t bufcnt, int is_final);! H2O -‐‑‒ the optimized HTTP server58 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
  59. 59. The handler API (cont'd) /**! * an object that generates a response.! * The object is typically constructed by handlers that call h2o_start_response.! */! typedef struct st_h2o_generator_t {! /**! * called by the core to request new data to be pushed via h2o_send! */! void (*proceed)(struct st_h2o_generator_t *self, h2o_req_t *req);! /**! * called by the core when there is a need to terminate the response! */! void (*stop)(struct st_h2o_generator_t *self, h2o_req_t *req);! Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. } h2o_generator_t;! H2O -‐‑‒ the optimized HTTP server59
  60. 60. Module examples n Simple examples exist in the examples/ dir n lib/chunked.c is a good example of the filter API H2O -‐‑‒ the optimized HTTP server60 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
  61. 61. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Current Status the Future H2O -‐‑‒ the optimized HTTP server61
  62. 62. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Development Status n core ⁃ mostly feature complete n protocol ⁃ http/1 – mostly feature complete ⁃ http/2 – interoperable n modules ⁃ file – complete ⁃ proxy – interoperable • name resolution is blocking • does not support keep-‐‑‒alive H2O -‐‑‒ the optimized HTTP server62
  63. 63. HTTP/2 status of H2O n interoperable, but some parts are missing Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. ⁃ HPACK resize ⁃ priority handling n priority handling is essential for HTTP/2 ⁃ without, HTTP/2 is slower than HTTP/1 L n need to tweak performance ⁃ SSL-‐‑‒related code is not yet optimized • first benchmark was taken last Saturday J H2O -‐‑‒ the optimized HTTP server63
  64. 64. HTTP/2 over TLS benchmark n need to fix the dropdown, likely caused by: ⁃ H2O uses writev to gather data into a single socket op., but OpenSSL does not provide scatter-‐‑‒gather I/O 120,000 100,000 80,000 60,000 40,000 20,000 H2O -‐‑‒ the optimized HTTP server64 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. 0 6 bytes 1,024 bytes 10,240 bytes HTTPS/2 (remote; linux) nghYpd h2o ⁃ in H2O, every file handler has its own buffer and pushes content to the protocol layer • nghttpd pulls instead, which is more memory-‐‑‒ efficient / no need
  65. 65. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Goal of the project n to become the best HTTP/2 server ⁃ with excellent performance in serving static files / as a reverse proxy • note: picohttpserver and other libraries are also used in the reverse proxy implementation n to become the favored HTTP server library ⁃ esp. for server products ⁃ to widen the acceptance of HTTP protocol even more H2O -‐‑‒ the optimized HTTP server65
  66. 66. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Help wanted n looking for contributors in all areas ⁃ addition of modules might be the easiest, since it would not interfere with the development of the core / protocol layer ⁃ examples, docs, tests are also welcome n it's easy to start ⁃ since the code-‐‑‒base is young and simple Subsystemwc –l (incl. unit-­‐tests) Core2,334 Library1,856 Socket event loop1,771 HTTP/1 (incl. picohYpparser)886 HTTP/22,507 Modules1,906 Server573 H2O -‐‑‒ the optimized HTTP server66
  67. 67. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Questions regarding HTTP/2 H2O -‐‑‒ the optimized HTTP server67
  68. 68. Sorry, I do not have much to talk n since it is a well-‐‑‒designed protocol n and in terms of performance, apparently binary protocols are easier to implement than a text protocol J ⁃ there's a efficient algorithm for the static Huffman decoder • @tatsuhiro-‐‑‒t implemented it, I copied Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. n OTOH I have some questions re HTTP/2 H2O -‐‑‒ the optimized HTTP server68
  69. 69. Q. would there be a max-‐‑‒open-‐‑‒files issue? n according to the draft, recommended value of MAX_̲CONCURRENT_̲STREAMS is = 100 n if max-‐‑‒connections is 1024, it would mean that the max fd would be above 10k ⁃ on linux, the default (NR_̲OPEN) is 1,048,576 and is adjustable ⁃ but on other OS? n H2O by default limits the number of in-‐‑‒flight requests internally to 16 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. ⁃ the value is configurable H2O -‐‑‒ the optimized HTTP server69
  70. 70. Q. good way to determine the window size? n initial window size (64k) might be too small to saturate the avaiable bandwidth depending on the latency ⁃ but for responsiveness we would not want the value to be too high ⁃ is there any recommendation on how we should tune the variable? H2O -‐‑‒ the optimized HTTP server70 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
  71. 71. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Q. should we continue to use CDN? n HTTP/2 has priority control ⁃ CDN and primary website would use different TCP connection • means that priority control would not work bet. CDN and the primary website n should we better serve all the asset files from the primary website? H2O -‐‑‒ the optimized HTTP server71
  72. 72. Never hide the Server header n name and version info. is essential for interoperability ⁃ many (if not all) webapps use the User-‐‑‒Agent value to evade bugs ⁃ used to be same at the HTTP/1 layer in the early days n there will be interoperability problems bet. HTTP/2 impls. ⁃ the Server header is essential for implementing workarounds n some believe that hiding the header improves security ⁃ we should speak that they are wrong; that security-‐‑‒by-‐‑‒ obscurity does not work on the Net, and hiding the value harms interoperability and the adoption of HTTP/ 2 H2O -‐‑‒ the optimized HTTP server72 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.
  73. 73. Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved. Summary H2O -‐‑‒ the optimized HTTP server73
  74. 74. Summary n H2O is an optimized HTTP server implementation ⁃ with neat design to support both HTTP/1 and HTTP/2 ⁃ is still very young • lots of areas to work on! • incl. improving the HTTP/2 support n help wanted! Let's write the HTTPD of the future! H2O -‐‑‒ the optimized HTTP server74 Copyright (C) 2014 DeNA Co.,Ltd. All Rights Reserved.

×