How to build a High Performance PSGI/Plack Server

  • 12,792 views
Uploaded on

How to build a High Performance PSGI/Plack Server …

How to build a High Performance PSGI/Plack Server
PSGI/Plack・Monocerosで学ぶ ハイパフォーマンス Webアプリケーションサーバの作り方

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
12,792
On Slideshare
0
From Embeds
0
Number of Embeds
25

Actions

Shares
Downloads
38
Comments
0
Likes
20

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. How to build a High Performance PSGI/Plack Server PSGI/Plack・Monocerosで学ぶ ハイパフォーマンス Webアプリケーションサーバの作り方 YAPC::Asia 2013 Tokyo Masahiro Nagano / @kazeburo
  • 2. Me • 長野雅広 Masahiro Nagano • @kazeburo • PAUSE:KAZEBURO • Operations Engineer, Site Reliability • LINE Corp. Development support LINE Family, livedoor
  • 3. livedoorBlog One of the largest Blog Hosting Service in Japan
  • 4. livedoorBlog uses
  • 5. Perl (5.16and 5.8)
  • 6. Carton
  • 7. Plack/PSGI and mod_perl
  • 8. $ curl -I http://blog.livedoor.jp/staff/| grep Server Server: Plack::Handler::Starlet
  • 9. Starlet handles 1 Billion(10億) reqs/day
  • 10. To get over this burst traffic, We need to improve Performance across all layers この負荷を乗り切るために様々なレイヤーで最適化をしています
  • 11. Layers Hardware / Network OS App Server Routing Cache Logic SQL Template Engine RDBMS Cached Web Server
  • 12. Hardware / Network OS Routing Cache Logic SQL Template Engine RDBMS Cached Web Server Today’s Topic
  • 13. Hardware / Network OS Routing Cache Logic SQL Template Engine RDBMS Cached Web Server Today’s Topic App Server
  • 14. By the way..
  • 15. “Open & Share” is our driver for excellence. And LOVE CPAN/OSS Open & Share は私たちの目指すところです。 CPAN/OSSを多く使い、また貢献もしています
  • 16. Improving Performance of livedoorBlog directly linked Performance of Plack/Starlet on CPAN livedoorBlogのパフォーマンス改善で行った事は CPAN上のPlack/Starletにも当然影響してきます
  • 17. 0 3500 7000 10500 14000 13083 6241 “Hello World”Reqs/Sec Plack 1.0016 1.0029 Starlet 0.16 0.20 2013/02 2013/09 mod_perl era plack era
  • 18. Monoceros
  • 19. Monoceros is a yet another Plack/PSGI Server for Performance
  • 20. the Goal
  • 21. Reduce TCP 3way hand shake between Proxy and PSGI Server
  • 22. Reverse Proxy App Server GET / HTTP/1.1 Host: example.com SYN ACK SYN+ACK HTTP/1.1 200 OK Content-Type: text/html FIN ACK GET /favicon.ico HTTP/1.1 Host: example.com SYN ACK SYN+ACK HTTP/1.1 404 NOT FOUND Content-Type: text/html FIN ACK
  • 23. HTTP/1.0-1.1 have KeepAlive
  • 24. Reverse Proxy App Server GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK HTTP/1.0 200 OK Content-Type: text/html Connection: keep-alive Content-Length: 941 GET /favicon.ico HTTP/1.1 Host: example.com HTTP/1.1 200 OK Content-Type: image/vnd.microsoft.icon Transfer-Encoding: chunked GET /site.css HTTP/1.1 Host: example.com HTTP/1.1 200 OK Content-Type: text/css Content-Length: 1013
  • 25. C10K problem
  • 26. nginx C10K Ready Reverse Proxy nginx nginx Starlet Starman App Server KeepAlive Req KeepAlive Req KeepAlive Req
  • 27. Starman, Starlet’s Preforking model requires 1 connection per 1 process By default Starman: 5 procs Starlet: 10 procs
  • 28. Monoceros adopts Preforking model, But C10K ready
  • 29. Worker Process Worker Process Worker Process Worker Process Manager Process SOCK Client GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html
  • 30. Worker Process Worker Process Worker Process Worker Process Manager Process SOCK Client GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html
  • 31. Worker Process Worker Process Worker Process Worker Process Manager Process SOCK Client GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html GET / HTTP/1.1 Host: example.com Event Driven
  • 32. Worker Process Worker Process Worker Process Worker Process Manager Process SOCK Client GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html Event Driven
  • 33. Monoceros Workers that inherits“Starlet” not C10K ready
  • 34. Event Driven Manager Process C10K ready built with AnyEvent
  • 35. KeepAlive Benchmark like a Browser 1) connect 2) do requests certain number 3) leave alone a socket 4) timeout and close
  • 36. 250 conn / 200 reqs Starlet Monoceros Total time (sec) 54.51 8.74 Failed reqs 971 0
  • 37. Plack/PSGI Basics
  • 38. PSGI = specification Plack = implementation
  • 39. PSGI Interface my $app = sub { my $env = shift; ... return [200, [‘Content-Type’ => ‘text/html’], [‘Hello World’] ]; };
  • 40. PSGI environment hash * CGI keys REQUEST_METHOD,SCRIPT_NAME, PATH_INFO,REQUEST_URI, QUERY_STRING,SERVER_PROTOCOL,HTTP_* * PSGI-specific keys psgi.version, psgi.url_scheme, psgi.input, psgi.errors, psgi.multiprocess, psgi.streaming psgi.nonblocking $env->{...}
  • 41. PSGI Response (1) ArrayRef [200, #status code [ ‘Content-Type’ => ‘text/html’, ‘Content-Length => 10 ], [ ‘Hello’, ‘World’ ] ];
  • 42. PSGI Response (1’) arrayref+IO::Handle open my $fh, ‘<’, ‘/path/icon.jpg’; [200, [ ‘Content-Type’ => ‘image/jpeg’, ‘Content-Length => 123456789 ], $fh ];
  • 43. PSGI Response (2) Delayed and Streaming sub { my $env = shift; return sub { my $responder = shift; ... $responder->([ 200, $headers, [$body] ]); } };
  • 44. PSGI Response (2’) Delayed and Streaming return sub { my $responder = shift; my $writer = $responder->([200, $headers]); wait_for_events(sub { my $new_event = shift; if ($new_event) { $writer->write($new_event->as_json . "n"); } else { $writer->close; } }); };
  • 45. Role of“PSGI Server”
  • 46. PSGI Server “A PSGI Server is a Perl program providing an environment for a PSGI application to run in”
  • 47. PSGI Server App $env $res Apache Nginx Apache Proxy Browser CGI mod_perl FCGI HTTP Perl direct
  • 48. PSGI Server is called “Plack Handler”
  • 49. Plack Handler Adaptor interface Plack and PSGI Server. Make PSGI Server to run with“plackup”
  • 50. e.g. Starman Starman::Server = PSGI Server Plack::Handler::Starman = Plack Handler
  • 51. Make PSGI/Plack Server a High Performance
  • 52. Tiny Standalone PSGI Web Server
  • 53. my $null_io = do { open my $io, "<", ""; $io }; my $app = sub { my $env = shift return [200,['Content-Type'=>'text/html'],['Hello','World',"n"]]; }; my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1, ); while ( my $conn = $listen->accept ) { my $env = { SERVER_PORT => '5000', SERVER_NAME => 'localhost', SCRIPT_NAME => '', REMOTE_ADDR => $conn->peerhost, 'psgi.version' => [ 1, 1 ], 'psgi.errors' => *STDERR, 'psgi.url_scheme' => 'http', 'psgi.run_once' => Plack::Util::FALSE, 'psgi.multithread' => Plack::Util::FALSE, 'psgi.multiprocess' => Plack::Util::FALSE, 'psgi.streaming' => Plack::Util::FALSE, 'psgi.nonblocking' => Plack::Util::FALSE, 'psgi.input' => $null_io, }; $conn->sysread( my $buf, 4096); my $reqlen = Plack::HTTPParser::parse_http_request($buf, $env); my $res = Plack::Util::run_app $app, $env; my @lines = ("HTTP/1.1 $res->[0] @{[ status_message($res->[0]) ]}015012"); for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]015012"; } push @lines, "Connection: close0151201512"; $conn->syswrite(join "",@lines); Plack::Util::foreach($res->[2], sub { $conn->syswrite(shift); }); $conn->close; } 60 lines
  • 54. Listen and Accept
  • 55. my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1, ); while ( my $conn = $listen->accept ) { ... }
  • 56. Read a request
  • 57. use Plack::HTTPParser qw/parse_http_request/; my $null_io = do { open my $io, "<", ""; $io }; while ( my $conn = $listen->accept ) { my $env = { SERVER_PORT => '5000', SERVER_NAME => 'localhost', SCRIPT_NAME => '', REMOTE_ADDR => $conn->peerhost, 'psgi.version' => [ 1, 1 ], 'psgi.errors' => *STDERR, 'psgi.url_scheme' => 'http', 'psgi.multiprocess' => Plack::Util::FALSE, 'psgi.streaming' => Plack::Util::FALSE, 'psgi.nonblocking' => Plack::Util::FALSE, 'psgi.input' => $null_io, }; $conn->sysread(my $buf, 4096); my $reqlen = parse_http_request($buf, $env);
  • 58. Run App
  • 59. my $res = $app->($env); or my $res = Plack::Util::run_app $app, $env;
  • 60. Write a response
  • 61. use HTTP::Status qw/status_message/; my $res = .. my @lines = ("HTTP/1.1 $res->[0] @{[ status_message($res->[0]) ]}015012"); for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]015012"; } push @lines, "Connection: close0151201512"; $conn->syswrite(join "",@lines); foreach my $buf ( @{$res->[2]} ) { $conn->syswrite($buf); }); $conn->close;
  • 62. This PSGI Server has some problem * handle only one at once * no timeout * may not fast
  • 63. Increase concurrency
  • 64. Multi Process IO Multiplexing or Both
  • 65. Preforking model Simple, Scaling
  • 66. Manager
  • 67. Manager bind listen
  • 68. Worker accept Worker accept Worker accept Worker accept Manager bind listen fork fork fork fork
  • 69. Worker accept Worker accept Worker accept Worker accept Manager bind listen fork fork fork fork Client Client ClientClient
  • 70. use Parallel::Prefork; my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1, ); my $pm = Parallel::Prefork->new({ max_workers => 5, trap_signals => { TERM => 'TERM', HUP => 'TERM', } }); while ( $pm->signal_received ne 'TERM') { $pm->start(sub{ while ( my $conn = $listen->accept ) { my $env = {..}
  • 71. NO Accept Serialization
  • 72. os/kernel Worker accept Worker accept Worker accept Worker accept Manager bind listen Zzz.. Zzz.. Zzz.. Zzz..
  • 73. os/kernel Worker accept Worker accept Worker accept Worker accept Client Manager bind listen Zzz.. Zzz.. Zzz.. Zzz..
  • 74. os/kernel Worker accept Worker accept Worker accept Worker accept Client Manager bind listen Zzz.. Zzz.. Zzz.. Zzz..
  • 75. os/kernel Worker accept Worker accept Worker accept Worker accept Client Manager bind listen Thundering Herd突然の負荷 WakeUp WakeUp WakeUp WakeUP
  • 76. os/kernel Worker accept Worker accept Worker accept Worker accept Client Manager bind listen Thundering Herd突然の負荷 WakeUp WakeUp WakeUp WakeUP
  • 77. Thundering Herd is an old story
  • 78. Worker accept Worker accept Worker Worker Manager bind listen accept accept Client Zzz.. Zzz.. Zzz..Zzz..
  • 79. Worker accept Worker accept Worker Worker Manager bind listen accept accept Client Zzz.. Zzz.. Zzz..Zzz..
  • 80. Worker accept Worker accept Worker Worker Manager bind listen accept accept Client modern os/kernel Zzz.. Zzz.. Zzz..Zzz..
  • 81. Worker accept Worker accept Worker Worker Manager bind listen accept accept Client modern os/kernel Zzz.. Zzz.. Zzz..Zzz..
  • 82. Worker accept Worker accept Worker Worker Manager bind listen accept accept Client modern os/kernel Zzz.. Zzz..Zzz.. WakeUp
  • 83. NO Accept Serialization (except for multiple interface)
  • 84. TCP_DEFER_ACCEPT
  • 85. Wake up a process when DATA arrived not established コネクションが完了したタイミングではなく、 データが到着した段階でプロセスを起こします
  • 86. client A client B GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK SYN ACK SYN+ACK GET / HTTP/1.0 Host: example.com Connection: keep-alive default defer_accept Accept RunApp block to read
  • 87. RunApp client A client B GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK SYN ACK SYN+ACK GET / HTTP/1.0 Host: example.com Connection: keep-alive default defer_accept Accept RunApp Accept block to read
  • 88. RunApp client A client B GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK SYN ACK SYN+ACK GET / HTTP/1.0 Host: example.com Connection: keep-alive default defer_accept Accept RunApp Accept block to read idle
  • 89. RunApp client A client B GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK SYN ACK SYN+ACK GET / HTTP/1.0 Host: example.com Connection: keep-alive default defer_accept Accept RunApp Accept RunApp Accept RunApp Accept block to read idle
  • 90. use Socket qw(IPPROTO_TCP); my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1, ); if ($^O eq 'linux') { setsockopt($listen, IPPROTO_TCP, 9, 1); }
  • 91. timeout to read header
  • 92. alarm
  • 93. my $READ_TIMEOUT = 5; eval { local $SIG{ALRM} = sub { die "Timed outn"; }; alarm( $READ_TIMEOUT ); $conn->sysread(my $buf, 4096); }; alarm(0); next if ( $@ && $@ =~ /Timed out/ ); my $reqlen = parse_http_request($buf, $env);
  • 94. nonblocking + select
  • 95. use IO::Select; my $READ_TIMEOUT = 5; while( my $conn = $listen->accept ) { $conn->blocking(0); my $select = IO::Select->new($conn); my @ready = $select->can_read($READ_TIMEOUT); next unless @ready; $conn->sysread($buf, 4096); my $reqlen = parse_http_request($buf, $env);
  • 96. alarm vs. nonblocking + select
  • 97. Fewer syscalls is good
  • 98. Hardwares User Application OS/Kernel system calls listen,fork, accept, read, write, select, alarm Worker Worker Worker Worker Worker
  • 99. alarm
  • 100. rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 rt_sigaction(SIGALRM, {0x47e5b0, [], SA_RESTORER, 0x7ff7d6e0cba0}, {SIG_DFL, [], SA_RESTORER, 0x7ff7d6e0cba0}, 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 alarm(5) = 0 read(7, "GET / HTTP/1.1rnUser-Agent: curl"..., 65536) = 155 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, 0x7ff7d6e0cba0}, {0x47e5b0, [], SA_RESTORER, 0x7ff7d6e0cba0}, 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 alarm(0) = 5 9 syscalls
  • 101. non-blocking + select
  • 102. fcntl(5, F_GETFL) = 0x2 (flags O_RDWR) fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 select(8, [5], NULL, [5], {300, 0}) = 1 (in [5], left {299, 999897}) read(5, "GET / HTTP/1.1rnUser-Agent: curl"..., 131072) = 155 4 syscalls
  • 103. Parse a request with“C”
  • 104. $ cpanm HTTP::Parser::XS Plack::HTTPParser uses H::P::XS if installed
  • 105. TCP_NODELAY
  • 106. When data was written TCP packets does not immediately send “TCP uses Nagle's algorithm to collect small packets for send all at once by default”
  • 107. write(“foo”) write(“bar”) os/kernel network interface Application buffering “foobar”
  • 108. write(“foo”) write(“bar”) os/kernel network interface Application “foo” TCP_NODELAY “bar”
  • 109. Take care of excessive fragmentation of TCP packets
  • 110. Write in once join content in Server
  • 111. my @lines = ("HTTP/1.1 $res->[0] @{[ status_message($res->[0]) ]}015012"); for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]015012"; } push @lines, "Connection: close0151201512"; $conn->syswrite(join "",@lines, @{$res->[2]});
  • 112. accept4, writev
  • 113. Choose PSGI/Plack Server
  • 114. CPAN has many PSGI Server & Plack::Hanlder:**
  • 115. Standalone (HTTP::Server::PSGI) Default server for plackup Single process Web Server For development
  • 116. Starman Preforking Web Server HTTP/1.1, HTTPS, Multiple interfaces, unix-domain socket, hot deploy using Server::Starter
  • 117. Starlet Preforking Web Server HTTP/1.1(0.20~) hot deploy using Server::Starter Simple and Fast
  • 118. Monoceros C10K Ready Preforking Web Server HTTP/1.1 hot deploy using Server::Starter
  • 119. Twiggy based on AnyEvent nonblocking, streaming Single Process
  • 120. Twiggy::Prefork based on Twiggy and Parallel::Prefork nonblocking, streaming Multi Process hot deploy using Server::Starter
  • 121. Feersum Web server based on EV/libev nonblocking, streaming Single/Multi Process
  • 122. How to choose PSGI Server
  • 123. Single Process Multi Process CPU Intensive - Starlet Starman Monoceros Requires Event Driven Twiggy Feersum Twiggy::Prefork Feersum TypeofWebApplication
  • 124. Finding Bottlenecks of Performance
  • 125. use Devel::NYTProf
  • 126. Flame Graph is awesome Profile nytprof.out.{PID} for preforking server $ nytprofhtml -f nytprof.out.1210 $ open nytprof/index.html
  • 127. use strace or dtruss trace syscalls
  • 128. $ strace -tt -s 200 -p {pid} 2>&1 | tee /tmp/trace.txt
  • 129. Process 30929 attached - interrupt to quit 16:13:46.826828 accept(4, {sa_family=AF_INET, sin_port=htons(43783), sin_addr=inet_addr("127. 16:13:48.916233 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.916392 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.916493 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.916573 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.916661 fcntl(5, F_SETFD, FD_CLOEXEC) = 0 16:13:48.916873 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR) 16:13:48.916959 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 16:13:48.917095 setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 0 16:13:48.917362 read(5, "GET / HTTP/1.0rnHost: 127.0.0.1:5005rnUser-Agent: ApacheBench/2. 16:13:48.917613 brk(0x1e8e000) = 0x1e8e000 16:13:48.917746 gettimeofday({1379402028, 917802}, NULL) = 0 16:13:48.917953 write(5, "HTTP/1.1 200 OKrnDate: Tue, 17 Sep 2013 07:13:48 GMTrnServer: P 16:13:48.918187 close(5) = 0 16:13:48.918428 accept(4, {sa_family=AF_INET, sin_port=htons(43793), sin_addr=inet_addr("127. 16:13:48.923736 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.923843 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.923924 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.924461 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.924600 fcntl(5, F_SETFD, FD_CLOEXEC) = 0 16:13:48.924762 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR) 16:13:48.924853 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 16:13:48.924939 setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 0 16:13:48.925162 read(5, "GET / HTTP/1.0rnHost: 127.0.0.1:5005rnUser-Agent: ApacheBench/2. 16:13:48.925445 gettimeofday({1379402028, 925494}, NULL) = 0 16:13:48.925629 write(5, "HTTP/1.1 200 OKrnDate: Tue, 17 Sep 2013 07:13:48 GMTrnServer: P 16:13:48.925854 close(5) = 0 16:13:48.926084 accept(4, {sa_family=AF_INET, sin_port=htons(43803), sin_addr=inet_addr("127. 16:13:48.930480 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.930626 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.930744 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.930838 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.930915 fcntl(5, F_SETFD, FD_CLOEXEC) = 0 16:13:48.931070 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)
  • 130. in conclusion
  • 131. PSGI Server get Faster.
  • 132. PSGI/Plack Rocks Stable, Fast Found problems? RT, GitHub Issue, PullReqs IRC #perl @kazeburo
  • 133. #fin. Thank you!