• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
How to build a High Performance PSGI/Plack Server
 

How to build a High Performance PSGI/Plack Server

on

  • 12,691 views

How to build a High Performance PSGI/Plack Server

How to build a High Performance PSGI/Plack Server
PSGI/Plack・Monocerosで学ぶ ハイパフォーマンス Webアプリケーションサーバの作り方

Statistics

Views

Total Views
12,691
Views on SlideShare
5,602
Embed Views
7,089

Actions

Likes
20
Downloads
29
Comments
0

21 Embeds 7,089

http://blog.nomadscafe.jp 4132
http://yapcasia.org 941
http://yosuke-furukawa.hatenablog.com 831
http://takiguchi0817.github.io 622
http://nomadscafe.jp 191
https://twitter.com 120
http://rejasupotaro.github.io 85
http://cloud.feedly.com 83
http://localhost 31
http://newsblur.com 12
http://reader.aol.com 7
http://webcache.googleusercontent.com 6
http://www.newsblur.com 5
http://www.feedspot.com 4
http://blog-new.dev.livedoor.jp 4
http://translate.googleusercontent.com 4
http://feedly.com 3
http://summary 3
http://digg.com 3
http://news.google.com 1
http://131.253.14.98 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    How to build a High Performance PSGI/Plack Server How to build a High Performance PSGI/Plack Server Presentation Transcript

    • How to build a High Performance PSGI/Plack Server PSGI/Plack・Monocerosで学ぶ ハイパフォーマンス Webアプリケーションサーバの作り方 YAPC::Asia 2013 Tokyo Masahiro Nagano / @kazeburo
    • Me • 長野雅広 Masahiro Nagano • @kazeburo • PAUSE:KAZEBURO • Operations Engineer, Site Reliability • LINE Corp. Development support LINE Family, livedoor
    • livedoorBlog One of the largest Blog Hosting Service in Japan
    • livedoorBlog uses
    • Perl (5.16and 5.8)
    • Carton
    • Plack/PSGI and mod_perl
    • $ curl -I http://blog.livedoor.jp/staff/| grep Server Server: Plack::Handler::Starlet
    • Starlet handles 1 Billion(10億) reqs/day
    • To get over this burst traffic, We need to improve Performance across all layers この負荷を乗り切るために様々なレイヤーで最適化をしています
    • Layers Hardware / Network OS App Server Routing Cache Logic SQL Template Engine RDBMS Cached Web Server
    • Hardware / Network OS Routing Cache Logic SQL Template Engine RDBMS Cached Web Server Today’s Topic
    • Hardware / Network OS Routing Cache Logic SQL Template Engine RDBMS Cached Web Server Today’s Topic App Server
    • By the way..
    • “Open & Share” is our driver for excellence. And LOVE CPAN/OSS Open & Share は私たちの目指すところです。 CPAN/OSSを多く使い、また貢献もしています
    • Improving Performance of livedoorBlog directly linked Performance of Plack/Starlet on CPAN livedoorBlogのパフォーマンス改善で行った事は CPAN上のPlack/Starletにも当然影響してきます
    • 0 3500 7000 10500 14000 13083 6241 “Hello World”Reqs/Sec Plack 1.0016 1.0029 Starlet 0.16 0.20 2013/02 2013/09 mod_perl era plack era
    • Monoceros
    • Monoceros is a yet another Plack/PSGI Server for Performance
    • the Goal
    • Reduce TCP 3way hand shake between Proxy and PSGI Server
    • Reverse Proxy App Server GET / HTTP/1.1 Host: example.com SYN ACK SYN+ACK HTTP/1.1 200 OK Content-Type: text/html FIN ACK GET /favicon.ico HTTP/1.1 Host: example.com SYN ACK SYN+ACK HTTP/1.1 404 NOT FOUND Content-Type: text/html FIN ACK
    • HTTP/1.0-1.1 have KeepAlive
    • Reverse Proxy App Server GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK HTTP/1.0 200 OK Content-Type: text/html Connection: keep-alive Content-Length: 941 GET /favicon.ico HTTP/1.1 Host: example.com HTTP/1.1 200 OK Content-Type: image/vnd.microsoft.icon Transfer-Encoding: chunked GET /site.css HTTP/1.1 Host: example.com HTTP/1.1 200 OK Content-Type: text/css Content-Length: 1013
    • C10K problem
    • nginx C10K Ready Reverse Proxy nginx nginx Starlet Starman App Server KeepAlive Req KeepAlive Req KeepAlive Req
    • Starman, Starlet’s Preforking model requires 1 connection per 1 process By default Starman: 5 procs Starlet: 10 procs
    • Monoceros adopts Preforking model, But C10K ready
    • Worker Process Worker Process Worker Process Worker Process Manager Process SOCK Client GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html
    • Worker Process Worker Process Worker Process Worker Process Manager Process SOCK Client GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html
    • Worker Process Worker Process Worker Process Worker Process Manager Process SOCK Client GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html GET / HTTP/1.1 Host: example.com Event Driven
    • Worker Process Worker Process Worker Process Worker Process Manager Process SOCK Client GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html Event Driven
    • Monoceros Workers that inherits“Starlet” not C10K ready
    • Event Driven Manager Process C10K ready built with AnyEvent
    • KeepAlive Benchmark like a Browser 1) connect 2) do requests certain number 3) leave alone a socket 4) timeout and close
    • 250 conn / 200 reqs Starlet Monoceros Total time (sec) 54.51 8.74 Failed reqs 971 0
    • Plack/PSGI Basics
    • PSGI = specification Plack = implementation
    • PSGI Interface my $app = sub { my $env = shift; ... return [200, [‘Content-Type’ => ‘text/html’], [‘Hello World’] ]; };
    • PSGI environment hash * CGI keys REQUEST_METHOD,SCRIPT_NAME, PATH_INFO,REQUEST_URI, QUERY_STRING,SERVER_PROTOCOL,HTTP_* * PSGI-specific keys psgi.version, psgi.url_scheme, psgi.input, psgi.errors, psgi.multiprocess, psgi.streaming psgi.nonblocking $env->{...}
    • PSGI Response (1) ArrayRef [200, #status code [ ‘Content-Type’ => ‘text/html’, ‘Content-Length => 10 ], [ ‘Hello’, ‘World’ ] ];
    • PSGI Response (1’) arrayref+IO::Handle open my $fh, ‘<’, ‘/path/icon.jpg’; [200, [ ‘Content-Type’ => ‘image/jpeg’, ‘Content-Length => 123456789 ], $fh ];
    • PSGI Response (2) Delayed and Streaming sub { my $env = shift; return sub { my $responder = shift; ... $responder->([ 200, $headers, [$body] ]); } };
    • PSGI Response (2’) Delayed and Streaming return sub { my $responder = shift; my $writer = $responder->([200, $headers]); wait_for_events(sub { my $new_event = shift; if ($new_event) { $writer->write($new_event->as_json . "n"); } else { $writer->close; } }); };
    • Role of“PSGI Server”
    • PSGI Server “A PSGI Server is a Perl program providing an environment for a PSGI application to run in”
    • PSGI Server App $env $res Apache Nginx Apache Proxy Browser CGI mod_perl FCGI HTTP Perl direct
    • PSGI Server is called “Plack Handler”
    • Plack Handler Adaptor interface Plack and PSGI Server. Make PSGI Server to run with“plackup”
    • e.g. Starman Starman::Server = PSGI Server Plack::Handler::Starman = Plack Handler
    • Make PSGI/Plack Server a High Performance
    • Tiny Standalone PSGI Web Server
    • my $null_io = do { open my $io, "<", ""; $io }; my $app = sub { my $env = shift return [200,['Content-Type'=>'text/html'],['Hello','World',"n"]]; }; my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1, ); while ( my $conn = $listen->accept ) { my $env = { SERVER_PORT => '5000', SERVER_NAME => 'localhost', SCRIPT_NAME => '', REMOTE_ADDR => $conn->peerhost, 'psgi.version' => [ 1, 1 ], 'psgi.errors' => *STDERR, 'psgi.url_scheme' => 'http', 'psgi.run_once' => Plack::Util::FALSE, 'psgi.multithread' => Plack::Util::FALSE, 'psgi.multiprocess' => Plack::Util::FALSE, 'psgi.streaming' => Plack::Util::FALSE, 'psgi.nonblocking' => Plack::Util::FALSE, 'psgi.input' => $null_io, }; $conn->sysread( my $buf, 4096); my $reqlen = Plack::HTTPParser::parse_http_request($buf, $env); my $res = Plack::Util::run_app $app, $env; my @lines = ("HTTP/1.1 $res->[0] @{[ status_message($res->[0]) ]}015012"); for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]015012"; } push @lines, "Connection: close0151201512"; $conn->syswrite(join "",@lines); Plack::Util::foreach($res->[2], sub { $conn->syswrite(shift); }); $conn->close; } 60 lines
    • Listen and Accept
    • my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1, ); while ( my $conn = $listen->accept ) { ... }
    • Read a request
    • use Plack::HTTPParser qw/parse_http_request/; my $null_io = do { open my $io, "<", ""; $io }; while ( my $conn = $listen->accept ) { my $env = { SERVER_PORT => '5000', SERVER_NAME => 'localhost', SCRIPT_NAME => '', REMOTE_ADDR => $conn->peerhost, 'psgi.version' => [ 1, 1 ], 'psgi.errors' => *STDERR, 'psgi.url_scheme' => 'http', 'psgi.multiprocess' => Plack::Util::FALSE, 'psgi.streaming' => Plack::Util::FALSE, 'psgi.nonblocking' => Plack::Util::FALSE, 'psgi.input' => $null_io, }; $conn->sysread(my $buf, 4096); my $reqlen = parse_http_request($buf, $env);
    • Run App
    • my $res = $app->($env); or my $res = Plack::Util::run_app $app, $env;
    • Write a response
    • use HTTP::Status qw/status_message/; my $res = .. my @lines = ("HTTP/1.1 $res->[0] @{[ status_message($res->[0]) ]}015012"); for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]015012"; } push @lines, "Connection: close0151201512"; $conn->syswrite(join "",@lines); foreach my $buf ( @{$res->[2]} ) { $conn->syswrite($buf); }); $conn->close;
    • This PSGI Server has some problem * handle only one at once * no timeout * may not fast
    • Increase concurrency
    • Multi Process IO Multiplexing or Both
    • Preforking model Simple, Scaling
    • Manager
    • Manager bind listen
    • Worker accept Worker accept Worker accept Worker accept Manager bind listen fork fork fork fork
    • Worker accept Worker accept Worker accept Worker accept Manager bind listen fork fork fork fork Client Client ClientClient
    • use Parallel::Prefork; my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1, ); my $pm = Parallel::Prefork->new({ max_workers => 5, trap_signals => { TERM => 'TERM', HUP => 'TERM', } }); while ( $pm->signal_received ne 'TERM') { $pm->start(sub{ while ( my $conn = $listen->accept ) { my $env = {..}
    • NO Accept Serialization
    • os/kernel Worker accept Worker accept Worker accept Worker accept Manager bind listen Zzz.. Zzz.. Zzz.. Zzz..
    • os/kernel Worker accept Worker accept Worker accept Worker accept Client Manager bind listen Zzz.. Zzz.. Zzz.. Zzz..
    • os/kernel Worker accept Worker accept Worker accept Worker accept Client Manager bind listen Zzz.. Zzz.. Zzz.. Zzz..
    • os/kernel Worker accept Worker accept Worker accept Worker accept Client Manager bind listen Thundering Herd突然の負荷 WakeUp WakeUp WakeUp WakeUP
    • os/kernel Worker accept Worker accept Worker accept Worker accept Client Manager bind listen Thundering Herd突然の負荷 WakeUp WakeUp WakeUp WakeUP
    • Thundering Herd is an old story
    • Worker accept Worker accept Worker Worker Manager bind listen accept accept Client Zzz.. Zzz.. Zzz..Zzz..
    • Worker accept Worker accept Worker Worker Manager bind listen accept accept Client Zzz.. Zzz.. Zzz..Zzz..
    • Worker accept Worker accept Worker Worker Manager bind listen accept accept Client modern os/kernel Zzz.. Zzz.. Zzz..Zzz..
    • Worker accept Worker accept Worker Worker Manager bind listen accept accept Client modern os/kernel Zzz.. Zzz.. Zzz..Zzz..
    • Worker accept Worker accept Worker Worker Manager bind listen accept accept Client modern os/kernel Zzz.. Zzz..Zzz.. WakeUp
    • NO Accept Serialization (except for multiple interface)
    • TCP_DEFER_ACCEPT
    • Wake up a process when DATA arrived not established コネクションが完了したタイミングではなく、 データが到着した段階でプロセスを起こします
    • client A client B GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK SYN ACK SYN+ACK GET / HTTP/1.0 Host: example.com Connection: keep-alive default defer_accept Accept RunApp block to read
    • RunApp client A client B GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK SYN ACK SYN+ACK GET / HTTP/1.0 Host: example.com Connection: keep-alive default defer_accept Accept RunApp Accept block to read
    • RunApp client A client B GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK SYN ACK SYN+ACK GET / HTTP/1.0 Host: example.com Connection: keep-alive default defer_accept Accept RunApp Accept block to read idle
    • RunApp client A client B GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK SYN ACK SYN+ACK GET / HTTP/1.0 Host: example.com Connection: keep-alive default defer_accept Accept RunApp Accept RunApp Accept RunApp Accept block to read idle
    • use Socket qw(IPPROTO_TCP); my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1, ); if ($^O eq 'linux') { setsockopt($listen, IPPROTO_TCP, 9, 1); }
    • timeout to read header
    • alarm
    • my $READ_TIMEOUT = 5; eval { local $SIG{ALRM} = sub { die "Timed outn"; }; alarm( $READ_TIMEOUT ); $conn->sysread(my $buf, 4096); }; alarm(0); next if ( $@ && $@ =~ /Timed out/ ); my $reqlen = parse_http_request($buf, $env);
    • nonblocking + select
    • use IO::Select; my $READ_TIMEOUT = 5; while( my $conn = $listen->accept ) { $conn->blocking(0); my $select = IO::Select->new($conn); my @ready = $select->can_read($READ_TIMEOUT); next unless @ready; $conn->sysread($buf, 4096); my $reqlen = parse_http_request($buf, $env);
    • alarm vs. nonblocking + select
    • Fewer syscalls is good
    • Hardwares User Application OS/Kernel system calls listen,fork, accept, read, write, select, alarm Worker Worker Worker Worker Worker
    • alarm
    • rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 rt_sigaction(SIGALRM, {0x47e5b0, [], SA_RESTORER, 0x7ff7d6e0cba0}, {SIG_DFL, [], SA_RESTORER, 0x7ff7d6e0cba0}, 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 alarm(5) = 0 read(7, "GET / HTTP/1.1rnUser-Agent: curl"..., 65536) = 155 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, 0x7ff7d6e0cba0}, {0x47e5b0, [], SA_RESTORER, 0x7ff7d6e0cba0}, 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 alarm(0) = 5 9 syscalls
    • non-blocking + select
    • fcntl(5, F_GETFL) = 0x2 (flags O_RDWR) fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 select(8, [5], NULL, [5], {300, 0}) = 1 (in [5], left {299, 999897}) read(5, "GET / HTTP/1.1rnUser-Agent: curl"..., 131072) = 155 4 syscalls
    • Parse a request with“C”
    • $ cpanm HTTP::Parser::XS Plack::HTTPParser uses H::P::XS if installed
    • TCP_NODELAY
    • When data was written TCP packets does not immediately send “TCP uses Nagle's algorithm to collect small packets for send all at once by default”
    • write(“foo”) write(“bar”) os/kernel network interface Application buffering “foobar”
    • write(“foo”) write(“bar”) os/kernel network interface Application “foo” TCP_NODELAY “bar”
    • Take care of excessive fragmentation of TCP packets
    • Write in once join content in Server
    • my @lines = ("HTTP/1.1 $res->[0] @{[ status_message($res->[0]) ]}015012"); for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]015012"; } push @lines, "Connection: close0151201512"; $conn->syswrite(join "",@lines, @{$res->[2]});
    • accept4, writev
    • Choose PSGI/Plack Server
    • CPAN has many PSGI Server & Plack::Hanlder:**
    • Standalone (HTTP::Server::PSGI) Default server for plackup Single process Web Server For development
    • Starman Preforking Web Server HTTP/1.1, HTTPS, Multiple interfaces, unix-domain socket, hot deploy using Server::Starter
    • Starlet Preforking Web Server HTTP/1.1(0.20~) hot deploy using Server::Starter Simple and Fast
    • Monoceros C10K Ready Preforking Web Server HTTP/1.1 hot deploy using Server::Starter
    • Twiggy based on AnyEvent nonblocking, streaming Single Process
    • Twiggy::Prefork based on Twiggy and Parallel::Prefork nonblocking, streaming Multi Process hot deploy using Server::Starter
    • Feersum Web server based on EV/libev nonblocking, streaming Single/Multi Process
    • How to choose PSGI Server
    • Single Process Multi Process CPU Intensive - Starlet Starman Monoceros Requires Event Driven Twiggy Feersum Twiggy::Prefork Feersum TypeofWebApplication
    • Finding Bottlenecks of Performance
    • use Devel::NYTProf
    • Flame Graph is awesome Profile nytprof.out.{PID} for preforking server $ nytprofhtml -f nytprof.out.1210 $ open nytprof/index.html
    • use strace or dtruss trace syscalls
    • $ strace -tt -s 200 -p {pid} 2>&1 | tee /tmp/trace.txt
    • Process 30929 attached - interrupt to quit 16:13:46.826828 accept(4, {sa_family=AF_INET, sin_port=htons(43783), sin_addr=inet_addr("127. 16:13:48.916233 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.916392 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.916493 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.916573 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.916661 fcntl(5, F_SETFD, FD_CLOEXEC) = 0 16:13:48.916873 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR) 16:13:48.916959 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 16:13:48.917095 setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 0 16:13:48.917362 read(5, "GET / HTTP/1.0rnHost: 127.0.0.1:5005rnUser-Agent: ApacheBench/2. 16:13:48.917613 brk(0x1e8e000) = 0x1e8e000 16:13:48.917746 gettimeofday({1379402028, 917802}, NULL) = 0 16:13:48.917953 write(5, "HTTP/1.1 200 OKrnDate: Tue, 17 Sep 2013 07:13:48 GMTrnServer: P 16:13:48.918187 close(5) = 0 16:13:48.918428 accept(4, {sa_family=AF_INET, sin_port=htons(43793), sin_addr=inet_addr("127. 16:13:48.923736 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.923843 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.923924 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.924461 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.924600 fcntl(5, F_SETFD, FD_CLOEXEC) = 0 16:13:48.924762 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR) 16:13:48.924853 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 16:13:48.924939 setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 0 16:13:48.925162 read(5, "GET / HTTP/1.0rnHost: 127.0.0.1:5005rnUser-Agent: ApacheBench/2. 16:13:48.925445 gettimeofday({1379402028, 925494}, NULL) = 0 16:13:48.925629 write(5, "HTTP/1.1 200 OKrnDate: Tue, 17 Sep 2013 07:13:48 GMTrnServer: P 16:13:48.925854 close(5) = 0 16:13:48.926084 accept(4, {sa_family=AF_INET, sin_port=htons(43803), sin_addr=inet_addr("127. 16:13:48.930480 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.930626 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.930744 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.930838 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.930915 fcntl(5, F_SETFD, FD_CLOEXEC) = 0 16:13:48.931070 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)
    • in conclusion
    • PSGI Server get Faster.
    • PSGI/Plack Rocks Stable, Fast Found problems? RT, GitHub Issue, PullReqs IRC #perl @kazeburo
    • #fin. Thank you!