How to build a High Performance PSGI/Plack Server
PSGI/Plack・Monocerosで学ぶ
ハイパフォーマンス
Webアプリケーションサーバの作り方
YAPC::Asia 2013 Tok...
Me
• 長野雅広 Masahiro Nagano
• @kazeburo
• PAUSE:KAZEBURO
• Operations Engineer, Site Reliability
• LINE Corp.
Development su...
livedoorBlog
One of the largest Blog Hosting Service in Japan
livedoorBlog uses
Perl (5.16and 5.8)
Carton
Plack/PSGI and mod_perl
$ curl -I http://blog.livedoor.jp/staff/| grep Server
Server: Plack::Handler::Starlet
Starlet handles
1 Billion(10億) reqs/day
To get over
this burst traffic,
We need to improve
Performance
across all layers
この負荷を乗り切るために様々なレイヤーで最適化をしています
Layers
Hardware / Network
OS
App Server
Routing
Cache Logic
SQL
Template Engine
RDBMS
Cached
Web Server
Hardware / Network
OS
Routing
Cache Logic
SQL
Template Engine
RDBMS
Cached
Web Server
Today’s Topic
Hardware / Network
OS
Routing
Cache Logic
SQL
Template Engine
RDBMS
Cached
Web Server
Today’s Topic
App Server
By the way..
“Open & Share”
is our driver
for excellence.
And LOVE CPAN/OSS
Open & Share は私たちの目指すところです。
CPAN/OSSを多く使い、また貢献もしています
Improving
Performance of livedoorBlog
directly linked
Performance of Plack/Starlet
on CPAN
livedoorBlogのパフォーマンス改善で行った事は
CP...
0
3500
7000
10500
14000 13083
6241
“Hello World”Reqs/Sec
Plack 1.0016 1.0029
Starlet 0.16 0.20
2013/02 2013/09
mod_perl er...
Monoceros
Monoceros is
a yet another
Plack/PSGI Server
for Performance
the Goal
Reduce
TCP 3way hand shake
between
Proxy and PSGI Server
Reverse
Proxy
App
Server
GET / HTTP/1.1
Host: example.com
SYN
ACK
SYN+ACK
HTTP/1.1 200 OK
Content-Type: text/html
FIN
ACK
...
HTTP/1.0-1.1 have
KeepAlive
Reverse
Proxy
App
Server
GET / HTTP/1.0
Host: example.com
Connection: keep-alive
SYN
ACK
SYN+ACK
HTTP/1.0 200 OK
Content-T...
C10K problem
nginx
C10K
Ready
Reverse
Proxy
nginx
nginx
Starlet
Starman
App
Server
KeepAlive Req
KeepAlive Req
KeepAlive Req
Starman, Starlet’s
Preforking model requires
1 connection per 1 process
By default
Starman: 5 procs
Starlet: 10 procs
Monoceros adopts
Preforking model,
But C10K ready
Worker
Process
Worker
Process
Worker
Process
Worker
Process
Manager
Process
SOCK
Client
GET / HTTP/1.1
Host: example.com
2...
Worker
Process
Worker
Process
Worker
Process
Worker
Process
Manager
Process
SOCK
Client
GET / HTTP/1.1
Host: example.com
2...
Worker
Process
Worker
Process
Worker
Process
Worker
Process
Manager
Process
SOCK
Client
GET / HTTP/1.1
Host: example.com
2...
Worker
Process
Worker
Process
Worker
Process
Worker
Process
Manager
Process
SOCK
Client
GET / HTTP/1.1
Host: example.com
2...
Monoceros Workers
that inherits“Starlet”
not C10K ready
Event Driven
Manager Process
C10K ready
built with AnyEvent
KeepAlive Benchmark
like a Browser
1) connect
2) do requests certain number
3) leave alone a socket
4) timeout and close
250 conn / 200 reqs
Starlet Monoceros
Total time (sec) 54.51 8.74
Failed reqs 971 0
Plack/PSGI
Basics
PSGI = specification
Plack = implementation
PSGI Interface
my $app = sub {
my $env = shift;
...
return [200,
[‘Content-Type’ => ‘text/html’],
[‘Hello World’]
];
};
PSGI environment hash
* CGI keys
REQUEST_METHOD,SCRIPT_NAME,
PATH_INFO,REQUEST_URI,
QUERY_STRING,SERVER_PROTOCOL,HTTP_*
* ...
PSGI Response (1)
ArrayRef
[200, #status code
[
‘Content-Type’ => ‘text/html’,
‘Content-Length => 10
],
[
‘Hello’,
‘World’...
PSGI Response (1’)
arrayref+IO::Handle
open my $fh, ‘<’, ‘/path/icon.jpg’;
[200,
[
‘Content-Type’ => ‘image/jpeg’,
‘Conten...
PSGI Response (2)
Delayed and Streaming
sub {
my $env = shift;
return sub {
my $responder = shift;
...
$responder->([ 200,...
PSGI Response (2’)
Delayed and Streaming
return sub {
my $responder = shift;
my $writer = $responder->([200, $headers]);
w...
Role of“PSGI Server”
PSGI Server
“A PSGI Server is a Perl program
providing an environment for a PSGI
application to run in”
PSGI
Server
App
$env
$res
Apache
Nginx
Apache
Proxy
Browser
CGI
mod_perl
FCGI
HTTP
Perl direct
PSGI Server
is called
“Plack Handler”
Plack Handler
Adaptor interface Plack and PSGI Server.
Make PSGI Server to run
with“plackup”
e.g. Starman
Starman::Server
= PSGI Server
Plack::Handler::Starman
= Plack Handler
Make
PSGI/Plack
Server
a High Performance
Tiny Standalone
PSGI Web Server
my $null_io = do { open my $io, "<", ""; $io };
my $app = sub {
my $env = shift
return [200,['Content-Type'=>'text/html'],...
Listen and Accept
my $listen = IO::Socket::INET->new(
Listen => 5,
LocalAddr => 'localhost',
LocalPort => 5000,
ReuseAddr => 1,
);
while ( m...
Read a request
use Plack::HTTPParser qw/parse_http_request/;
my $null_io = do { open my $io, "<", ""; $io };
while ( my $conn = $listen->...
Run App
my $res = $app->($env);
or
my $res = Plack::Util::run_app $app, $env;
Write a response
use HTTP::Status qw/status_message/;
my $res = ..
my @lines = ("HTTP/1.1 $res->[0] 
@{[ status_message($res->[0]) ]}015012...
This PSGI Server
has some problem
* handle only one at once
* no timeout
* may not fast
Increase concurrency
Multi Process
IO Multiplexing
or
Both
Preforking model
Simple, Scaling
Manager
Manager
bind
listen
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Manager
bind
listen
fork fork fork fork
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Manager
bind
listen
fork fork fork fork
Client Client ClientClient
use Parallel::Prefork;
my $listen = IO::Socket::INET->new(
Listen => 5,
LocalAddr => 'localhost',
LocalPort => 5000,
Reuse...
NO Accept
Serialization
os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Manager
bind
listen
Zzz.. Zzz.. Zzz.. Zzz..
os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Client
Manager
bind
listen
Zzz.. Zzz.. Zzz.. Zzz..
os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Client
Manager
bind
listen
Zzz.. Zzz.. Zzz.. Zzz..
os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Client
Manager
bind
listen
Thundering Herd突然の負荷
WakeUp W...
os/kernel
Worker
accept
Worker
accept
Worker
accept
Worker
accept
Client
Manager
bind
listen
Thundering Herd突然の負荷
WakeUp W...
Thundering Herd
is an old story
Worker
accept
Worker
accept
Worker Worker
Manager
bind
listen accept accept
Client
Zzz.. Zzz.. Zzz..Zzz..
Worker
accept
Worker
accept
Worker Worker
Manager
bind
listen accept accept
Client
Zzz.. Zzz.. Zzz..Zzz..
Worker
accept
Worker
accept
Worker Worker
Manager
bind
listen accept accept
Client
modern os/kernel
Zzz.. Zzz.. Zzz..Zzz..
Worker
accept
Worker
accept
Worker Worker
Manager
bind
listen accept accept
Client
modern os/kernel
Zzz.. Zzz.. Zzz..Zzz..
Worker
accept
Worker
accept
Worker Worker
Manager
bind
listen accept accept
Client
modern os/kernel
Zzz.. Zzz..Zzz..
WakeUp
NO Accept Serialization
(except for multiple interface)
TCP_DEFER_ACCEPT
Wake up a process
when DATA arrived
not established
コネクションが完了したタイミングではなく、
データが到着した段階でプロセスを起こします
client
A
client
B
GET / HTTP/1.0
Host: example.com
Connection: keep-alive
SYN
ACK
SYN+ACK
SYN
ACK
SYN+ACK
GET / HTTP/1.0
H...
RunApp
client
A
client
B
GET / HTTP/1.0
Host: example.com
Connection: keep-alive
SYN
ACK
SYN+ACK
SYN
ACK
SYN+ACK
GET / HTT...
RunApp
client
A
client
B
GET / HTTP/1.0
Host: example.com
Connection: keep-alive
SYN
ACK
SYN+ACK
SYN
ACK
SYN+ACK
GET / HTT...
RunApp
client
A
client
B
GET / HTTP/1.0
Host: example.com
Connection: keep-alive
SYN
ACK
SYN+ACK
SYN
ACK
SYN+ACK
GET / HTT...
use Socket qw(IPPROTO_TCP);
my $listen = IO::Socket::INET->new(
Listen => 5,
LocalAddr => 'localhost',
LocalPort => 5000,
...
timeout to
read header
alarm
my $READ_TIMEOUT = 5;
eval {
local $SIG{ALRM} = sub { die "Timed outn"; };
alarm( $READ_TIMEOUT );
$conn->sysread(my $buf,...
nonblocking + select
use IO::Select;
my $READ_TIMEOUT = 5;
while( my $conn = $listen->accept ) {
$conn->blocking(0);
my $select = IO::Select->n...
alarm
vs.
nonblocking + select
Fewer syscalls is good
Hardwares
User Application
OS/Kernel
system calls
listen,fork, accept, read, write, select, alarm
Worker Worker Worker Wor...
alarm
rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0
rt_sigaction(SIGALRM, {0x47e5b0, [], SA_RESTORER,
0x7ff7d6e0cba0}, {SIG_DFL, ...
non-blocking + select
fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
select(8, [5], NULL, [5], {300, 0}) = 1
(i...
Parse a request with“C”
$ cpanm HTTP::Parser::XS
Plack::HTTPParser uses H::P::XS if installed
TCP_NODELAY
When data was written
TCP packets does not
immediately send
“TCP uses Nagle's algorithm to
collect small packets
for send ...
write(“foo”)
write(“bar”)
os/kernel
network
interface
Application
buffering
“foobar”
write(“foo”)
write(“bar”)
os/kernel
network
interface
Application
“foo”
TCP_NODELAY
“bar”
Take care of excessive
fragmentation of TCP
packets
Write in once
join content in Server
my @lines = ("HTTP/1.1 $res->[0]
@{[ status_message($res->[0]) ]}015012");
for (my $i = 0; $i < @{$res->[1]}; $i += 2) {
n...
accept4, writev
Choose
PSGI/Plack
Server
CPAN has many
PSGI Server &
Plack::Hanlder:**
Standalone
(HTTP::Server::PSGI)
Default server for plackup
Single process Web Server
For development
Starman
Preforking Web Server
HTTP/1.1, HTTPS,
Multiple interfaces,
unix-domain socket,
hot deploy using Server::Starter
Starlet
Preforking Web Server
HTTP/1.1(0.20~)
hot deploy using Server::Starter
Simple and Fast
Monoceros
C10K Ready Preforking Web Server
HTTP/1.1
hot deploy using Server::Starter
Twiggy
based on AnyEvent
nonblocking, streaming
Single Process
Twiggy::Prefork
based on Twiggy and
Parallel::Prefork
nonblocking, streaming
Multi Process
hot deploy using Server::Starter
Feersum
Web server based on EV/libev
nonblocking, streaming
Single/Multi Process
How to choose
PSGI Server
Single
Process
Multi
Process
CPU Intensive -
Starlet
Starman
Monoceros
Requires
Event Driven
Twiggy
Feersum
Twiggy::Prefor...
Finding
Bottlenecks of
Performance
use Devel::NYTProf
Flame Graph is awesome
Profile nytprof.out.{PID}
for preforking server
$ nytprofhtml -f nytprof.out.1210
$ open nytprof/in...
use strace or dtruss
trace syscalls
$ strace -tt -s 200 -p {pid} 
2>&1 | tee /tmp/trace.txt
Process 30929 attached - interrupt to quit
16:13:46.826828 accept(4, {sa_family=AF_INET, sin_port=htons(43783), sin_addr=i...
in conclusion
PSGI Server get Faster.
PSGI/Plack Rocks
Stable, Fast
Found problems?
RT, GitHub Issue, PullReqs
IRC #perl @kazeburo
#fin. Thank you!
How to build a High Performance PSGI/Plack Server
Upcoming SlideShare
Loading in...5
×

How to build a High Performance PSGI/Plack Server

13,830

Published on

How to build a High Performance PSGI/Plack Server
PSGI/Plack・Monocerosで学ぶ ハイパフォーマンス Webアプリケーションサーバの作り方

Published in: Technology
0 Comments
22 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
13,830
On Slideshare
0
From Embeds
0
Number of Embeds
26
Actions
Shares
0
Downloads
44
Comments
0
Likes
22
Embeds 0
No embeds

No notes for slide

How to build a High Performance PSGI/Plack Server

  1. 1. How to build a High Performance PSGI/Plack Server PSGI/Plack・Monocerosで学ぶ ハイパフォーマンス Webアプリケーションサーバの作り方 YAPC::Asia 2013 Tokyo Masahiro Nagano / @kazeburo
  2. 2. Me • 長野雅広 Masahiro Nagano • @kazeburo • PAUSE:KAZEBURO • Operations Engineer, Site Reliability • LINE Corp. Development support LINE Family, livedoor
  3. 3. livedoorBlog One of the largest Blog Hosting Service in Japan
  4. 4. livedoorBlog uses
  5. 5. Perl (5.16and 5.8)
  6. 6. Carton
  7. 7. Plack/PSGI and mod_perl
  8. 8. $ curl -I http://blog.livedoor.jp/staff/| grep Server Server: Plack::Handler::Starlet
  9. 9. Starlet handles 1 Billion(10億) reqs/day
  10. 10. To get over this burst traffic, We need to improve Performance across all layers この負荷を乗り切るために様々なレイヤーで最適化をしています
  11. 11. Layers Hardware / Network OS App Server Routing Cache Logic SQL Template Engine RDBMS Cached Web Server
  12. 12. Hardware / Network OS Routing Cache Logic SQL Template Engine RDBMS Cached Web Server Today’s Topic
  13. 13. Hardware / Network OS Routing Cache Logic SQL Template Engine RDBMS Cached Web Server Today’s Topic App Server
  14. 14. By the way..
  15. 15. “Open & Share” is our driver for excellence. And LOVE CPAN/OSS Open & Share は私たちの目指すところです。 CPAN/OSSを多く使い、また貢献もしています
  16. 16. Improving Performance of livedoorBlog directly linked Performance of Plack/Starlet on CPAN livedoorBlogのパフォーマンス改善で行った事は CPAN上のPlack/Starletにも当然影響してきます
  17. 17. 0 3500 7000 10500 14000 13083 6241 “Hello World”Reqs/Sec Plack 1.0016 1.0029 Starlet 0.16 0.20 2013/02 2013/09 mod_perl era plack era
  18. 18. Monoceros
  19. 19. Monoceros is a yet another Plack/PSGI Server for Performance
  20. 20. the Goal
  21. 21. Reduce TCP 3way hand shake between Proxy and PSGI Server
  22. 22. Reverse Proxy App Server GET / HTTP/1.1 Host: example.com SYN ACK SYN+ACK HTTP/1.1 200 OK Content-Type: text/html FIN ACK GET /favicon.ico HTTP/1.1 Host: example.com SYN ACK SYN+ACK HTTP/1.1 404 NOT FOUND Content-Type: text/html FIN ACK
  23. 23. HTTP/1.0-1.1 have KeepAlive
  24. 24. Reverse Proxy App Server GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK HTTP/1.0 200 OK Content-Type: text/html Connection: keep-alive Content-Length: 941 GET /favicon.ico HTTP/1.1 Host: example.com HTTP/1.1 200 OK Content-Type: image/vnd.microsoft.icon Transfer-Encoding: chunked GET /site.css HTTP/1.1 Host: example.com HTTP/1.1 200 OK Content-Type: text/css Content-Length: 1013
  25. 25. C10K problem
  26. 26. nginx C10K Ready Reverse Proxy nginx nginx Starlet Starman App Server KeepAlive Req KeepAlive Req KeepAlive Req
  27. 27. Starman, Starlet’s Preforking model requires 1 connection per 1 process By default Starman: 5 procs Starlet: 10 procs
  28. 28. Monoceros adopts Preforking model, But C10K ready
  29. 29. Worker Process Worker Process Worker Process Worker Process Manager Process SOCK Client GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html
  30. 30. Worker Process Worker Process Worker Process Worker Process Manager Process SOCK Client GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html
  31. 31. Worker Process Worker Process Worker Process Worker Process Manager Process SOCK Client GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html GET / HTTP/1.1 Host: example.com Event Driven
  32. 32. Worker Process Worker Process Worker Process Worker Process Manager Process SOCK Client GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html GET / HTTP/1.1 Host: example.com 200 OK Content-Type: text/html Event Driven
  33. 33. Monoceros Workers that inherits“Starlet” not C10K ready
  34. 34. Event Driven Manager Process C10K ready built with AnyEvent
  35. 35. KeepAlive Benchmark like a Browser 1) connect 2) do requests certain number 3) leave alone a socket 4) timeout and close
  36. 36. 250 conn / 200 reqs Starlet Monoceros Total time (sec) 54.51 8.74 Failed reqs 971 0
  37. 37. Plack/PSGI Basics
  38. 38. PSGI = specification Plack = implementation
  39. 39. PSGI Interface my $app = sub { my $env = shift; ... return [200, [‘Content-Type’ => ‘text/html’], [‘Hello World’] ]; };
  40. 40. PSGI environment hash * CGI keys REQUEST_METHOD,SCRIPT_NAME, PATH_INFO,REQUEST_URI, QUERY_STRING,SERVER_PROTOCOL,HTTP_* * PSGI-specific keys psgi.version, psgi.url_scheme, psgi.input, psgi.errors, psgi.multiprocess, psgi.streaming psgi.nonblocking $env->{...}
  41. 41. PSGI Response (1) ArrayRef [200, #status code [ ‘Content-Type’ => ‘text/html’, ‘Content-Length => 10 ], [ ‘Hello’, ‘World’ ] ];
  42. 42. PSGI Response (1’) arrayref+IO::Handle open my $fh, ‘<’, ‘/path/icon.jpg’; [200, [ ‘Content-Type’ => ‘image/jpeg’, ‘Content-Length => 123456789 ], $fh ];
  43. 43. PSGI Response (2) Delayed and Streaming sub { my $env = shift; return sub { my $responder = shift; ... $responder->([ 200, $headers, [$body] ]); } };
  44. 44. PSGI Response (2’) Delayed and Streaming return sub { my $responder = shift; my $writer = $responder->([200, $headers]); wait_for_events(sub { my $new_event = shift; if ($new_event) { $writer->write($new_event->as_json . "n"); } else { $writer->close; } }); };
  45. 45. Role of“PSGI Server”
  46. 46. PSGI Server “A PSGI Server is a Perl program providing an environment for a PSGI application to run in”
  47. 47. PSGI Server App $env $res Apache Nginx Apache Proxy Browser CGI mod_perl FCGI HTTP Perl direct
  48. 48. PSGI Server is called “Plack Handler”
  49. 49. Plack Handler Adaptor interface Plack and PSGI Server. Make PSGI Server to run with“plackup”
  50. 50. e.g. Starman Starman::Server = PSGI Server Plack::Handler::Starman = Plack Handler
  51. 51. Make PSGI/Plack Server a High Performance
  52. 52. Tiny Standalone PSGI Web Server
  53. 53. my $null_io = do { open my $io, "<", ""; $io }; my $app = sub { my $env = shift return [200,['Content-Type'=>'text/html'],['Hello','World',"n"]]; }; my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1, ); while ( my $conn = $listen->accept ) { my $env = { SERVER_PORT => '5000', SERVER_NAME => 'localhost', SCRIPT_NAME => '', REMOTE_ADDR => $conn->peerhost, 'psgi.version' => [ 1, 1 ], 'psgi.errors' => *STDERR, 'psgi.url_scheme' => 'http', 'psgi.run_once' => Plack::Util::FALSE, 'psgi.multithread' => Plack::Util::FALSE, 'psgi.multiprocess' => Plack::Util::FALSE, 'psgi.streaming' => Plack::Util::FALSE, 'psgi.nonblocking' => Plack::Util::FALSE, 'psgi.input' => $null_io, }; $conn->sysread( my $buf, 4096); my $reqlen = Plack::HTTPParser::parse_http_request($buf, $env); my $res = Plack::Util::run_app $app, $env; my @lines = ("HTTP/1.1 $res->[0] @{[ status_message($res->[0]) ]}015012"); for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]015012"; } push @lines, "Connection: close0151201512"; $conn->syswrite(join "",@lines); Plack::Util::foreach($res->[2], sub { $conn->syswrite(shift); }); $conn->close; } 60 lines
  54. 54. Listen and Accept
  55. 55. my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1, ); while ( my $conn = $listen->accept ) { ... }
  56. 56. Read a request
  57. 57. use Plack::HTTPParser qw/parse_http_request/; my $null_io = do { open my $io, "<", ""; $io }; while ( my $conn = $listen->accept ) { my $env = { SERVER_PORT => '5000', SERVER_NAME => 'localhost', SCRIPT_NAME => '', REMOTE_ADDR => $conn->peerhost, 'psgi.version' => [ 1, 1 ], 'psgi.errors' => *STDERR, 'psgi.url_scheme' => 'http', 'psgi.multiprocess' => Plack::Util::FALSE, 'psgi.streaming' => Plack::Util::FALSE, 'psgi.nonblocking' => Plack::Util::FALSE, 'psgi.input' => $null_io, }; $conn->sysread(my $buf, 4096); my $reqlen = parse_http_request($buf, $env);
  58. 58. Run App
  59. 59. my $res = $app->($env); or my $res = Plack::Util::run_app $app, $env;
  60. 60. Write a response
  61. 61. use HTTP::Status qw/status_message/; my $res = .. my @lines = ("HTTP/1.1 $res->[0] @{[ status_message($res->[0]) ]}015012"); for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]015012"; } push @lines, "Connection: close0151201512"; $conn->syswrite(join "",@lines); foreach my $buf ( @{$res->[2]} ) { $conn->syswrite($buf); }); $conn->close;
  62. 62. This PSGI Server has some problem * handle only one at once * no timeout * may not fast
  63. 63. Increase concurrency
  64. 64. Multi Process IO Multiplexing or Both
  65. 65. Preforking model Simple, Scaling
  66. 66. Manager
  67. 67. Manager bind listen
  68. 68. Worker accept Worker accept Worker accept Worker accept Manager bind listen fork fork fork fork
  69. 69. Worker accept Worker accept Worker accept Worker accept Manager bind listen fork fork fork fork Client Client ClientClient
  70. 70. use Parallel::Prefork; my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1, ); my $pm = Parallel::Prefork->new({ max_workers => 5, trap_signals => { TERM => 'TERM', HUP => 'TERM', } }); while ( $pm->signal_received ne 'TERM') { $pm->start(sub{ while ( my $conn = $listen->accept ) { my $env = {..}
  71. 71. NO Accept Serialization
  72. 72. os/kernel Worker accept Worker accept Worker accept Worker accept Manager bind listen Zzz.. Zzz.. Zzz.. Zzz..
  73. 73. os/kernel Worker accept Worker accept Worker accept Worker accept Client Manager bind listen Zzz.. Zzz.. Zzz.. Zzz..
  74. 74. os/kernel Worker accept Worker accept Worker accept Worker accept Client Manager bind listen Zzz.. Zzz.. Zzz.. Zzz..
  75. 75. os/kernel Worker accept Worker accept Worker accept Worker accept Client Manager bind listen Thundering Herd突然の負荷 WakeUp WakeUp WakeUp WakeUP
  76. 76. os/kernel Worker accept Worker accept Worker accept Worker accept Client Manager bind listen Thundering Herd突然の負荷 WakeUp WakeUp WakeUp WakeUP
  77. 77. Thundering Herd is an old story
  78. 78. Worker accept Worker accept Worker Worker Manager bind listen accept accept Client Zzz.. Zzz.. Zzz..Zzz..
  79. 79. Worker accept Worker accept Worker Worker Manager bind listen accept accept Client Zzz.. Zzz.. Zzz..Zzz..
  80. 80. Worker accept Worker accept Worker Worker Manager bind listen accept accept Client modern os/kernel Zzz.. Zzz.. Zzz..Zzz..
  81. 81. Worker accept Worker accept Worker Worker Manager bind listen accept accept Client modern os/kernel Zzz.. Zzz.. Zzz..Zzz..
  82. 82. Worker accept Worker accept Worker Worker Manager bind listen accept accept Client modern os/kernel Zzz.. Zzz..Zzz.. WakeUp
  83. 83. NO Accept Serialization (except for multiple interface)
  84. 84. TCP_DEFER_ACCEPT
  85. 85. Wake up a process when DATA arrived not established コネクションが完了したタイミングではなく、 データが到着した段階でプロセスを起こします
  86. 86. client A client B GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK SYN ACK SYN+ACK GET / HTTP/1.0 Host: example.com Connection: keep-alive default defer_accept Accept RunApp block to read
  87. 87. RunApp client A client B GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK SYN ACK SYN+ACK GET / HTTP/1.0 Host: example.com Connection: keep-alive default defer_accept Accept RunApp Accept block to read
  88. 88. RunApp client A client B GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK SYN ACK SYN+ACK GET / HTTP/1.0 Host: example.com Connection: keep-alive default defer_accept Accept RunApp Accept block to read idle
  89. 89. RunApp client A client B GET / HTTP/1.0 Host: example.com Connection: keep-alive SYN ACK SYN+ACK SYN ACK SYN+ACK GET / HTTP/1.0 Host: example.com Connection: keep-alive default defer_accept Accept RunApp Accept RunApp Accept RunApp Accept block to read idle
  90. 90. use Socket qw(IPPROTO_TCP); my $listen = IO::Socket::INET->new( Listen => 5, LocalAddr => 'localhost', LocalPort => 5000, ReuseAddr => 1, ); if ($^O eq 'linux') { setsockopt($listen, IPPROTO_TCP, 9, 1); }
  91. 91. timeout to read header
  92. 92. alarm
  93. 93. my $READ_TIMEOUT = 5; eval { local $SIG{ALRM} = sub { die "Timed outn"; }; alarm( $READ_TIMEOUT ); $conn->sysread(my $buf, 4096); }; alarm(0); next if ( $@ && $@ =~ /Timed out/ ); my $reqlen = parse_http_request($buf, $env);
  94. 94. nonblocking + select
  95. 95. use IO::Select; my $READ_TIMEOUT = 5; while( my $conn = $listen->accept ) { $conn->blocking(0); my $select = IO::Select->new($conn); my @ready = $select->can_read($READ_TIMEOUT); next unless @ready; $conn->sysread($buf, 4096); my $reqlen = parse_http_request($buf, $env);
  96. 96. alarm vs. nonblocking + select
  97. 97. Fewer syscalls is good
  98. 98. Hardwares User Application OS/Kernel system calls listen,fork, accept, read, write, select, alarm Worker Worker Worker Worker Worker
  99. 99. alarm
  100. 100. rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 rt_sigaction(SIGALRM, {0x47e5b0, [], SA_RESTORER, 0x7ff7d6e0cba0}, {SIG_DFL, [], SA_RESTORER, 0x7ff7d6e0cba0}, 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 alarm(5) = 0 read(7, "GET / HTTP/1.1rnUser-Agent: curl"..., 65536) = 155 rt_sigprocmask(SIG_BLOCK, [ALRM], [], 8) = 0 rt_sigaction(SIGALRM, {SIG_DFL, [], SA_RESTORER, 0x7ff7d6e0cba0}, {0x47e5b0, [], SA_RESTORER, 0x7ff7d6e0cba0}, 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 alarm(0) = 5 9 syscalls
  101. 101. non-blocking + select
  102. 102. fcntl(5, F_GETFL) = 0x2 (flags O_RDWR) fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 select(8, [5], NULL, [5], {300, 0}) = 1 (in [5], left {299, 999897}) read(5, "GET / HTTP/1.1rnUser-Agent: curl"..., 131072) = 155 4 syscalls
  103. 103. Parse a request with“C”
  104. 104. $ cpanm HTTP::Parser::XS Plack::HTTPParser uses H::P::XS if installed
  105. 105. TCP_NODELAY
  106. 106. When data was written TCP packets does not immediately send “TCP uses Nagle's algorithm to collect small packets for send all at once by default”
  107. 107. write(“foo”) write(“bar”) os/kernel network interface Application buffering “foobar”
  108. 108. write(“foo”) write(“bar”) os/kernel network interface Application “foo” TCP_NODELAY “bar”
  109. 109. Take care of excessive fragmentation of TCP packets
  110. 110. Write in once join content in Server
  111. 111. my @lines = ("HTTP/1.1 $res->[0] @{[ status_message($res->[0]) ]}015012"); for (my $i = 0; $i < @{$res->[1]}; $i += 2) { next if $res->[1][$i] eq 'Connection'; push @lines, "$res->[1][$i]: $res->[1][$i + 1]015012"; } push @lines, "Connection: close0151201512"; $conn->syswrite(join "",@lines, @{$res->[2]});
  112. 112. accept4, writev
  113. 113. Choose PSGI/Plack Server
  114. 114. CPAN has many PSGI Server & Plack::Hanlder:**
  115. 115. Standalone (HTTP::Server::PSGI) Default server for plackup Single process Web Server For development
  116. 116. Starman Preforking Web Server HTTP/1.1, HTTPS, Multiple interfaces, unix-domain socket, hot deploy using Server::Starter
  117. 117. Starlet Preforking Web Server HTTP/1.1(0.20~) hot deploy using Server::Starter Simple and Fast
  118. 118. Monoceros C10K Ready Preforking Web Server HTTP/1.1 hot deploy using Server::Starter
  119. 119. Twiggy based on AnyEvent nonblocking, streaming Single Process
  120. 120. Twiggy::Prefork based on Twiggy and Parallel::Prefork nonblocking, streaming Multi Process hot deploy using Server::Starter
  121. 121. Feersum Web server based on EV/libev nonblocking, streaming Single/Multi Process
  122. 122. How to choose PSGI Server
  123. 123. Single Process Multi Process CPU Intensive - Starlet Starman Monoceros Requires Event Driven Twiggy Feersum Twiggy::Prefork Feersum TypeofWebApplication
  124. 124. Finding Bottlenecks of Performance
  125. 125. use Devel::NYTProf
  126. 126. Flame Graph is awesome Profile nytprof.out.{PID} for preforking server $ nytprofhtml -f nytprof.out.1210 $ open nytprof/index.html
  127. 127. use strace or dtruss trace syscalls
  128. 128. $ strace -tt -s 200 -p {pid} 2>&1 | tee /tmp/trace.txt
  129. 129. Process 30929 attached - interrupt to quit 16:13:46.826828 accept(4, {sa_family=AF_INET, sin_port=htons(43783), sin_addr=inet_addr("127. 16:13:48.916233 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.916392 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.916493 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.916573 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.916661 fcntl(5, F_SETFD, FD_CLOEXEC) = 0 16:13:48.916873 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR) 16:13:48.916959 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 16:13:48.917095 setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 0 16:13:48.917362 read(5, "GET / HTTP/1.0rnHost: 127.0.0.1:5005rnUser-Agent: ApacheBench/2. 16:13:48.917613 brk(0x1e8e000) = 0x1e8e000 16:13:48.917746 gettimeofday({1379402028, 917802}, NULL) = 0 16:13:48.917953 write(5, "HTTP/1.1 200 OKrnDate: Tue, 17 Sep 2013 07:13:48 GMTrnServer: P 16:13:48.918187 close(5) = 0 16:13:48.918428 accept(4, {sa_family=AF_INET, sin_port=htons(43793), sin_addr=inet_addr("127. 16:13:48.923736 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.923843 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.923924 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.924461 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.924600 fcntl(5, F_SETFD, FD_CLOEXEC) = 0 16:13:48.924762 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR) 16:13:48.924853 fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0 16:13:48.924939 setsockopt(5, SOL_TCP, TCP_NODELAY, [1], 4) = 0 16:13:48.925162 read(5, "GET / HTTP/1.0rnHost: 127.0.0.1:5005rnUser-Agent: ApacheBench/2. 16:13:48.925445 gettimeofday({1379402028, 925494}, NULL) = 0 16:13:48.925629 write(5, "HTTP/1.1 200 OKrnDate: Tue, 17 Sep 2013 07:13:48 GMTrnServer: P 16:13:48.925854 close(5) = 0 16:13:48.926084 accept(4, {sa_family=AF_INET, sin_port=htons(43803), sin_addr=inet_addr("127. 16:13:48.930480 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.930626 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.930744 ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff2fb61730) = -1 EINVAL (Invalid 16:13:48.930838 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) 16:13:48.930915 fcntl(5, F_SETFD, FD_CLOEXEC) = 0 16:13:48.931070 fcntl(5, F_GETFL) = 0x2 (flags O_RDWR)
  130. 130. in conclusion
  131. 131. PSGI Server get Faster.
  132. 132. PSGI/Plack Rocks Stable, Fast Found problems? RT, GitHub Issue, PullReqs IRC #perl @kazeburo
  133. 133. #fin. Thank you!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×