Apache::LogFormat::Compiler YAPC::Asia 2013 Tokyo LT-Thon

  • 5,899 views
Uploaded on

Apache::LogFormat::Compiler YAPC::Asia 2013 Tokyo LT-Thon

Apache::LogFormat::Compiler YAPC::Asia 2013 Tokyo LT-Thon

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
5,899
On Slideshare
0
From Embeds
0
Number of Embeds
19

Actions

Shares
Downloads
3
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. ALFCYAPC::Asia 2013 Tokyo 前夜祭 LT-thon Masahiro Nagano @kazeburo
  • 2. Me • 長野雅広 Masahiro Nagano • @kazeburo • PAUSE: KAZEBURO • Operations Engineer, Site Reliability • LINE Corp. Development support on LINE Familly, livedoor
  • 3. ALFC =
  • 4. Apache::LogFormat::Compiler
  • 5. Convert
  • 6. ‘%h %l %u %t "%r" %>s %b’
  • 7. to perl-code
  • 8. my $code = <<EOF; sub { my ($env,$res,$length,$reqtime,$time) = @_; $time = time() if ! defined $time; my @lt = localtime($time);; my $t = sprintf '%02d/%s/%04d:%02d:%02d:%02d %s', $lt[3], $abbr[$lt[4]], $lt[5]+1900, $lt[2], $lt[1], $lt[0], $tz; q!! . ($env->{REMOTE_ADDR} || '-') . q! ! . '-' . q! ! . ($env->{REMOTE_USER} || '-') . q! ! . "[" . $t . "]" . q! "! . _safe($env->{REQUEST_METHOD}) . " " . _safe($env- >{REQUEST_URI}) . " " . $env->{SERVER_PROTOCOL} . q!" ! . $res->[0] . q! ! . (defined $length ? $length : '-') . q!! } EOF
  • 9. and compile
  • 10. my $log_line = eval $code; $log_line->($env,$res);
  • 11. SYNOPSIS
  • 12. my $log_handler = Apache::LogFormat::Compiler->new( '%h %l %u %t "%r" %>s %b' ); warn $log_handler->log_line($env,$res);
  • 13. use Plack::Builder; use Time::HiRes qw/gettimeofday tv_interval/; use Apache::LogFormat::Compiler; my $log_handler = Apache::LogFormat::Compiler->new( '%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-agent}i" %D' ); my $compile_log_app = builder { enable sub { my $app = shift; sub { my $env = shift; my $t0 = [gettimeofday]; my $res = $app->(); my $reqtime = int(tv_interval($t0) * 1_000_000); $env->{psgi.error}->print($log_handler->log_line( $env,$res,6,$reqtime, $t0->[0])); } }; $app };
  • 14. OR
  • 15. Plack::Middleware::AxsLog yet another AccessLog Middleware
  • 16. use Plack::Builder; builder { enable 'AxsLog', combined => 1, response_time => 1; $app };
  • 17. OR
  • 18. Plack::Middleware::AccessLog Plack (1.0023~)
  • 19. Benchmark
  • 20. 0 10000 20000 30000 40000 50000 60000 51262 8826 Plack-1.0022 Plack-1.0029 Req/Sec
  • 21. A
  • 22.   Story
  • 23.   of
  • 24.   ALFC とある Ops と Dev のおはなし∼
  • 25. I'm
  • 26.   tuning
  • 27.   up
  • 28.   this
  • 29.   web
  • 30.   application.
  • 31.    NYTProf
  • 32.   said
  • 33.   that
  • 34.   PM::AccessLog
  • 35.   is
  • 36.   slow
  • 37.    Yes,
  • 38.   I
  • 39.   know.
  • 40.   It's
  • 41.   slow
  • 42.   because..
  • 43.    Ops Dev
  • 44. Why PM::AccessLog was slow?
  • 45. my %char_handler = ( h => sub { $env->{REMOTE_ADDR} || '-' }, t => sub { "[" . $strftime->('%d/%b/%Y:%H:%M:%S %z', localtime) . "]" }, r => sub { _safe($env->{REQUEST_METHOD}) . " " . _safe($env->{REQUEST_URI}) . " " . $env->{SERVER_PROTOCOL} }, ); my $char_handler = sub { my $char = shift; exists $char_handler{$char} ? $char_handler{$char}->($char) : "-"; }; $fmt =~ s! (?: %{(.+?)}([a-z]) | %(?:[<>])?([a-zA-Z%]) ) ! $1 ? $block_handler->($1, $2) : $char_handler->($3) !egx;
  • 46. I
  • 47.   can't
  • 48.   live
  • 49.   with
  • 50.   PM::AccessLog!! I
  • 51.   have
  • 52.   idea
  • 53.   for
  • 54.   speedup Ops Dev
  • 55. Solution: combine strings
  • 56. if ( $self->{format} eq 'common' ) { $log_line = _string($env->{REMOTE_ADDR}) . " " . '- ' . _string($env->{REMOTE_USER}) . " " . q![!. $t . q!] ! . _safe(q!"! . $env->{REQUEST_METHOD} . " " . $env->{REQUEST_URI} . " " . $env->{SERVER_PROTOCOL} . q!" !) . $res->[0] . " " . (defined $length ? "$length" : '-'); } elsif ( $self->{format} eq 'ltsv' ) { ... }
  • 57. This middleware was released as PM::AxsLog 2012-09-12
  • 58. This isn’t extensible, only supports 3 formats. common,combined,ltsv but Fast.
  • 59. 0 10000 20000 30000 40000 50000 60000 53315 8826 Plack-1.0022 AccessLog AxsLog-0.03 Req/Sec
  • 60. 2013-03-01
  • 61. Ah,
  • 62.   PM::AxsLog
  • 63.   can't
  • 64.   modify
  • 65.   log_format? I
  • 66.   want
  • 67.   to
  • 68.   store
  • 69.   a
  • 70.   value
  • 71.   of
  • 72.   X-Moe
  • 73.   to
  • 74.   logfile What's?
  • 75.   PM::AxsLog
  • 76.   is
  • 77.   fast
  • 78.   because
  • 79.    it
  • 80.   supports
  • 81.   only
  • 82.   fixed
  • 83.   format. Ooops,
  • 84.   I
  • 85.   should
  • 86.   return
  • 87.   to
  • 88.   use
  • 89.   PM::AccessLog... Wait!
  • 90.   I
  • 91.   have
  • 92.   another
  • 93.   idea
  • 94.    Ops Dev
  • 95. A few hours later, Ops released Apache::LogFormat::Compiler (2013-03-01)
  • 96. AxsLog changed to use ALFC
  • 97. 0 11000 22000 33000 44000 55000 5464053315 AxsLog-0.03 AxsLog-0.12 Req/Sec
  • 98. 2013-03-19
  • 99. @ops
  • 100.   Can
  • 101.   I
  • 102.   replace
  • 103.   PM::AccessLog
  • 104.   by
  • 105.   AxsLog? I
  • 106.   recommend
  • 107.   to
  • 108.   use
  • 109.   ALFC
  • 110.   in
  • 111.    PM::AccesssLog
  • 112.   rather
  • 113.   than
  • 114.   replace. Because
  • 115.   option
  • 116.   of
  • 117.   AxsLog
  • 118.   isn’t
  • 119.    comaptible
  • 120.   with
  • 121.   AccessLog Okay,
  • 122.   I
  • 123.   am
  • 124.   looking
  • 125.   forward
  • 126.   to
  • 127.   pull
  • 128.   request. Ops Mr. committer
  • 129. Plack-1.0023 was released with ALFC. And Ops removed text "Faster than AccessLog" from PM::AxsLog’s POD
  • 130. They lived happily ever after. End
  • 131. Appendix
  • 132. Different of AxsLog and AccessLog
  • 133. (1)
  • 134. AxsLog supports Duration by default. AccessLog doesn’t support it. AccessLog::Timed required
  • 135. use Plack::Builder; builder { enable 'AxsLog', format => ‘%h %l %u %t "%r" %>s %b %D’; $app };
  • 136. use Plack::Builder; builder { enable 'AccessLog::Timed', format => ‘%h %l %u %t "%r" %>s %b %D’; $app };
  • 137. (2)
  • 138. Timing of timestamp
  • 139. AxsLog retrieve time() at begining of a request. AccessLog retrieve time() at end of a request.
  • 140. sub call { my ($self,$env) = @_; my $time = time(); #before app my $res = $self->app->($env); $self->log_line($env,$res,$length,$time); } sub call { my ($self,$env) = @_; my $res = $self->app->($env); my $time = time(); #after app $self->log_line($env,$res,$length,$time); } PM::AxsLog PM::AccessLog
  • 141. 予告
  • 142. “PSGI/Plack・Monocerosで学ぶ ハイパフォーマンス Webアプリケーションサーバの作り方” “How to build a High Performance Web Application Server” YAPC::Asia 2013 Tokyo Day#1 11:00~ 藤原洋記念ホール
  • 143. “Inside amon2-livedoor-setup.pl with web application development 2013” Kazuhiro Yappo Osawa YAPC::Asia 2013 Tokyo Day#1 13:00~ 多目的教室3
  • 144. “本当にあったレガシーな話” Daisuke Maki @lestrrat YAPC::Asia 2013 Tokyo Day#2 14:10~ 藤原洋記念ホール
  • 145. See
  • 146.   you
  • 147.   tomorrow!! 明日また来てください。 本物のWebAppサーバの高速化を 見せてあげますよ
  • 148. now on sale. go to amazon.co.jp #fin