Apache::LogFormat::Compiler YAPC::Asia 2013 Tokyo LT-Thon
Upcoming SlideShare
Loading in...5
×
 

Apache::LogFormat::Compiler YAPC::Asia 2013 Tokyo LT-Thon

on

  • 6,259 views

Apache::LogFormat::Compiler YAPC::Asia 2013 Tokyo LT-Thon

Apache::LogFormat::Compiler YAPC::Asia 2013 Tokyo LT-Thon

Statistics

Views

Total Views
6,259
Slideshare-icon Views on SlideShare
1,764
Embed Views
4,495

Actions

Likes
3
Downloads
3
Comments
0

16 Embeds 4,495

http://blog.nomadscafe.jp 4154
http://nomadscafe.jp 196
http://cloud.feedly.com 53
https://twitter.com 47
http://newsblur.com 10
http://reader.aol.com 7
http://webcache.googleusercontent.com 7
http://blog-new.dev.livedoor.jp 4
http://www.newsblur.com 4
http://translate.googleusercontent.com 4
http://digg.com 2
http://summary 2
http://www.feedspot.com 2
https://duckduckgo.com 1
http://feedly.com 1
http://131.253.14.98 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Apache::LogFormat::Compiler YAPC::Asia 2013 Tokyo LT-Thon Apache::LogFormat::Compiler YAPC::Asia 2013 Tokyo LT-Thon Presentation Transcript

    • ALFCYAPC::Asia 2013 Tokyo 前夜祭 LT-thon Masahiro Nagano @kazeburo
    • Me • 長野雅広 Masahiro Nagano • @kazeburo • PAUSE: KAZEBURO • Operations Engineer, Site Reliability • LINE Corp. Development support on LINE Familly, livedoor
    • ALFC =
    • Apache::LogFormat::Compiler
    • Convert
    • ‘%h %l %u %t "%r" %>s %b’
    • to perl-code
    • my $code = <<EOF; sub { my ($env,$res,$length,$reqtime,$time) = @_; $time = time() if ! defined $time; my @lt = localtime($time);; my $t = sprintf '%02d/%s/%04d:%02d:%02d:%02d %s', $lt[3], $abbr[$lt[4]], $lt[5]+1900, $lt[2], $lt[1], $lt[0], $tz; q!! . ($env->{REMOTE_ADDR} || '-') . q! ! . '-' . q! ! . ($env->{REMOTE_USER} || '-') . q! ! . "[" . $t . "]" . q! "! . _safe($env->{REQUEST_METHOD}) . " " . _safe($env- >{REQUEST_URI}) . " " . $env->{SERVER_PROTOCOL} . q!" ! . $res->[0] . q! ! . (defined $length ? $length : '-') . q!! } EOF
    • and compile
    • my $log_line = eval $code; $log_line->($env,$res);
    • SYNOPSIS
    • my $log_handler = Apache::LogFormat::Compiler->new( '%h %l %u %t "%r" %>s %b' ); warn $log_handler->log_line($env,$res);
    • use Plack::Builder; use Time::HiRes qw/gettimeofday tv_interval/; use Apache::LogFormat::Compiler; my $log_handler = Apache::LogFormat::Compiler->new( '%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-agent}i" %D' ); my $compile_log_app = builder { enable sub { my $app = shift; sub { my $env = shift; my $t0 = [gettimeofday]; my $res = $app->(); my $reqtime = int(tv_interval($t0) * 1_000_000); $env->{psgi.error}->print($log_handler->log_line( $env,$res,6,$reqtime, $t0->[0])); } }; $app };
    • OR
    • Plack::Middleware::AxsLog yet another AccessLog Middleware
    • use Plack::Builder; builder { enable 'AxsLog', combined => 1, response_time => 1; $app };
    • OR
    • Plack::Middleware::AccessLog Plack (1.0023~)
    • Benchmark
    • 0 10000 20000 30000 40000 50000 60000 51262 8826 Plack-1.0022 Plack-1.0029 Req/Sec
    • A
    •   Story
    •   of
    •   ALFC とある Ops と Dev のおはなし∼
    • I'm
    •   tuning
    •   up
    •   this
    •   web
    •   application.
    •    NYTProf
    •   said
    •   that
    •   PM::AccessLog
    •   is
    •   slow
    •    Yes,
    •   I
    •   know.
    •   It's
    •   slow
    •   because..
    •    Ops Dev
    • Why PM::AccessLog was slow?
    • my %char_handler = ( h => sub { $env->{REMOTE_ADDR} || '-' }, t => sub { "[" . $strftime->('%d/%b/%Y:%H:%M:%S %z', localtime) . "]" }, r => sub { _safe($env->{REQUEST_METHOD}) . " " . _safe($env->{REQUEST_URI}) . " " . $env->{SERVER_PROTOCOL} }, ); my $char_handler = sub { my $char = shift; exists $char_handler{$char} ? $char_handler{$char}->($char) : "-"; }; $fmt =~ s! (?: %{(.+?)}([a-z]) | %(?:[<>])?([a-zA-Z%]) ) ! $1 ? $block_handler->($1, $2) : $char_handler->($3) !egx;
    • I
    •   can't
    •   live
    •   with
    •   PM::AccessLog!! I
    •   have
    •   idea
    •   for
    •   speedup Ops Dev
    • Solution: combine strings
    • if ( $self->{format} eq 'common' ) { $log_line = _string($env->{REMOTE_ADDR}) . " " . '- ' . _string($env->{REMOTE_USER}) . " " . q![!. $t . q!] ! . _safe(q!"! . $env->{REQUEST_METHOD} . " " . $env->{REQUEST_URI} . " " . $env->{SERVER_PROTOCOL} . q!" !) . $res->[0] . " " . (defined $length ? "$length" : '-'); } elsif ( $self->{format} eq 'ltsv' ) { ... }
    • This middleware was released as PM::AxsLog 2012-09-12
    • This isn’t extensible, only supports 3 formats. common,combined,ltsv but Fast.
    • 0 10000 20000 30000 40000 50000 60000 53315 8826 Plack-1.0022 AccessLog AxsLog-0.03 Req/Sec
    • 2013-03-01
    • Ah,
    •   PM::AxsLog
    •   can't
    •   modify
    •   log_format? I
    •   want
    •   to
    •   store
    •   a
    •   value
    •   of
    •   X-Moe
    •   to
    •   logfile What's?
    •   PM::AxsLog
    •   is
    •   fast
    •   because
    •    it
    •   supports
    •   only
    •   fixed
    •   format. Ooops,
    •   I
    •   should
    •   return
    •   to
    •   use
    •   PM::AccessLog... Wait!
    •   I
    •   have
    •   another
    •   idea
    •    Ops Dev
    • A few hours later, Ops released Apache::LogFormat::Compiler (2013-03-01)
    • AxsLog changed to use ALFC
    • 0 11000 22000 33000 44000 55000 5464053315 AxsLog-0.03 AxsLog-0.12 Req/Sec
    • 2013-03-19
    • @ops
    •   Can
    •   I
    •   replace
    •   PM::AccessLog
    •   by
    •   AxsLog? I
    •   recommend
    •   to
    •   use
    •   ALFC
    •   in
    •    PM::AccesssLog
    •   rather
    •   than
    •   replace. Because
    •   option
    •   of
    •   AxsLog
    •   isn’t
    •    comaptible
    •   with
    •   AccessLog Okay,
    •   I
    •   am
    •   looking
    •   forward
    •   to
    •   pull
    •   request. Ops Mr. committer
    • Plack-1.0023 was released with ALFC. And Ops removed text "Faster than AccessLog" from PM::AxsLog’s POD
    • They lived happily ever after. End
    • Appendix
    • Different of AxsLog and AccessLog
    • (1)
    • AxsLog supports Duration by default. AccessLog doesn’t support it. AccessLog::Timed required
    • use Plack::Builder; builder { enable 'AxsLog', format => ‘%h %l %u %t "%r" %>s %b %D’; $app };
    • use Plack::Builder; builder { enable 'AccessLog::Timed', format => ‘%h %l %u %t "%r" %>s %b %D’; $app };
    • (2)
    • Timing of timestamp
    • AxsLog retrieve time() at begining of a request. AccessLog retrieve time() at end of a request.
    • sub call { my ($self,$env) = @_; my $time = time(); #before app my $res = $self->app->($env); $self->log_line($env,$res,$length,$time); } sub call { my ($self,$env) = @_; my $res = $self->app->($env); my $time = time(); #after app $self->log_line($env,$res,$length,$time); } PM::AxsLog PM::AccessLog
    • 予告
    • “PSGI/Plack・Monocerosで学ぶ ハイパフォーマンス Webアプリケーションサーバの作り方” “How to build a High Performance Web Application Server” YAPC::Asia 2013 Tokyo Day#1 11:00~ 藤原洋記念ホール
    • “Inside amon2-livedoor-setup.pl with web application development 2013” Kazuhiro Yappo Osawa YAPC::Asia 2013 Tokyo Day#1 13:00~ 多目的教室3
    • “本当にあったレガシーな話” Daisuke Maki @lestrrat YAPC::Asia 2013 Tokyo Day#2 14:10~ 藤原洋記念ホール
    • See
    •   you
    •   tomorrow!! 明日また来てください。 本物のWebAppサーバの高速化を 見せてあげますよ
    • now on sale. go to amazon.co.jp #fin