Apache::LogFormat::Compiler YAPC::Asia 2013 Tokyo LT-Thon
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Apache::LogFormat::Compiler YAPC::Asia 2013 Tokyo LT-Thon

  • 6,542 views
Uploaded on

Apache::LogFormat::Compiler YAPC::Asia 2013 Tokyo LT-Thon

Apache::LogFormat::Compiler YAPC::Asia 2013 Tokyo LT-Thon

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
6,542
On Slideshare
1,959
From Embeds
4,583
Number of Embeds
16

Actions

Shares
Downloads
3
Comments
0
Likes
3

Embeds 4,583

http://blog.nomadscafe.jp 4,205
http://nomadscafe.jp 232
http://cloud.feedly.com 53
https://twitter.com 48
http://newsblur.com 10
http://reader.aol.com 7
http://webcache.googleusercontent.com 7
http://blog-new.dev.livedoor.jp 4
http://www.newsblur.com 4
http://translate.googleusercontent.com 4
http://digg.com 2
http://summary 2
http://www.feedspot.com 2
https://duckduckgo.com 1
http://feedly.com 1
http://131.253.14.98 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. ALFCYAPC::Asia 2013 Tokyo 前夜祭 LT-thon Masahiro Nagano @kazeburo
  • 2. Me • 長野雅広 Masahiro Nagano • @kazeburo • PAUSE: KAZEBURO • Operations Engineer, Site Reliability • LINE Corp. Development support on LINE Familly, livedoor
  • 3. ALFC =
  • 4. Apache::LogFormat::Compiler
  • 5. Convert
  • 6. ‘%h %l %u %t "%r" %>s %b’
  • 7. to perl-code
  • 8. my $code = <<EOF; sub { my ($env,$res,$length,$reqtime,$time) = @_; $time = time() if ! defined $time; my @lt = localtime($time);; my $t = sprintf '%02d/%s/%04d:%02d:%02d:%02d %s', $lt[3], $abbr[$lt[4]], $lt[5]+1900, $lt[2], $lt[1], $lt[0], $tz; q!! . ($env->{REMOTE_ADDR} || '-') . q! ! . '-' . q! ! . ($env->{REMOTE_USER} || '-') . q! ! . "[" . $t . "]" . q! "! . _safe($env->{REQUEST_METHOD}) . " " . _safe($env- >{REQUEST_URI}) . " " . $env->{SERVER_PROTOCOL} . q!" ! . $res->[0] . q! ! . (defined $length ? $length : '-') . q!! } EOF
  • 9. and compile
  • 10. my $log_line = eval $code; $log_line->($env,$res);
  • 11. SYNOPSIS
  • 12. my $log_handler = Apache::LogFormat::Compiler->new( '%h %l %u %t "%r" %>s %b' ); warn $log_handler->log_line($env,$res);
  • 13. use Plack::Builder; use Time::HiRes qw/gettimeofday tv_interval/; use Apache::LogFormat::Compiler; my $log_handler = Apache::LogFormat::Compiler->new( '%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-agent}i" %D' ); my $compile_log_app = builder { enable sub { my $app = shift; sub { my $env = shift; my $t0 = [gettimeofday]; my $res = $app->(); my $reqtime = int(tv_interval($t0) * 1_000_000); $env->{psgi.error}->print($log_handler->log_line( $env,$res,6,$reqtime, $t0->[0])); } }; $app };
  • 14. OR
  • 15. Plack::Middleware::AxsLog yet another AccessLog Middleware
  • 16. use Plack::Builder; builder { enable 'AxsLog', combined => 1, response_time => 1; $app };
  • 17. OR
  • 18. Plack::Middleware::AccessLog Plack (1.0023~)
  • 19. Benchmark
  • 20. 0 10000 20000 30000 40000 50000 60000 51262 8826 Plack-1.0022 Plack-1.0029 Req/Sec
  • 21. A
  • 22.   Story
  • 23.   of
  • 24.   ALFC とある Ops と Dev のおはなし∼
  • 25. I'm
  • 26.   tuning
  • 27.   up
  • 28.   this
  • 29.   web
  • 30.   application.
  • 31.    NYTProf
  • 32.   said
  • 33.   that
  • 34.   PM::AccessLog
  • 35.   is
  • 36.   slow
  • 37.    Yes,
  • 38.   I
  • 39.   know.
  • 40.   It's
  • 41.   slow
  • 42.   because..
  • 43.    Ops Dev
  • 44. Why PM::AccessLog was slow?
  • 45. my %char_handler = ( h => sub { $env->{REMOTE_ADDR} || '-' }, t => sub { "[" . $strftime->('%d/%b/%Y:%H:%M:%S %z', localtime) . "]" }, r => sub { _safe($env->{REQUEST_METHOD}) . " " . _safe($env->{REQUEST_URI}) . " " . $env->{SERVER_PROTOCOL} }, ); my $char_handler = sub { my $char = shift; exists $char_handler{$char} ? $char_handler{$char}->($char) : "-"; }; $fmt =~ s! (?: %{(.+?)}([a-z]) | %(?:[<>])?([a-zA-Z%]) ) ! $1 ? $block_handler->($1, $2) : $char_handler->($3) !egx;
  • 46. I
  • 47.   can't
  • 48.   live
  • 49.   with
  • 50.   PM::AccessLog!! I
  • 51.   have
  • 52.   idea
  • 53.   for
  • 54.   speedup Ops Dev
  • 55. Solution: combine strings
  • 56. if ( $self->{format} eq 'common' ) { $log_line = _string($env->{REMOTE_ADDR}) . " " . '- ' . _string($env->{REMOTE_USER}) . " " . q![!. $t . q!] ! . _safe(q!"! . $env->{REQUEST_METHOD} . " " . $env->{REQUEST_URI} . " " . $env->{SERVER_PROTOCOL} . q!" !) . $res->[0] . " " . (defined $length ? "$length" : '-'); } elsif ( $self->{format} eq 'ltsv' ) { ... }
  • 57. This middleware was released as PM::AxsLog 2012-09-12
  • 58. This isn’t extensible, only supports 3 formats. common,combined,ltsv but Fast.
  • 59. 0 10000 20000 30000 40000 50000 60000 53315 8826 Plack-1.0022 AccessLog AxsLog-0.03 Req/Sec
  • 60. 2013-03-01
  • 61. Ah,
  • 62.   PM::AxsLog
  • 63.   can't
  • 64.   modify
  • 65.   log_format? I
  • 66.   want
  • 67.   to
  • 68.   store
  • 69.   a
  • 70.   value
  • 71.   of
  • 72.   X-Moe
  • 73.   to
  • 74.   logfile What's?
  • 75.   PM::AxsLog
  • 76.   is
  • 77.   fast
  • 78.   because
  • 79.    it
  • 80.   supports
  • 81.   only
  • 82.   fixed
  • 83.   format. Ooops,
  • 84.   I
  • 85.   should
  • 86.   return
  • 87.   to
  • 88.   use
  • 89.   PM::AccessLog... Wait!
  • 90.   I
  • 91.   have
  • 92.   another
  • 93.   idea
  • 94.    Ops Dev
  • 95. A few hours later, Ops released Apache::LogFormat::Compiler (2013-03-01)
  • 96. AxsLog changed to use ALFC
  • 97. 0 11000 22000 33000 44000 55000 5464053315 AxsLog-0.03 AxsLog-0.12 Req/Sec
  • 98. 2013-03-19
  • 99. @ops
  • 100.   Can
  • 101.   I
  • 102.   replace
  • 103.   PM::AccessLog
  • 104.   by
  • 105.   AxsLog? I
  • 106.   recommend
  • 107.   to
  • 108.   use
  • 109.   ALFC
  • 110.   in
  • 111.    PM::AccesssLog
  • 112.   rather
  • 113.   than
  • 114.   replace. Because
  • 115.   option
  • 116.   of
  • 117.   AxsLog
  • 118.   isn’t
  • 119.    comaptible
  • 120.   with
  • 121.   AccessLog Okay,
  • 122.   I
  • 123.   am
  • 124.   looking
  • 125.   forward
  • 126.   to
  • 127.   pull
  • 128.   request. Ops Mr. committer
  • 129. Plack-1.0023 was released with ALFC. And Ops removed text "Faster than AccessLog" from PM::AxsLog’s POD
  • 130. They lived happily ever after. End
  • 131. Appendix
  • 132. Different of AxsLog and AccessLog
  • 133. (1)
  • 134. AxsLog supports Duration by default. AccessLog doesn’t support it. AccessLog::Timed required
  • 135. use Plack::Builder; builder { enable 'AxsLog', format => ‘%h %l %u %t "%r" %>s %b %D’; $app };
  • 136. use Plack::Builder; builder { enable 'AccessLog::Timed', format => ‘%h %l %u %t "%r" %>s %b %D’; $app };
  • 137. (2)
  • 138. Timing of timestamp
  • 139. AxsLog retrieve time() at begining of a request. AccessLog retrieve time() at end of a request.
  • 140. sub call { my ($self,$env) = @_; my $time = time(); #before app my $res = $self->app->($env); $self->log_line($env,$res,$length,$time); } sub call { my ($self,$env) = @_; my $res = $self->app->($env); my $time = time(); #after app $self->log_line($env,$res,$length,$time); } PM::AxsLog PM::AccessLog
  • 141. 予告
  • 142. “PSGI/Plack・Monocerosで学ぶ ハイパフォーマンス Webアプリケーションサーバの作り方” “How to build a High Performance Web Application Server” YAPC::Asia 2013 Tokyo Day#1 11:00~ 藤原洋記念ホール
  • 143. “Inside amon2-livedoor-setup.pl with web application development 2013” Kazuhiro Yappo Osawa YAPC::Asia 2013 Tokyo Day#1 13:00~ 多目的教室3
  • 144. “本当にあったレガシーな話” Daisuke Maki @lestrrat YAPC::Asia 2013 Tokyo Day#2 14:10~ 藤原洋記念ホール
  • 145. See
  • 146.   you
  • 147.   tomorrow!! 明日また来てください。 本物のWebAppサーバの高速化を 見せてあげますよ
  • 148. now on sale. go to amazon.co.jp #fin