Apache::LogFormat::Compiler YAPC::Asia 2013 Tokyo LT-Thon

1. ALFCYAPC::Asia 2013 Tokyo
前夜祭 LT-thon
Masahiro Nagano @kazeburo

2. Me
• 長野雅広 Masahiro Nagano
• @kazeburo
• PAUSE: KAZEBURO
• Operations Engineer, Site Reliability
• LINE Corp.
Development support
on LINE Familly, livedoor

3. ALFC =

4. Apache::LogFormat::Compiler

5. Convert

6. ‘%h %l %u %t "%r" %>s %b’

7. to perl-code

8. my $code = <<EOF;
sub {
my ($env,$res,$length,$reqtime,$time) = @_;
$time = time() if ! defined $time;
my @lt = localtime($time);;
my $t = sprintf '%02d/%s/%04d:%02d:%02d:%02d %s', $lt[3],
$abbr[$lt[4]], $lt[5]+1900, $lt[2], $lt[1], $lt[0], $tz;
q!! . ($env->{REMOTE_ADDR} || '-')
. q! ! . '-'
. q! ! . ($env->{REMOTE_USER} || '-')
. q! ! . "[" . $t . "]"
. q! "! . _safe($env->{REQUEST_METHOD}) . " " . _safe($env-
>{REQUEST_URI}) . " " . $env->{SERVER_PROTOCOL}
. q!" ! . $res->[0]
. q! ! . (defined $length ? $length : '-')
. q!!
}
EOF

9. and compile

10. my $log_line = eval $code;
$log_line->($env,$res);

11. SYNOPSIS

12. my $log_handler = Apache::LogFormat::Compiler->new(
'%h %l %u %t "%r" %>s %b'
);
warn $log_handler->log_line($env,$res);

13. use Plack::Builder;
use Time::HiRes qw/gettimeofday tv_interval/;
use Apache::LogFormat::Compiler;
my $log_handler = Apache::LogFormat::Compiler->new(
'%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-agent}i" %D'
);
my $compile_log_app = builder {
enable sub {
my $app = shift;
sub {
my $env = shift;
my $t0 = [gettimeofday];
my $res = $app->();
my $reqtime = int(tv_interval($t0) * 1_000_000);
$env->{psgi.error}->print($log_handler->log_line(
$env,$res,6,$reqtime, $t0->[0]));
}
};
$app
};

14. OR

15. Plack::Middleware::AxsLog
yet another AccessLog Middleware

16. use Plack::Builder;
builder {
enable 'AxsLog',
combined => 1,
response_time => 1;
$app
};

17. OR

18. Plack::Middleware::AccessLog
Plack (1.0023~)

19. Benchmark

20. 0
10000
20000
30000
40000
50000
60000
51262
8826
Plack-1.0022 Plack-1.0029
Req/Sec

21. A

22. Story

23. of

24. ALFC
とある Ops と Dev のおはなし∼

25. I'm

26. tuning

27. up

28. this

29. web

30. application.

31.
NYTProf

32. said

33. that

34. PM::AccessLog

35. is

36. slow

37.
Yes,

38. I

39. know.

40. It's

41. slow

42. because..

43.
Ops Dev

44. Why PM::AccessLog
was slow?

45. my %char_handler = (
h = sub { $env-{REMOTE_ADDR} || '-' },
t = sub { [ . $strftime-('%d/%b/%Y:%H:%M:%S %z', localtime) . ] },
r = sub { _safe($env-{REQUEST_METHOD}) .
. _safe($env-{REQUEST_URI}) .
. $env-{SERVER_PROTOCOL} },
);
my $char_handler = sub {
my $char = shift;
exists $char_handler{$char}
? $char_handler{$char}-($char) : -;
};
$fmt =~ s!
(?:
%{(.+?)}([a-z]) |
%(?:[])?([a-zA-Z%])
)
! $1 ? $block_handler-($1, $2) : $char_handler-($3) !egx;

46. I

47. can't

48. live

49. with

50. PM::AccessLog!!
I

51. have

52. idea

53. for

54. speedup
Ops Dev

55. Solution: combine strings

56. if ( $self-{format} eq 'common' ) {
$log_line = _string($env-{REMOTE_ADDR}) .
. '- '
. _string($env-{REMOTE_USER}) .
. q![!. $t . q!] !
. _safe(q!! . $env-{REQUEST_METHOD} .
. $env-{REQUEST_URI} .
. $env-{SERVER_PROTOCOL} . q! !)
. $res-[0] .
. (defined $length ? $length : '-');
} elsif ( $self-{format} eq 'ltsv' ) {
...
}

57. This middleware was
released as PM::AxsLog
2012-09-12

58. This isn’t extensible,
only supports
3 formats.
common,combined,ltsv
but Fast.

59. 0
10000
20000
30000
40000
50000
60000
53315
8826
Plack-1.0022 AccessLog AxsLog-0.03
Req/Sec

60. 2013-03-01

61. Ah,

62. PM::AxsLog

63. can't

64. modify

65. log_format?
I

66. want

67. to

68. store

69. a

70. value

71. of

72. X-Moe

73. to

74. logfile
What's?

75. PM::AxsLog

76. is

77. fast

78. because

79.
it

80. supports

81. only

82. fixed

83. format.
Ooops,

84. I

85. should

86. return

87. to

88. use

89. PM::AccessLog...
Wait!

90. I

91. have

92. another

93. idea

94.
Ops Dev

95. A few hours later,
Ops released
Apache::LogFormat::Compiler
(2013-03-01)

96. AxsLog changed to
use ALFC

97. 0
11000
22000
33000
44000
55000
5464053315
AxsLog-0.03 AxsLog-0.12
Req/Sec

98. 2013-03-19

99. @ops

100. Can

101. I

102. replace

103. PM::AccessLog

104. by

105. AxsLog?
I

106. recommend

107. to

108. use

109. ALFC

110. in

111.
PM::AccesssLog

112. rather

113. than

114. replace.
Because

115. option

116. of

117. AxsLog

118. isn’t

119.
comaptible

120. with

121. AccessLog
Okay,

122. I

123. am

124. looking

125. forward

126. to

127. pull

128. request.
Ops Mr. committer

129. Plack-1.0023 was
released with ALFC.
And Ops removed text
Faster than AccessLog
from PM::AxsLog’s POD

130. They lived happily ever after.
End

131. Appendix

132. Different of AxsLog
and AccessLog

133. (1)

134. AxsLog supports Duration by default.
AccessLog doesn’t support it.
AccessLog::Timed required

135. use Plack::Builder;
builder {
enable 'AxsLog',
format = ‘%h %l %u %t %r %s %b %D’;
$app
};

136. use Plack::Builder;
builder {
enable 'AccessLog::Timed',
format = ‘%h %l %u %t %r %s %b %D’;
$app
};

137. (2)

138. Timing of timestamp

139. AxsLog retrieve time()
at begining of a request.
AccessLog retrieve time()
at end of a request.

140. sub call {
my ($self,$env) = @_;
my $time = time(); #before app
my $res = $self-app-($env);
$self-log_line($env,$res,$length,$time);
}
sub call {
my ($self,$env) = @_;
my $res = $self-app-($env);
my $time = time(); #after app
$self-log_line($env,$res,$length,$time);
}
PM::AxsLog
PM::AccessLog

141. 予告

142. “PSGI/Plack・Monocerosで学ぶ
ハイパフォーマンス
Webアプリケーションサーバの作り方”
“How to build a High Performance
Web Application Server”
YAPC::Asia 2013 Tokyo
Day#1 11:00~
藤原洋記念ホール

143. “Inside amon2-livedoor-setup.pl with
web application development 2013”
Kazuhiro Yappo Osawa
YAPC::Asia 2013 Tokyo
Day#1 13:00~
多目的教室3

144. “本当にあったレガシーな話”
Daisuke Maki @lestrrat
YAPC::Asia 2013 Tokyo
Day#2 14:10~
藤原洋記念ホール

145. See

146. you

147. tomorrow!!
明日また来てください。
本物のWebAppサーバの高速化を
見せてあげますよ

148. now on sale. go to amazon.co.jp
#fin