1. Easy Logging
with Log::Message::Structured and Message::Passing

2. Me
Damien 'dams' Krotkine
Paris.pm member
French Perl Monger vice-president
Perl Dancer developer
11 modules on CPAN
(pretty weak)
Author of Moderne Perl
(french book, go buy it)

4. • Let’s say I have a big application

5. • Let’s say I have a big application
• It processes a lot of data

6. • Let’s say I have a big application
• It processes a lot of data
• It computes lots of lines

7. • Let’s say I have a big application
• It processes a lot of data
• It computes lots of lines
• Per user, and per account

8. account 1
user 1
lines
user 2
process
user 3
account n result
running on multiple hosts

10. • It runs fine

11. • It runs fine
• Well, more or less

12. • It runs fine
• Well, more or less
• I’d like to know what’s going on with it

13. When the app crashes

14. When the app crashes
• Why did it crash ? ( reason, stack trace )

15. When the app crashes
• Why did it crash ? ( reason, stack trace )
• At crash time

16. When the app crashes
• Why did it crash ? ( reason, stack trace )
• At crash time
• What was the situation ?

17. When the app crashes
• Why did it crash ? ( reason, stack trace )
• At crash time
• What was the situation ?
• What was the data it was processing ?

18. When there are issues

19. When there are issues
• How bad is this failure ?

20. When there are issues
• How bad is this failure ?
• Why did it fail to process this line ?

21. When there are issues
• How bad is this failure ?
• Why did it fail to process this line ?
• Is it only this line ?

22. When there are issues
• How bad is this failure ?
• Why did it fail to process this line ?
• Is it only this line ?
• Is it reproducible ?

23. When there are issues
• How bad is this failure ?
• Why did it fail to process this line ?
• Is it only this line ?
• Is it reproducible ?
• Does the rest fail ? (user, account)

24. When the app runs fine

25. When the app runs fine
• Information harvesting loop

26. When the app runs fine
• Information harvesting loop
• 1: get more info about the process run

27. When the app runs fine
• Information harvesting loop
• 1: get more info about the process run
• 2: go see my boss/colleague/...

28. When the app runs fine
• Information harvesting loop
• 1: get more info about the process run
• 2: go see my boss/colleague/...
• 3: (s)he asks something I don’t know

29. When the app runs fine
• Information harvesting loop
• 1: get more info about the process run
• 2: go see my boss/colleague/...
• 3: (s)he asks something I don’t know
• 4: go back to step 1

30. So yeah, I need some logs

31. So yeah, I need some logs
But we knew that, didn’t we ?

32. Logs

33. Logs

34. Logs

36. Exceptions

37. Exceptions

38. Exceptions
• An exception is an issue

39. Exceptions
• An exception is an issue
• All issues are not fatal (important)

40. Exceptions
• An exception is an issue
• All issues are not fatal (important)
• So, try / catch !

41. Exception Module

42. Exception Module
• Pick your own from CPAN

43. Exception Module
• Pick your own from CPAN
• I wrote mine anyway :)

44. Exception Module
• Pick your own from CPAN
• I wrote mine anyway :)
• Dancer::Exception

45. Exception Module
• Pick your own from CPAN
• I wrote mine anyway :)
• Dancer::Exception
• Extracted it as a standalone module

46. Exception Module
• Pick your own from CPAN
• I wrote mine anyway :)
• Dancer::Exception
• Extracted it as a standalone module
• Exceptions are objects with message
pattern, stack trace... raise(), throw(),
register(), introspection

47. Exception Module
• Pick your own from CPAN
• I wrote mine anyway :)
• Dancer::Exception
• Extracted it as a standalone module
• Exceptions are objects with message
pattern, stack trace... raise(), throw(),
register(), introspection
• Should end up on CPAN one day

48. foreach account {
try {
foreach user {
try {
foreach data line {
try {
process_line()
} catch {
log warning
};
}
} catch {
log important
};
}
} catch {
log important
};
}
log fatal

49. Logs

51. Logging a line

52. Logging a line
• say "warning, data is not good"

53. Logging a line
• say "warning, data is not good"
• say "warning, data '$data' is not good"

54. Logging a line
• say "warning, data is not good"
• say "warning, data '$data' is not good"
• say "$level, data '$data' is not good"

55. Logging a line
• say "warning, data is not good"
• say "warning, data '$data' is not good"
• say "$level, data '$data' is not good"
• log( in_file => "$level, '$data' is not good")

56. Logging a line
• say "warning, data is not good"
• say "warning, data '$data' is not good"
• say "$level, data '$data' is not good"
• log( in_file => "$level, '$data' is not good")
• my $logger = MyOwnLogger->new(write_in => 'file',
display_level => 'WARNING');

57. Logging a line
• say "warning, data is not good"
• say "warning, data '$data' is not good"
• say "$level, data '$data' is not good"
• log( in_file => "$level, '$data' is not good")
• my $logger = MyOwnLogger->new(write_in => 'file',
display_level => 'WARNING');
• $logger->log(warning =>"$Level, '$data' is not good");

58. Logging a line
• say "warning, data is not good"
• say "warning, data '$data' is not good"
• say "$level, data '$data' is not good"
• log( in_file => "$level, '$data' is not good")
• my $logger = MyOwnLogger->new(write_in => 'file',
display_level => 'WARNING');
• $logger->log(warning =>"$Level, '$data' is not good");
• Put that in a function, then a module, etc

59. Logging a line
• say "warning, data is not good"
• say "warning, data '$data' is not good"
• say "$level, data '$data' is not good"
• log( in_file => "$level, '$data' is not good")
• my $logger = MyOwnLogger->new(write_in => 'file',
display_level => 'WARNING');
• $logger->log(warning =>"$Level, '$data' is not good");
• Put that in a function, then a module, etc

60. The wheel...

61. The wheel...
• Home-made logging system : I've seen that
in every single company I've worked.

62. The wheel...
• Home-made logging system : I've seen that
in every single company I've worked.
• Sometimes multiple home-made logging
system in a single company

63. The wheel...
• Home-made logging system : I've seen that
in every single company I've worked.
• Sometimes multiple home-made logging
system in a single company
• Sometimes multiple home-made logging
system in a single project

64. The wheel...
• Home-made logging system : I've seen that
in every single company I've worked.
• Sometimes multiple home-made logging
system in a single company
• Sometimes multiple home-made logging
system in a single project
• It's so easy to start with a print STDERR...

65. The wheel...
• Home-made logging system : I've seen that
in every single company I've worked.
• Sometimes multiple home-made logging
system in a single company
• Sometimes multiple home-made logging
system in a single project
• It's so easy to start with a print STDERR...
• Stop reinventing the wheel

66. Pick up a logger

67. Pick up a logger
• Many Log modules on CPAN

68. Pick up a logger
• Many Log modules on CPAN
• Pick one you like

69. Pick up a logger
• Many Log modules on CPAN
• Pick one you like
• I choose Log::Dispatch

70. Pick up a logger
• Many Log modules on CPAN
• Pick one you like
• I choose Log::Dispatch
• $log->log(level => 'info', message => 'Blah' );

71. Pick up a logger
• Many Log modules on CPAN
• Pick one you like
• I choose Log::Dispatch
• $log->log(level => 'info', message => 'Blah' );
• Good ?

72. Pick up a logger
• Many Log modules on CPAN
• Pick one you like
• I choose Log::Dispatch
• $log->log(level => 'info', message => 'Blah' );
• Good ?
• No, we are still nowhere !

73. Pick up a logger
• Many Log modules on CPAN
• Pick one you like
• I choose Log::Dispatch
• $log->log(level => 'info', message => 'Blah' );
• Good ?
• No, we are still nowhere !
• We want contextual information

74. Contextual information

75. Contextual information
• What do we need to log ?

76. Contextual information
• What do we need to log ?
• A log message - ok, but

77. Contextual information
• What do we need to log ?
• A log message - ok, but
• Hardware info (Mem, CPU, disk, host)

78. Contextual information
• What do we need to log ?
• A log message - ok, but
• Hardware info (Mem, CPU, disk, host)
• Software info (time, user / account id...)

79. Contextual information
• What do we need to log ?
• A log message - ok, but
• Hardware info (Mem, CPU, disk, host)
• Software info (time, user / account id...)
• Message (filename, line nb, stacktrace...)

80. Contextual information
• What do we need to log ?
• A log message - ok, but
• Hardware info (Mem, CPU, disk, host)
• Software info (time, user / account id...)
• Message (filename, line nb, stacktrace...)
• Log line = mix contextual and specific info

81. Contextual information
• What do we need to log ?
• A log message - ok, but
• Hardware info (Mem, CPU, disk, host)
• Software info (time, user / account id...)
• Message (filename, line nb, stacktrace...)
• Log line = mix contextual and specific info
• Even with a great logger, need to build that

82. Richer logging

83. Richer logging
• How to pack these information together ?

84. Richer logging
• How to pack these information together ?
• Use a key-value structure

85. Richer logging
• How to pack these information together ?
• Use a key-value structure
• Better: a class, fields are attributes

86. Richer logging
• How to pack these information together ?
• Use a key-value structure
• Better: a class, fields are attributes
• So we can benefit of defaults, lazyness, etc

87. Richer logging
• How to pack these information together ?
• Use a key-value structure
• Better: a class, fields are attributes
• So we can benefit of defaults, lazyness, etc
• One log line = one class instance

88. Richer logging
• How to pack these information together ?
• Use a key-value structure
• Better: a class, fields are attributes
• So we can benefit of defaults, lazyness, etc
• One log line = one class instance
• We need it as a string at the end

89. Richer logging
• How to pack these information together ?
• Use a key-value structure
• Better: a class, fields are attributes
• So we can benefit of defaults, lazyness, etc
• One log line = one class instance
• We need it as a string at the end
• So stringify to JSON,YAML...

90. Log::Message::Structured

91. Log::Message::Structured
• A set of roles, consume in MyEvent class

92. Log::Message::Structured
• A set of roles, consume in MyEvent class
• 4 types of roles :

93. Log::Message::Structured
• A set of roles, consume in MyEvent class
• 4 types of roles :
• L::M::S : basic setup + "" overloading

94. Log::Message::Structured
• A set of roles, consume in MyEvent class
• 4 types of roles :
• L::M::S : basic setup + "" overloading
• L::M::S::Components : provides ready to
use additional fields

95. Log::Message::Structured
• A set of roles, consume in MyEvent class
• 4 types of roles :
• L::M::S : basic setup + "" overloading
• L::M::S::Components : provides ready to
use additional fields
• L::M::S::Stringify : provides stringifiers

96. Log::Message::Structured
• A set of roles, consume in MyEvent class
• 4 types of roles :
• L::M::S : basic setup + "" overloading
• L::M::S::Components : provides ready to
use additional fields
• L::M::S::Stringify : provides stringifiers
• Your custom fields : additional attributes

97. package MyLogEvent; use Moose;
with qw/
Log::Message::Structured
Log::Message::Structured::Component::Date
Log::Message::Structured::Component::Hostname
Log::Message::Structured::Stringify::AsJSON
/;
has message => ( is => 'ro', required => 1);
has account_id => ( is => 'ro', required => 1);
has user_id => ( is => 'ro' );

98. package MyLogEvent; use Moose;
with qw/
Log::Message::Structured
Log::Message::Structured::Component::Date
Log::Message::Structured::Component::Hostname
Log::Message::Structured::Stringify::AsJSON
/;
has message => ( is => 'ro', required => 1);
has account_id => ( is => 'ro', required => 1);
has user_id => ( is => 'ro' );
...elsewhere...

99. package MyLogEvent; use Moose;
with qw/
Log::Message::Structured
Log::Message::Structured::Component::Date
Log::Message::Structured::Component::Hostname
Log::Message::Structured::Stringify::AsJSON
/;
has message => ( is => 'ro', required => 1);
has account_id => ( is => 'ro', required => 1);
has user_id => ( is => 'ro' );
...elsewhere...
use My::Log::Event;
$logger->log( warning => MyLogEvent->new(
account_id => 42,
user_id => 12,
message => "watch out! behind you!" ));

100. package MyLogEvent; use Moose;
with qw/
Log::Message::Structured
Log::Message::Structured::Component::Date
Log::Message::Structured::Component::Hostname
Log::Message::Structured::Stringify::AsJSON
/;
has message => ( is => 'ro', required => 1);
has account_id => ( is => 'ro', required => 1);
has user_id => ( is => 'ro' );
...elsewhere...
use My::Log::Event;
$logger->log( warning => MyLogEvent->new(
account_id => 42,
user_id => 12,
message => "watch out! behind you!" ));
That's sending the stringified JSON to Log::Dispatch

101. Outputs as log:
'{"__CLASS__":"MyLogEvent",
"account_id":42,
"user_id":42,
"date":"2010-03-28T23:15:52Z",
"hostname":"mymachine.domain",
message=>"watch out! behind you!"}'

102. Outputs as log:
'{"__CLASS__":"MyLogEvent",
"account_id":42,
"user_id":42,
"date":"2010-03-28T23:15:52Z",
"hostname":"mymachine.domain",
message=>"watch out! behind you!"}'
L::M::S also supports options passed on the
command line

103. Outputs as log:
'{"__CLASS__":"MyLogEvent",
"account_id":42,
"user_id":42,
"date":"2010-03-28T23:15:52Z",
"hostname":"mymachine.domain",
message=>"watch out! behind you!"}'
L::M::S also supports options passed on the
command line
$ my_program.pl --account_id=42

104. Let's shorten the code
In my application I have:

105. Let's shorten the code
In my application I have:
while (my $account_id = AccountIterator->next) {
while (my $user_id = UserIterator->next) {
while (my $data_line = DataIterator->next) {
... do stuff ...

106. Let's shorten the code
In my application I have:
while (my $account_id = AccountIterator->next) {
while (my $user_id = UserIterator->next) {
while (my $data_line = DataIterator->next) {
... do stuff ...
package MyLogEvent; use Moose;
with qw/Log::Message::Structured .... /;
use AccountIterator; use UserIterator;
has account_id => ( is => 'ro',
default => sub { AccountIterator->current } );
has user_id => ( is => 'ro',
default => sub { UserIterator->current } );

107. Let's shorten the code

108. Let's shorten the code
Wrap Log::Dispatch call + L::M::S

109. Let's shorten the code
Wrap Log::Dispatch call + L::M::S
log_warning( message => "Achtung!");

110. Let's shorten the code
Wrap Log::Dispatch call + L::M::S
log_warning( message => "Achtung!");
actually calls

111. Let's shorten the code
Wrap Log::Dispatch call + L::M::S
log_warning( message => "Achtung!");
actually calls
$logger->log( warning => MyLogEvent->new( @_ );

112. Let's shorten the code
Wrap Log::Dispatch call + L::M::S
log_warning( message => "Achtung!");
actually calls
$logger->log( warning => MyLogEvent->new( @_ );
outputs

113. Let's shorten the code
Wrap Log::Dispatch call + L::M::S
log_warning( message => "Achtung!");
actually calls
$logger->log( warning => MyLogEvent->new( @_ );
outputs
'{"__CLASS__":"MyLogEvent",
"account_id":42,
"user_id":42,
"date":"2010-03-28T23:15:52Z",
"hostname":"mymachine.domain",
message=>"Achtung!"}'

114. Exceptions...
Remember ?

115. Exceptions...
Remember ?
try { ... } catch {
log_warning(exception => $_ )
};

116. Exceptions...
Remember ?
try { ... } catch {
log_warning(exception => $_ )
};
• To be able to write that, you need in your
MyLogEvent class:

117. Exceptions...
Remember ?
try { ... } catch {
log_warning(exception => $_ )
};
• To be able to write that, you need in your
MyLogEvent class:
• these attributes : 'exception', 'stack_trace',

118. Exceptions...
Remember ?
try { ... } catch {
log_warning(exception => $_ )
};
• To be able to write that, you need in your
MyLogEvent class:
• these attributes : 'exception', 'stack_trace',
• with a BUILDARGS

119. Exceptions...
Remember ?
try { ... } catch {
log_warning(exception => $_ )
};
• To be able to write that, you need in your
MyLogEvent class:
• these attributes : 'exception', 'stack_trace',
• with a BUILDARGS
• that looks at the exception, and set the
'message' and the 'stack_trace' arguments

120. around BUILDARGS => sub {
my $orig = shift;
my $class = shift;
my $h = $class->$orig(@_);
my $e = $h->{exception}
or return $h;
if ( blessed($e) && $e->isa('My::Exception::Base') ) {
$h->{message} = $e->message;
$h->{shortmess} = $e->{_shortmess};
} else {
$h->{message} = $e
}
return $h;
};

121. around BUILDARGS => sub {
my $orig = shift;
my $class = shift;
my $h = $class->$orig(@_);
my $e = $h->{exception}
or return $h;
if ( blessed($e) && $e->isa('My::Exception::Base') ) {
$h->{message} = $e->message;
$h->{shortmess} = $e->{_shortmess};
} else {
$h->{message} = $e
}
return $h;
};
try { ... } catch {
log_warning(exception => $_ )
That would now
}; work !

122. Log::Message::Structured

123. Log::Message::Structured
• Does the job

124. Log::Message::Structured
• Does the job
• Please contribute

125. Log::Message::Structured
• Does the job
• Please contribute
• we need more components

126. Log::Message::Structured
• Does the job
• Please contribute
• we need more components
• we need more stringifiers

127. Log::Message::Structured
• Does the job
• Please contribute
• we need more components
• we need more stringifiers
• compatibility with Moo

128. Log::Message::Structured
• Does the job
• Please contribute
• we need more components
• we need more stringifiers
• compatibility with Moo
• On CPAN / github

130. • We have all we need as JSON

131. • We have all we need as JSON
• But that's not enough :(

132. • We have all we need as JSON
• But that's not enough :(
• Your boss wants to look at the log

133. • We have all we need as JSON
• But that's not enough :(
• Your boss wants to look at the log
• => provide a simple interface

134. • We have all we need as JSON
• But that's not enough :(
• Your boss wants to look at the log
• => provide a simple interface
• You need to do complex lookups / research

135. • We have all we need as JSON
• But that's not enough :(
• Your boss wants to look at the log
• => provide a simple interface
• You need to do complex lookups / research
• thousands of files, grep not good enough

136. • We have all we need as JSON
• But that's not enough :(
• Your boss wants to look at the log
• => provide a simple interface
• You need to do complex lookups / research
• thousands of files, grep not good enough
• => need a powerful search engine

137. • We have all we need as JSON
• But that's not enough :(
• Your boss wants to look at the log
• => provide a simple interface
• You need to do complex lookups / research
• thousands of files, grep not good enough
• => need a powerful search engine
• Also, what if you have different log sources

138. • We have all we need as JSON
• But that's not enough :(
• Your boss wants to look at the log
• => provide a simple interface
• You need to do complex lookups / research
• thousands of files, grep not good enough
• => need a powerful search engine
• Also, what if you have different log sources
• From DB events, Redis MQ, system logs ?

139. Web interfaces

140. Web interfaces
• Two interfaces

141. Web interfaces
• Two interfaces
• First one is easy to use

142. Web interfaces
• Two interfaces
• First one is easy to use
• Based on Dancer + MongoDB

143. Web interfaces
• Two interfaces
• First one is easy to use
• Based on Dancer + MongoDB
• Shows logs in natural manner, grouped by
job, hostname, and presented by time

144. Web interfaces
• Two interfaces
• First one is easy to use
• Based on Dancer + MongoDB
• Shows logs in natural manner, grouped by
job, hostname, and presented by time
• Second one is more for data mining

145. Web interfaces
• Two interfaces
• First one is easy to use
• Based on Dancer + MongoDB
• Shows logs in natural manner, grouped by
job, hostname, and presented by time
• Second one is more for data mining
• Elasticsearch: deep searching, filtering, etc

146. We need some kind of big stuff
syslog
Standard log files
Redis MQ logs $stuff Web interface
Other logs Search Engine

147. We need some kind of big stuff
syslog
Standard log files
Redis MQ logs $stuff Web interface
Other logs Search Engine
$stuff = Message::Passing

148. Message::Passing

149. Message::Passing
• Written by Tomas Doran

150. Message::Passing
• Written by Tomas Doran
• Logstash-like but in Perl and with style

151. Message::Passing
• Written by Tomas Doran
• Logstash-like but in Perl and with style
• It's a daemon (AnyEvent based)

152. Message::Passing
• Written by Tomas Doran
• Logstash-like but in Perl and with style
• It's a daemon (AnyEvent based)
• basic concepts: inputs, filters, outputs

153. Message::Passing
• Written by Tomas Doran
• Logstash-like but in Perl and with style
• It's a daemon (AnyEvent based)
• basic concepts: inputs, filters, outputs
• run it as : command line, using the DSL,
using classes directly.

154. Real Life

155. Real Life
• My software runs on many machines

156. Real Life
• My software runs on many machines
• Connected to one syslog-ng server

157. Real Life
• My software runs on many machines
• Connected to one syslog-ng server
• Writing to log files

158. Real Life
• My software runs on many machines
• Connected to one syslog-ng server
• Writing to log files
• Configure syslog-ng to output to a pipe

159. Real Life
• My software runs on many machines
• Connected to one syslog-ng server
• Writing to log files
• Configure syslog-ng to output to a pipe
• Branch M::P on this named pipe

160. Real Life
• My software runs on many machines
• Connected to one syslog-ng server
• Writing to log files
• Configure syslog-ng to output to a pipe
• Branch M::P on this named pipe
• Web interface: Dancer + MongoDB

161. Real Life
• My software runs on many machines
• Connected to one syslog-ng server
• Writing to log files
• Configure syslog-ng to output to a pipe
• Branch M::P on this named pipe
• Web interface: Dancer + MongoDB
• Search interface : Elasticsearch + plugin

162. syslog-ng
many logs
machines
standard
log files

163. syslog-ng duplicated
flow
many logs
machines
standard
log files Message::Passing
ElasticSearch MongoDB
Web GUI Web GUI

164. logs MQ
Message::Passing
many
machines
In a better world
ElasticSearch MongoDB
Web GUI Web GUI

165. Let's use Message::Passing

166. Let's use Message::Passing
• step 1 : configure & run M::P daemon

167. Let's use Message::Passing
• step 1 : configure & run M::P daemon
• step 2 : ...

168. Let's use Message::Passing
• step 1 : configure & run M::P daemon
• step 2 : ...
• step 3 : PROFIT

169. Let's use Message::Passing
• step 1 : configure & run M::P daemon
• step 2 : ...
• step 3 : PROFIT
Remember the basic concepts?
inputs, filters, outputs

170. use Message::Passing::DSL; use Moose;
with 'Message::Passing::Role::Script';
sub build_chain { message_chain {
input filetail => (
class => 'FileTail',
filename => ‘/path/to/logs’,
output_to => 'cleanup_logs' );
filter cleanup_logs => (
class => '+My::Filter::DeJSON',
output_to => [ 'elasticsearch', 'mongodb' ] );
output elasticsearch_servers => (
class => 'ElasticSearch',
elasticsearch_servers => 'localhost:9200' );
output mongodb => (
class => 'MongoDB',
hostname => 'localhost',
database => 'reporting',
collection => 'logs',
indexes => [ # specify fields to index ] );}
}
__PACKAGE__->start;

171. My::Filter::DeJSON
package My::Filter::DeJSON;
use Moose;
sub consume {
my ( $self, $message ) = @_;
$message =~ m/(w+):JSON:(.*)/ or return;
my $structure_log = from_json($2);
$structure_log->{log_level} = $1;
foreach my $output_to ( @{ $self->output_to } ) {
$output_to->consume($structure_log);
}
}

172. Message::Passing::Output::MongoDB
use Moose; use MongoDB; use AnyEvent;
with qw/
Message::Passing::Role::Output
Message::Passing::Role::HasUsernameAndPassword
Message::Passing::Role::HasHostnameAndPort
/;
# ... hostname, port, user/pass attributes ...
has _db => (
is => 'ro', isa => 'MongoDB::Database', lazy => 1,
default => sub {
my $self = shift;
my $connection = MongoDB::Connection->new(
host => $self->hostname,
port => $self->port,
);
return $connection->get_database($self->database);
},
);

173. Message::Passing::Output::MongoDB
# ... _collection attribute
sub _build_logs_collection {
my ($self) = @_;
my $collection_name = $self->collection;
my $collection = $self->_db->$collection_name;
if ($self->_has_indexes) {
foreach my $index (@{$self->indexes}){
$collection->ensure_index(@$index);
}
}
return $collection;
}

174. Message::Passing::Output::MongoDB
sub consume {
my ($self, $data) = @_;
return unless $data;
my $date;
my $collection = $self->_collection;
$collection->insert($data)
or warn "Insertion failure: " . Dumper($data) . "n";
if ($self->verbose) {
$self->_inc_log_counter;
warn("Total " . $self->_log_counter
. " records inserted in MongoDBn");
}
}
On CPAN, by Bin Shu

175. Dancer web GUI

176. Dancer web GUI
• Very simple Dancer application

177. Dancer web GUI
• Very simple Dancer application
• Could have used Dancer::Plugin::Mongo

178. Dancer web GUI
• Very simple Dancer application
• Could have used Dancer::Plugin::Mongo
• A screen to display log

179. Dancer web GUI
• Very simple Dancer application
• Could have used Dancer::Plugin::Mongo
• A screen to display log
• A screen to search (filter) logs

180. Dancer web GUI
• Very simple Dancer application
• Could have used Dancer::Plugin::Mongo
• A screen to display log
• A screen to search (filter) logs
• Result page always show logs ordered by time

181. Dancer GUI

182. Dancer GUI

183. Dancer GUI routes

184. Dancer GUI routes
• get / :

185. Dancer GUI routes
• get / :
• a mongoDB group request per host, command, => /result

186. Dancer GUI routes
• get / :
• a mongoDB group request per host, command, => /result
• get /search :

187. Dancer GUI routes
• get / :
• a mongoDB group request per host, command, => /result
• get /search :
• a mapreduce to display asearch form, => /result

188. Dancer GUI routes
• get / :
• a mongoDB group request per host, command, => /result
• get /search :
• a mapreduce to display asearch form, => /result
• get /result :

189. Dancer GUI routes
• get / :
• a mongoDB group request per host, command, => /result
• get /search :
• a mapreduce to display asearch form, => /result
• get /result :
• get search params from url, issue a find request

190. Dancer GUI routes
• get / :
• a mongoDB group request per host, command, => /result
• get /search :
• a mapreduce to display asearch form, => /result
• get /result :
• get search params from url, issue a find request
• display in a table

191. Dancer GUI routes
• get / :
• a mongoDB group request per host, command, => /result
• get /search :
• a mapreduce to display asearch form, => /result
• get /result :
• get search params from url, issue a find request
• display in a table
• clicking on a result brings up a JS detailed result popup

192. Show me the video

193. Elasticsearch

194. Elasticsearch
• Installation : trivial

195. Elasticsearch
• Installation : trivial
• Configuration : none

196. Elasticsearch
• Installation : trivial
• Configuration : none
• Web GUI : elasticsearch-head

197. Elasticsearch
• Installation : trivial
• Configuration : none
• Web GUI : elasticsearch-head
• Code ? there is no code to show :)

198. Elasticsearch
• Installation : trivial
• Configuration : none
• Web GUI : elasticsearch-head
• Code ? there is no code to show :)
• Usage : perform advanced search on massive backlog

202. Show me the video

203. Additional stuff
• If there is an error in Message::Passing
• You should use the error channel
• To avoid loop of death

204. Conclusion
• That's all there is about it, really
• Message::Passing is easy and work great
• Log::Message::Structured : an awesome idea

205. Questions ?
Thanks !
Damien 'dams' Krotkine