This document discusses logging in Perl applications. It recommends using structured logging by creating log event objects that contain contextual information like date, hostname, etc. as well as custom fields. It introduces Log::Message::Structured, which provides roles to easily create log event classes that can be stringified to JSON or other formats and passed to logging modules like Log::Dispatch. This structured logging approach packages log data as objects for rich formatting and transmission while avoiding reimplementing basic logging functionality.

1. Easy Logging
with Log::Message::Structured and Message::Passing

2. Me
Damien 'dams' Krotkine
Paris.pm member
French Perl Monger vice-president
Perl Dancer developer
11 modules on CPAN
(pretty weak)
Author of Moderne Perl
(french book, go buy it)

4. • Let’s say I have a big application

5. • Let’s say I have a big application
• It processes a lot of data

6. • Let’s say I have a big application
• It processes a lot of data
• It computes lots of lines

7. • Let’s say I have a big application
• It processes a lot of data
• It computes lots of lines
• Per user, and per account

8. account 1
user 1
lines
user 2
process
user 3
account n result
running on multiple hosts

10. • It runs fine

11. • It runs fine
• Well, more or less

12. • It runs fine
• Well, more or less
• I’d like to know what’s going on with it

13. When the app crashes

14. When the app crashes
• Why did it crash ? ( reason, stack trace )

15. When the app crashes
• Why did it crash ? ( reason, stack trace )
• At crash time

16. When the app crashes
• Why did it crash ? ( reason, stack trace )
• At crash time
• What was the situation ?

17. When the app crashes
• Why did it crash ? ( reason, stack trace )
• At crash time
• What was the situation ?
• What was the data it was processing ?

18. When there are issues

19. When there are issues
• How bad is this failure ?

20. When there are issues
• How bad is this failure ?
• Why did it fail to process this line ?

21. When there are issues
• How bad is this failure ?
• Why did it fail to process this line ?
• Is it only this line ?

22. When there are issues
• How bad is this failure ?
• Why did it fail to process this line ?
• Is it only this line ?
• Is it reproducible ?

23. When there are issues
• How bad is this failure ?
• Why did it fail to process this line ?
• Is it only this line ?
• Is it reproducible ?
• Does the rest fail ? (user, account)

24. When the app runs fine

25. When the app runs fine
• Information harvesting loop

26. When the app runs fine
• Information harvesting loop
• 1: get more info about the process run

27. When the app runs fine
• Information harvesting loop
• 1: get more info about the process run
• 2: go see my boss/colleague/...

28. When the app runs fine
• Information harvesting loop
• 1: get more info about the process run
• 2: go see my boss/colleague/...
• 3: (s)he asks something I don’t know

29. When the app runs fine
• Information harvesting loop
• 1: get more info about the process run
• 2: go see my boss/colleague/...
• 3: (s)he asks something I don’t know
• 4: go back to step 1

30. So yeah, I need some logs

31. So yeah, I need some logs
But we knew that, didn’t we ?

32. Logs

33. Logs

34. Logs

36. Exceptions

37. Exceptions

38. Exceptions
• An exception is an issue

39. Exceptions
• An exception is an issue
• All issues are not fatal (important)

40. Exceptions
• An exception is an issue
• All issues are not fatal (important)
• So, try / catch !

41. Exception Module

42. Exception Module
• Pick your own from CPAN

43. Exception Module
• Pick your own from CPAN
• I wrote mine anyway :)

44. Exception Module
• Pick your own from CPAN
• I wrote mine anyway :)
• Dancer::Exception

45. Exception Module
• Pick your own from CPAN
• I wrote mine anyway :)
• Dancer::Exception
• Extracted it as a standalone module

46. Exception Module
• Pick your own from CPAN
• I wrote mine anyway :)
• Dancer::Exception
• Extracted it as a standalone module
• Exceptions are objects with message
pattern, stack trace... raise(), throw(),
register(), introspection

47. Exception Module
• Pick your own from CPAN
• I wrote mine anyway :)
• Dancer::Exception
• Extracted it as a standalone module
• Exceptions are objects with message
pattern, stack trace... raise(), throw(),
register(), introspection
• Should end up on CPAN one day

48. foreach account {
try {
foreach user {
try {
foreach data line {
try {
process_line()
} catch {
log warning
};
}
} catch {
log important
};
}
} catch {
log important
};
}
log fatal

49. Logs

51. Logging a line

52. Logging a line
• say "warning, data is not good"

53. Logging a line
• say "warning, data is not good"
• say "warning, data '$data' is not good"

54. Logging a line
• say "warning, data is not good"
• say "warning, data '$data' is not good"
• say "$level, data '$data' is not good"

55. Logging a line
• say "warning, data is not good"
• say "warning, data '$data' is not good"
• say "$level, data '$data' is not good"
• log( in_file => "$level, '$data' is not good")

56. Logging a line
• say "warning, data is not good"
• say "warning, data '$data' is not good"
• say "$level, data '$data' is not good"
• log( in_file => "$level, '$data' is not good")
• my $logger = MyOwnLogger->new(write_in => 'file',
display_level => 'WARNING');

57. Logging a line
• say "warning, data is not good"
• say "warning, data '$data' is not good"
• say "$level, data '$data' is not good"
• log( in_file => "$level, '$data' is not good")
• my $logger = MyOwnLogger->new(write_in => 'file',
display_level => 'WARNING');
• $logger->log(warning =>"$Level, '$data' is not good");

58. Logging a line
• say "warning, data is not good"
• say "warning, data '$data' is not good"
• say "$level, data '$data' is not good"
• log( in_file => "$level, '$data' is not good")
• my $logger = MyOwnLogger->new(write_in => 'file',
display_level => 'WARNING');
• $logger->log(warning =>"$Level, '$data' is not good");
• Put that in a function, then a module, etc

59. Logging a line
• say "warning, data is not good"
• say "warning, data '$data' is not good"
• say "$level, data '$data' is not good"
• log( in_file => "$level, '$data' is not good")
• my $logger = MyOwnLogger->new(write_in => 'file',
display_level => 'WARNING');
• $logger->log(warning =>"$Level, '$data' is not good");
• Put that in a function, then a module, etc

60. The wheel...

61. The wheel...
• Home-made logging system : I've seen that
in every single company I've worked.

62. The wheel...
• Home-made logging system : I've seen that
in every single company I've worked.
• Sometimes multiple home-made logging
system in a single company

63. The wheel...
• Home-made logging system : I've seen that
in every single company I've worked.
• Sometimes multiple home-made logging
system in a single company
• Sometimes multiple home-made logging
system in a single project

64. The wheel...
• Home-made logging system : I've seen that
in every single company I've worked.
• Sometimes multiple home-made logging
system in a single company
• Sometimes multiple home-made logging
system in a single project
• It's so easy to start with a print STDERR...

65. The wheel...
• Home-made logging system : I've seen that
in every single company I've worked.
• Sometimes multiple home-made logging
system in a single company
• Sometimes multiple home-made logging
system in a single project
• It's so easy to start with a print STDERR...
• Stop reinventing the wheel

66. Pick up a logger

67. Pick up a logger
• Many Log modules on CPAN

68. Pick up a logger
• Many Log modules on CPAN
• Pick one you like

69. Pick up a logger
• Many Log modules on CPAN
• Pick one you like
• I choose Log::Dispatch

70. Pick up a logger
• Many Log modules on CPAN
• Pick one you like
• I choose Log::Dispatch
• $log->log(level => 'info', message => 'Blah' );

71. Pick up a logger
• Many Log modules on CPAN
• Pick one you like
• I choose Log::Dispatch
• $log->log(level => 'info', message => 'Blah' );
• Good ?

72. Pick up a logger
• Many Log modules on CPAN
• Pick one you like
• I choose Log::Dispatch
• $log->log(level => 'info', message => 'Blah' );
• Good ?
• No, we are still nowhere !

73. Pick up a logger
• Many Log modules on CPAN
• Pick one you like
• I choose Log::Dispatch
• $log->log(level => 'info', message => 'Blah' );
• Good ?
• No, we are still nowhere !
• We want contextual information

74. Contextual information

75. Contextual information
• What do we need to log ?

76. Contextual information
• What do we need to log ?
• A log message - ok, but

77. Contextual information
• What do we need to log ?
• A log message - ok, but
• Hardware info (Mem, CPU, disk, host)

78. Contextual information
• What do we need to log ?
• A log message - ok, but
• Hardware info (Mem, CPU, disk, host)
• Software info (time, user / account id...)

79. Contextual information
• What do we need to log ?
• A log message - ok, but
• Hardware info (Mem, CPU, disk, host)
• Software info (time, user / account id...)
• Message (filename, line nb, stacktrace...)

80. Contextual information
• What do we need to log ?
• A log message - ok, but
• Hardware info (Mem, CPU, disk, host)
• Software info (time, user / account id...)
• Message (filename, line nb, stacktrace...)
• Log line = mix contextual and specific info

81. Contextual information
• What do we need to log ?
• A log message - ok, but
• Hardware info (Mem, CPU, disk, host)
• Software info (time, user / account id...)
• Message (filename, line nb, stacktrace...)
• Log line = mix contextual and specific info
• Even with a great logger, need to build that

82. Richer logging

83. Richer logging
• How to pack these information together ?

84. Richer logging
• How to pack these information together ?
• Use a key-value structure

85. Richer logging
• How to pack these information together ?
• Use a key-value structure
• Better: a class, fields are attributes

86. Richer logging
• How to pack these information together ?
• Use a key-value structure
• Better: a class, fields are attributes
• So we can benefit of defaults, lazyness, etc

87. Richer logging
• How to pack these information together ?
• Use a key-value structure
• Better: a class, fields are attributes
• So we can benefit of defaults, lazyness, etc
• One log line = one class instance

88. Richer logging
• How to pack these information together ?
• Use a key-value structure
• Better: a class, fields are attributes
• So we can benefit of defaults, lazyness, etc
• One log line = one class instance
• We need it as a string at the end

89. Richer logging
• How to pack these information together ?
• Use a key-value structure
• Better: a class, fields are attributes
• So we can benefit of defaults, lazyness, etc
• One log line = one class instance
• We need it as a string at the end
• So stringify to JSON,YAML...

90. Log::Message::Structured

91. Log::Message::Structured
• A set of roles, consume in MyEvent class

92. Log::Message::Structured
• A set of roles, consume in MyEvent class
• 4 types of roles :

93. Log::Message::Structured
• A set of roles, consume in MyEvent class
• 4 types of roles :
• L::M::S : basic setup + "" overloading

94. Log::Message::Structured
• A set of roles, consume in MyEvent class
• 4 types of roles :
• L::M::S : basic setup + "" overloading
• L::M::S::Components : provides ready to
use additional fields

95. Log::Message::Structured
• A set of roles, consume in MyEvent class
• 4 types of roles :
• L::M::S : basic setup + "" overloading
• L::M::S::Components : provides ready to
use additional fields
• L::M::S::Stringify : provides stringifiers

96. Log::Message::Structured
• A set of roles, consume in MyEvent class
• 4 types of roles :
• L::M::S : basic setup + "" overloading
• L::M::S::Components : provides ready to
use additional fields
• L::M::S::Stringify : provides stringifiers
• Your custom fields : additional attributes

97. package MyLogEvent; use Moose;
with qw/
Log::Message::Structured
Log::Message::Structured::Component::Date
Log::Message::Structured::Component::Hostname
Log::Message::Structured::Stringify::AsJSON
/;
has message => ( is => 'ro', required => 1);
has account_id => ( is => 'ro', required => 1);
has user_id => ( is => 'ro' );

98. package MyLogEvent; use Moose;
with qw/
Log::Message::Structured
Log::Message::Structured::Component::Date
Log::Message::Structured::Component::Hostname
Log::Message::Structured::Stringify::AsJSON
/;
has message => ( is => 'ro', required => 1);
has account_id => ( is => 'ro', required => 1);
has user_id => ( is => 'ro' );
...elsewhere...

99. package MyLogEvent; use Moose;
with qw/
Log::Message::Structured
Log::Message::Structured::Component::Date
Log::Message::Structured::Component::Hostname
Log::Message::Structured::Stringify::AsJSON
/;
has message => ( is => 'ro', required => 1);
has account_id => ( is => 'ro', required => 1);
has user_id => ( is => 'ro' );
...elsewhere...
use My::Log::Event;
$logger->log( warning => MyLogEvent->new(
account_id => 42,
user_id => 12,
message => "watch out! behind you!" ));

100. package MyLogEvent; use Moose;
with qw/
Log::Message::Structured
Log::Message::Structured::Component::Date
Log::Message::Structured::Component::Hostname
Log::Message::Structured::Stringify::AsJSON
/;
has message => ( is => 'ro', required => 1);
has account_id => ( is => 'ro', required => 1);
has user_id => ( is => 'ro' );
...elsewhere...
use My::Log::Event;
$logger->log( warning => MyLogEvent->new(
account_id => 42,
user_id => 12,
message => "watch out! behind you!" ));
That's sending the stringified JSON to Log::Dispatch

101. Outputs as log:
'{"__CLASS__":"MyLogEvent",
"account_id":42,
"user_id":42,
"date":"2010-03-28T23:15:52Z",
"hostname":"mymachine.domain",
message=>"watch out! behind you!"}'

102. Outputs as log:
'{"__CLASS__":"MyLogEvent",
"account_id":42,
"user_id":42,
"date":"2010-03-28T23:15:52Z",
"hostname":"mymachine.domain",
message=>"watch out! behind you!"}'
L::M::S also supports options passed on the
command line

103. Outputs as log:
'{"__CLASS__":"MyLogEvent",
"account_id":42,
"user_id":42,
"date":"2010-03-28T23:15:52Z",
"hostname":"mymachine.domain",
message=>"watch out! behind you!"}'
L::M::S also supports options passed on the
command line
$ my_program.pl --account_id=42

104. Let's shorten the code
In my application I have:

105. Let's shorten the code
In my application I have:
while (my $account_id = AccountIterator->next) {
while (my $user_id = UserIterator->next) {
while (my $data_line = DataIterator->next) {
... do stuff ...

106. Let's shorten the code
In my application I have:
while (my $account_id = AccountIterator->next) {
while (my $user_id = UserIterator->next) {
while (my $data_line = DataIterator->next) {
... do stuff ...
package MyLogEvent; use Moose;
with qw/Log::Message::Structured .... /;
use AccountIterator; use UserIterator;
has account_id => ( is => 'ro',
default => sub { AccountIterator->current } );
has user_id => ( is => 'ro',
default => sub { UserIterator->current } );

107. Let's shorten the code

108. Let's shorten the code
Wrap Log::Dispatch call + L::M::S

109. Let's shorten the code
Wrap Log::Dispatch call + L::M::S
log_warning( message => "Achtung!");

110. Let's shorten the code
Wrap Log::Dispatch call + L::M::S
log_warning( message => "Achtung!");
actually calls

111. Let's shorten the code
Wrap Log::Dispatch call + L::M::S
log_warning( message => "Achtung!");
actually calls
$logger->log( warning => MyLogEvent->new( @_ );

112. Let's shorten the code
Wrap Log::Dispatch call + L::M::S
log_warning( message => "Achtung!");
actually calls
$logger->log( warning => MyLogEvent->new( @_ );
outputs

113. Let's shorten the code
Wrap Log::Dispatch call + L::M::S
log_warning( message => "Achtung!");
actually calls
$logger->log( warning => MyLogEvent->new( @_ );
outputs
'{"__CLASS__":"MyLogEvent",
"account_id":42,
"user_id":42,
"date":"2010-03-28T23:15:52Z",
"hostname":"mymachine.domain",
message=>"Achtung!"}'

114. Exceptions...
Remember ?

115. Exceptions...
Remember ?
try { ... } catch {
log_warning(exception => $_ )
};

116. Exceptions...
Remember ?
try { ... } catch {
log_warning(exception => $_ )
};
• To be able to write that, you need in your
MyLogEvent class:

117. Exceptions...
Remember ?
try { ... } catch {
log_warning(exception => $_ )
};
• To be able to write that, you need in your
MyLogEvent class:
• these attributes : 'exception', 'stack_trace',

118. Exceptions...
Remember ?
try { ... } catch {
log_warning(exception => $_ )
};
• To be able to write that, you need in your
MyLogEvent class:
• these attributes : 'exception', 'stack_trace',
• with a BUILDARGS

119. Exceptions...
Remember ?
try { ... } catch {
log_warning(exception => $_ )
};
• To be able to write that, you need in your
MyLogEvent class:
• these attributes : 'exception', 'stack_trace',
• with a BUILDARGS
• that looks at the exception, and set the
'message' and the 'stack_trace' arguments

120. around BUILDARGS => sub {
my $orig = shift;
my $class = shift;
my $h = $class->$orig(@_);
my $e = $h->{exception}
or return $h;
if ( blessed($e) && $e->isa('My::Exception::Base') ) {
$h->{message} = $e->message;
$h->{shortmess} = $e->{_shortmess};
} else {
$h->{message} = $e
}
return $h;
};

121. around BUILDARGS => sub {
my $orig = shift;
my $class = shift;
my $h = $class->$orig(@_);
my $e = $h->{exception}
or return $h;
if ( blessed($e) && $e->isa('My::Exception::Base') ) {
$h->{message} = $e->message;
$h->{shortmess} = $e->{_shortmess};
} else {
$h->{message} = $e
}
return $h;
};
try { ... } catch {
log_warning(exception => $_ )
That would now
}; work !

122. Log::Message::Structured

123. Log::Message::Structured
• Does the job

124. Log::Message::Structured
• Does the job
• Please contribute

125. Log::Message::Structured
• Does the job
• Please contribute
• we need more components

126. Log::Message::Structured
• Does the job
• Please contribute
• we need more components
• we need more stringifiers

127. Log::Message::Structured
• Does the job
• Please contribute
• we need more components
• we need more stringifiers
• compatibility with Moo

128. Log::Message::Structured
• Does the job
• Please contribute
• we need more components
• we need more stringifiers
• compatibility with Moo
• On CPAN / github

130. • We have all we need as JSON

131. • We have all we need as JSON
• But that's not enough :(

132. • We have all we need as JSON
• But that's not enough :(
• Your boss wants to look at the log

133. • We have all we need as JSON
• But that's not enough :(
• Your boss wants to look at the log
• => provide a simple interface

134. • We have all we need as JSON
• But that's not enough :(
• Your boss wants to look at the log
• => provide a simple interface
• You need to do complex lookups / research

135. • We have all we need as JSON
• But that's not enough :(
• Your boss wants to look at the log
• => provide a simple interface
• You need to do complex lookups / research
• thousands of files, grep not good enough

136. • We have all we need as JSON
• But that's not enough :(
• Your boss wants to look at the log
• => provide a simple interface
• You need to do complex lookups / research
• thousands of files, grep not good enough
• => need a powerful search engine

137. • We have all we need as JSON
• But that's not enough :(
• Your boss wants to look at the log
• => provide a simple interface
• You need to do complex lookups / research
• thousands of files, grep not good enough
• => need a powerful search engine
• Also, what if you have different log sources

138. • We have all we need as JSON
• But that's not enough :(
• Your boss wants to look at the log
• => provide a simple interface
• You need to do complex lookups / research
• thousands of files, grep not good enough
• => need a powerful search engine
• Also, what if you have different log sources
• From DB events, Redis MQ, system logs ?

139. Web interfaces

140. Web interfaces
• Two interfaces

141. Web interfaces
• Two interfaces
• First one is easy to use

142. Web interfaces
• Two interfaces
• First one is easy to use
• Based on Dancer + MongoDB

143. Web interfaces
• Two interfaces
• First one is easy to use
• Based on Dancer + MongoDB
• Shows logs in natural manner, grouped by
job, hostname, and presented by time

144. Web interfaces
• Two interfaces
• First one is easy to use
• Based on Dancer + MongoDB
• Shows logs in natural manner, grouped by
job, hostname, and presented by time
• Second one is more for data mining

145. Web interfaces
• Two interfaces
• First one is easy to use
• Based on Dancer + MongoDB
• Shows logs in natural manner, grouped by
job, hostname, and presented by time
• Second one is more for data mining
• Elasticsearch: deep searching, filtering, etc

146. We need some kind of big stuff
syslog
Standard log files
Redis MQ logs $stuff Web interface
Other logs Search Engine

147. We need some kind of big stuff
syslog
Standard log files
Redis MQ logs $stuff Web interface
Other logs Search Engine
$stuff = Message::Passing

148. Message::Passing

149. Message::Passing
• Written by Tomas Doran

150. Message::Passing
• Written by Tomas Doran
• Logstash-like but in Perl and with style

151. Message::Passing
• Written by Tomas Doran
• Logstash-like but in Perl and with style
• It's a daemon (AnyEvent based)

152. Message::Passing
• Written by Tomas Doran
• Logstash-like but in Perl and with style
• It's a daemon (AnyEvent based)
• basic concepts: inputs, filters, outputs

153. Message::Passing
• Written by Tomas Doran
• Logstash-like but in Perl and with style
• It's a daemon (AnyEvent based)
• basic concepts: inputs, filters, outputs
• run it as : command line, using the DSL,
using classes directly.

154. Real Life

155. Real Life
• My software runs on many machines

156. Real Life
• My software runs on many machines
• Connected to one syslog-ng server

157. Real Life
• My software runs on many machines
• Connected to one syslog-ng server
• Writing to log files

158. Real Life
• My software runs on many machines
• Connected to one syslog-ng server
• Writing to log files
• Configure syslog-ng to output to a pipe

159. Real Life
• My software runs on many machines
• Connected to one syslog-ng server
• Writing to log files
• Configure syslog-ng to output to a pipe
• Branch M::P on this named pipe

160. Real Life
• My software runs on many machines
• Connected to one syslog-ng server
• Writing to log files
• Configure syslog-ng to output to a pipe
• Branch M::P on this named pipe
• Web interface: Dancer + MongoDB

161. Real Life
• My software runs on many machines
• Connected to one syslog-ng server
• Writing to log files
• Configure syslog-ng to output to a pipe
• Branch M::P on this named pipe
• Web interface: Dancer + MongoDB
• Search interface : Elasticsearch + plugin

162. syslog-ng
many logs
machines
standard
log files

163. syslog-ng duplicated
flow
many logs
machines
standard
log files Message::Passing
ElasticSearch MongoDB
Web GUI Web GUI

164. logs MQ
Message::Passing
many
machines
In a better world
ElasticSearch MongoDB
Web GUI Web GUI

165. Let's use Message::Passing

166. Let's use Message::Passing
• step 1 : configure & run M::P daemon

167. Let's use Message::Passing
• step 1 : configure & run M::P daemon
• step 2 : ...

168. Let's use Message::Passing
• step 1 : configure & run M::P daemon
• step 2 : ...
• step 3 : PROFIT

169. Let's use Message::Passing
• step 1 : configure & run M::P daemon
• step 2 : ...
• step 3 : PROFIT
Remember the basic concepts?
inputs, filters, outputs

170. use Message::Passing::DSL; use Moose;
with 'Message::Passing::Role::Script';
sub build_chain { message_chain {
input filetail => (
class => 'FileTail',
filename => ‘/path/to/logs’,
output_to => 'cleanup_logs' );
filter cleanup_logs => (
class => '+My::Filter::DeJSON',
output_to => [ 'elasticsearch', 'mongodb' ] );
output elasticsearch_servers => (
class => 'ElasticSearch',
elasticsearch_servers => 'localhost:9200' );
output mongodb => (
class => 'MongoDB',
hostname => 'localhost',
database => 'reporting',
collection => 'logs',
indexes => [ # specify fields to index ] );}
}
__PACKAGE__->start;

171. My::Filter::DeJSON
package My::Filter::DeJSON;
use Moose;
sub consume {
my ( $self, $message ) = @_;
$message =~ m/(w+):JSON:(.*)/ or return;
my $structure_log = from_json($2);
$structure_log->{log_level} = $1;
foreach my $output_to ( @{ $self->output_to } ) {
$output_to->consume($structure_log);
}
}

172. Message::Passing::Output::MongoDB
use Moose; use MongoDB; use AnyEvent;
with qw/
Message::Passing::Role::Output
Message::Passing::Role::HasUsernameAndPassword
Message::Passing::Role::HasHostnameAndPort
/;
# ... hostname, port, user/pass attributes ...
has _db => (
is => 'ro', isa => 'MongoDB::Database', lazy => 1,
default => sub {
my $self = shift;
my $connection = MongoDB::Connection->new(
host => $self->hostname,
port => $self->port,
);
return $connection->get_database($self->database);
},
);

173. Message::Passing::Output::MongoDB
# ... _collection attribute
sub _build_logs_collection {
my ($self) = @_;
my $collection_name = $self->collection;
my $collection = $self->_db->$collection_name;
if ($self->_has_indexes) {
foreach my $index (@{$self->indexes}){
$collection->ensure_index(@$index);
}
}
return $collection;
}

174. Message::Passing::Output::MongoDB
sub consume {
my ($self, $data) = @_;
return unless $data;
my $date;
my $collection = $self->_collection;
$collection->insert($data)
or warn "Insertion failure: " . Dumper($data) . "n";
if ($self->verbose) {
$self->_inc_log_counter;
warn("Total " . $self->_log_counter
. " records inserted in MongoDBn");
}
}
On CPAN, by Bin Shu

175. Dancer web GUI

176. Dancer web GUI
• Very simple Dancer application

177. Dancer web GUI
• Very simple Dancer application
• Could have used Dancer::Plugin::Mongo

178. Dancer web GUI
• Very simple Dancer application
• Could have used Dancer::Plugin::Mongo
• A screen to display log

179. Dancer web GUI
• Very simple Dancer application
• Could have used Dancer::Plugin::Mongo
• A screen to display log
• A screen to search (filter) logs

180. Dancer web GUI
• Very simple Dancer application
• Could have used Dancer::Plugin::Mongo
• A screen to display log
• A screen to search (filter) logs
• Result page always show logs ordered by time

181. Dancer GUI

182. Dancer GUI

183. Dancer GUI routes

184. Dancer GUI routes
• get / :

185. Dancer GUI routes
• get / :
• a mongoDB group request per host, command, => /result

186. Dancer GUI routes
• get / :
• a mongoDB group request per host, command, => /result
• get /search :

187. Dancer GUI routes
• get / :
• a mongoDB group request per host, command, => /result
• get /search :
• a mapreduce to display asearch form, => /result

188. Dancer GUI routes
• get / :
• a mongoDB group request per host, command, => /result
• get /search :
• a mapreduce to display asearch form, => /result
• get /result :

189. Dancer GUI routes
• get / :
• a mongoDB group request per host, command, => /result
• get /search :
• a mapreduce to display asearch form, => /result
• get /result :
• get search params from url, issue a find request

190. Dancer GUI routes
• get / :
• a mongoDB group request per host, command, => /result
• get /search :
• a mapreduce to display asearch form, => /result
• get /result :
• get search params from url, issue a find request
• display in a table

191. Dancer GUI routes
• get / :
• a mongoDB group request per host, command, => /result
• get /search :
• a mapreduce to display asearch form, => /result
• get /result :
• get search params from url, issue a find request
• display in a table
• clicking on a result brings up a JS detailed result popup

192. Show me the video

193. Elasticsearch

194. Elasticsearch
• Installation : trivial

195. Elasticsearch
• Installation : trivial
• Configuration : none

196. Elasticsearch
• Installation : trivial
• Configuration : none
• Web GUI : elasticsearch-head

197. Elasticsearch
• Installation : trivial
• Configuration : none
• Web GUI : elasticsearch-head
• Code ? there is no code to show :)

198. Elasticsearch
• Installation : trivial
• Configuration : none
• Web GUI : elasticsearch-head
• Code ? there is no code to show :)
• Usage : perform advanced search on massive backlog

202. Show me the video

203. Additional stuff
• If there is an error in Message::Passing
• You should use the error channel
• To avoid loop of death

204. Conclusion
• That's all there is about it, really
• Message::Passing is easy and work great
• Log::Message::Structured : an awesome idea

205. Questions ?
Thanks !
Damien 'dams' Krotkine