Writing Pluggable Software Tatsuhiko Miyagawa   [email_address] Six Apart, Ltd. / Shibuya Perl Mongers YAPC::Asia 2007 Tokyo
For non-JP attendees … <ul><li>If you find  in the code, </li></ul><ul><li>Replace that with backslash. </li></ul><ul><li>...
<ul><li>Plaggable </li></ul><ul><li>Software </li></ul>
<ul><li>Plaggable </li></ul><ul><li>Software </li></ul>
<ul><li>Pl u ggable </li></ul><ul><li>Software </li></ul>
<ul><li>Agenda </li></ul>
<ul><li>#1 </li></ul><ul><li>How to make </li></ul><ul><li>your app pluggable </li></ul>
<ul><li>#2 </li></ul><ul><li>TMTOWTDP </li></ul><ul><li>There's More Than One Way To Deploy Plugins </li></ul><ul><li>Pros...
<ul><li>First-of-all: </li></ul><ul><li>Why pluggable? </li></ul>
<ul><li>Benefits </li></ul>
<ul><li>#1 </li></ul><ul><li>Keep the app design </li></ul><ul><li>and code simple </li></ul>
<ul><li>#2 </li></ul><ul><li>Let the app users </li></ul><ul><li>customize the behavior </li></ul><ul><li>(without hacking...
<ul><li>#3 </li></ul><ul><li>It's  fun  to write plugins </li></ul><ul><li>for most hackers </li></ul><ul><li>(see: Plagge...
<ul><li>&quot;Can your app do XXX?&quot; </li></ul><ul><li>&quot;Yes, by plugins.&quot; </li></ul>
<ul><li>&quot;Your app has a bug in YYY&quot; </li></ul><ul><li>&quot;No, it's the bug in plugin YYY, </li></ul><ul><li>No...
<ul><li>Good Enough </li></ul><ul><li>Reasons, huh? </li></ul>
<ul><li>#1 </li></ul><ul><li>Make your app pluggable </li></ul>
<ul><li>Example </li></ul>
<ul><li>ack (App::Ack) </li></ul>
grep –r for programmers
<ul><li>Ack is a &quot;full-stack&quot; </li></ul><ul><li>software now. </li></ul>
<ul><li>By &quot;full-stack&quot; I mean: </li></ul><ul><li>Easy install </li></ul><ul><li>No configuration </li></ul><ul>...
<ul><li>Specifically: </li></ul><ul><li>These are hardcoded </li></ul><ul><li>Ignored directories </li></ul><ul><li>Filena...
Ignored Directories <ul><li>@ignore_dirs = qw( blib CVS RCS SCCS .svn </li></ul><ul><li>_darcs .git ); </li></ul>
Filenames and languages mapping <ul><li>%mappings = ( </li></ul><ul><li>asm  => [qw( s S )], </li></ul><ul><li>binary  => ...
<ul><li>What if making </li></ul><ul><li>these pluggable? </li></ul>
<ul><li>DISCLAIMER </li></ul>
<ul><li>Don't get me wrong Andy, </li></ul><ul><li>I love ack the way it is… </li></ul><ul><li>Just thought it can be </li...
<ul><li>Quickstart: </li></ul><ul><li>Class::Trigger Module::Pluggable </li></ul><ul><li>© Six Apart Ltd. Employees </li><...
Class::Trigger SYNOPSIS <ul><li>package Foo; </li></ul><ul><li>use Class::Trigger; </li></ul><ul><li>sub foo { </li></ul><...
<ul><li>Class::Trigger </li></ul><ul><li>Helps you to implement </li></ul><ul><li>Observer Pattern. </li></ul><ul><li>(Rai...
Module::Pluggable SYNOPSIS <ul><li>package MyClass; </li></ul><ul><li>use Module::Pluggable; </li></ul><ul><li>use MyClass...
Setup plugins in App::Ack <ul><li>package App::Ack; </li></ul><ul><li>use Class::Trigger; </li></ul><ul><li>use Module::Pl...
Setup plugins in App::Ack <ul><li>package App::Ack; </li></ul><ul><li>use Class::Trigger; </li></ul><ul><li>use Module::Pl...
Ignored Directories (Before) <ul><li>@ignore_dirs = qw( blib CVS RCS SCCS .svn </li></ul><ul><li>_darcs .git ); </li></ul>
Ignored Directories (After) <ul><li># lib/App/Ack.pm </li></ul><ul><li>__PACKAGE__->call_trigger(' ignore_dirs.add ', @ign...
Ignored Directories (plugins) <ul><li># lib/App/Ack/Plugin/IgnorePerlBuildDir.pm </li></ul><ul><li>package App::Ack::Plugi...
Ignored Directories (plugins) <ul><li># lib/App/Ack/Plugin/IgnoreSourceControlDir.pm </li></ul><ul><li>package App::Ack::P...
Filenames and languages (before) <ul><li>%mappings = ( </li></ul><ul><li>asm  => [qw( s S )], </li></ul><ul><li>binary  =>...
Filenames and languages (after) <ul><li># lib/App/Ack.pm </li></ul><ul><li>__PACKAGE__->call_trigger('mappings.add', mappi...
Filenames and languages (plugins) <ul><li>package App::Ack::Plugin::MappingCFamily; </li></ul><ul><li>use strict; </li></u...
<ul><li>Works great  </li></ul><ul><li>with few lines of code! </li></ul>
<ul><li>Now it's time to add  </li></ul><ul><li>Some useful stuff. </li></ul>
<ul><li>Example Plugin: </li></ul><ul><li>Content Filter </li></ul>
<ul><li>sub _search { </li></ul><ul><li>my $fh = shift; </li></ul><ul><li>my $is_binary = shift; </li></ul><ul><li>my $fil...
<ul><li>Example: </li></ul><ul><li>Search PDF content </li></ul><ul><li>with ack </li></ul>
PDF filter plugin <ul><li>package App::Ack::Plugin::ExtractContentPDF; </li></ul><ul><li>use strict; </li></ul><ul><li>use...
PDF filter plugin (cont.) <ul><li>App::Ack->add_trigger( </li></ul><ul><li>'filter.binary' => sub { </li></ul><ul><li>my($...
PDF search <ul><li>> ack --type=pdf Audrey </li></ul><ul><li>yapcasia2007-pugs.pdf:3:Audrey Tang </li></ul>
<ul><li>Homework </li></ul><ul><li>Use File::Extract </li></ul><ul><li>To handle arbitrary media files </li></ul>
<ul><li>Homework 2: </li></ul><ul><li>Search non UTF-8 files </li></ul><ul><li>(hint: use Encode::Guess) </li></ul><ul><li...
Summary <ul><li>Class::Trigger </li></ul><ul><li>+ Module::Pluggable </li></ul><ul><li>= Pluggable app easy </li></ul>
<ul><li>#2 </li></ul><ul><li>TMTOWTDP </li></ul><ul><li>There's More Than One Way To Deploy Plugins </li></ul>
<ul><li>Module::Pluggable </li></ul><ul><li>+ Class::Trigger </li></ul><ul><li>= Simple and Nice </li></ul><ul><li>but has...
<ul><li>In Reality,  </li></ul><ul><li>we need more control over how plugins behave </li></ul>
<ul><li>1) </li></ul><ul><li>The order of  </li></ul><ul><li>plugin executions </li></ul>
<ul><li>2) </li></ul><ul><li>Per user configurations </li></ul><ul><li>for plugins </li></ul>
<ul><li>3) </li></ul><ul><li>Temporarily </li></ul><ul><li>Disable plugins </li></ul><ul><li>Should be easy </li></ul>
<ul><li>4) </li></ul><ul><li>How to install </li></ul><ul><li>& upgrade plugins </li></ul>
<ul><li>5) </li></ul><ul><li>Let plugins  </li></ul><ul><li>have storage area </li></ul>
<ul><li>Etc, etc. </li></ul>
<ul><li>Examples: </li></ul><ul><li>Kwiki </li></ul><ul><li>Plagger </li></ul><ul><li>qpsmtpd </li></ul><ul><li>Movable Ty...
<ul><li>I won't talk about </li></ul><ul><li>Catalyst plugins </li></ul><ul><li>(and other framework thingy) </li></ul>
<ul><li>Because they're  </li></ul><ul><li>NOT </li></ul><ul><li>&quot;plug-ins&quot; </li></ul>
<ul><li>Install plugins  </li></ul><ul><li>And now you write  </li></ul><ul><li>MORE CODE </li></ul>
<ul><li>95% of Catalyst plugins </li></ul><ul><li>Are NOT &quot;plugins&quot; </li></ul><ul><li>But &quot;components&quot;...
<ul><li>Kwiki 1.0 </li></ul>
Kwiki Plugin code <ul><li>package Kwiki::URLBL; </li></ul><ul><li>use  Kwiki::Plugin -Base ; </li></ul><ul><li>use Kwiki::...
Kwiki Plugin (cont.) <ul><li>sub urlbl_hook { </li></ul><ul><li>my $hook = pop; </li></ul><ul><li>my $old_page = $self->hu...
<ul><li>Magic implemented </li></ul><ul><li>in Spoon(::Hooks) </li></ul>
&quot;Install&quot; Kwiki Plugins <ul><li># order doesn't matter here (according to Ingy) </li></ul><ul><li>Kwiki::Display...
Kwiki plugin config <ul><li># in Kwiki::URLBL plugin </li></ul><ul><li>__config/urlbl.yaml__ </li></ul><ul><li>urlbl_dns: ...
Kwiki plugins are CPAN modules
Install and Upgrade plugins <ul><li>cpan> install Kwiki::SomeStuff </li></ul>
Using CPAN as a repository <ul><li>Pros #1: </li></ul><ul><li>reuse most of current CPAN infrastructure. </li></ul>
Using CPAN as a repository <ul><li>Pros #2: </li></ul><ul><li>Increasing # of modules </li></ul><ul><li>= good motivation ...
<ul><li>Cons #1: </li></ul><ul><li>Installing CPAN deps </li></ul><ul><li>could be a mess </li></ul><ul><li>(especially fo...
<ul><li>Cons #2: </li></ul><ul><li>Whenever Ingy releases </li></ul><ul><li>new Kwiki, lots of plugins just break. </li></ul>
Kwiki plugin storage <ul><li>return if  </li></ul><ul><li>grep {$page->id} </li></ul><ul><li>@{$self-> config->cached_disp...
<ul><li>Kwiki 2.0 </li></ul>
<ul><li>Same as Kwiki 1.0 </li></ul>
<ul><li>Except: </li></ul><ul><li>plugins are now </li></ul><ul><li>in SVN repository </li></ul>
 
Plagger plugin <ul><li>package Plagger::Plugin::Publish::iCal; </li></ul><ul><li>use strict; </li></ul><ul><li>use base qw...
Plagger plugin (cont) <ul><li>sub plugin_init { </li></ul><ul><li>my($self, $context) = @_; </li></ul><ul><li>my $dir =  $...
Plagger plugin storage <ul><li>$self->conf->{invindex} ||= </li></ul><ul><li>$self-> cache->path_to ('invindex'); </li></ul>
Plagger plugin config <ul><li># The order matters in config.yaml </li></ul><ul><li># if they're in the same hooks </li></u...
Plugins Install & Upgrade <ul><li>cpan> notest install Plagger </li></ul><ul><li># or … </li></ul><ul><li>> svn co http://...
<ul><li>Plagger impl. </li></ul><ul><li>ripped off by </li></ul><ul><li>many apps now </li></ul>
<ul><li>qpsmtpd </li></ul>
<ul><li>mod_perl for SMTP </li></ul><ul><li>Runs with tcpserver, forkserver  </li></ul><ul><li>or Danga::Socket standalone...
Plugins: Flat files <ul><li>rock:/home/miyagawa/svn/qpsmtpd> ls -F plugins </li></ul><ul><li>async/  greylisting </li></ul...
qpsmtpd plugin <ul><li>sub  hook_mail  { </li></ul><ul><li>my ($self, $transaction, $sender, %param) = @_; </li></ul><ul><...
<ul><li>Actually qpsmtpd </li></ul><ul><li>Plugins are &quot;compiled&quot; </li></ul><ul><li>to modules </li></ul>
<ul><li>my $eval = join(&quot;
&quot;, </li></ul><ul><li>&quot;package $package;&quot;, </li></ul><ul><li>'use Qpsmtpd::Co...
qpsmtpd plugin config <ul><li>rock:/home/miyagawa/svn/qpsmtpd> ls config.sample/ </li></ul><ul><li>config.sample: </li></u...
config/plugins <ul><li># content filters </li></ul><ul><li>virus/klez_filter </li></ul><ul><li># rejects mails with a SA s...
config/badhelo <ul><li># these domains never uses their domain when greeting us, so reject transactions </li></ul><ul><li>...
Install & Upgrade plugins <ul><li>Just use subversion </li></ul>
 
<ul><li>MT plugins are  </li></ul><ul><li>flat-files </li></ul><ul><li>(or scripts that call modules) </li></ul>
MT plugin code <ul><li>package MT::Plugin::BanASCII;  </li></ul><ul><li>our $Method = &quot;deny&quot;; </li></ul><ul><li>...
MT plugin code (cont) <ul><li>sub init_app {  </li></ul><ul><li>my $plugin = shift; </li></ul><ul><li>$plugin->SUPER::init...
 
 
MT plugin storage <ul><li>require MT::PluginData; </li></ul><ul><li>my $data = MT::PluginData->load({ </li></ul><ul><li>pl...
Order control <ul><li>MT->add_callback('CMSPostEntrySave',  9 , $rightfields, &CMSPostEntrySave); </li></ul><ul><li>MT->ad...
Conclusion <ul><li>Flat-files </li></ul><ul><li>vs. </li></ul><ul><li>Modules </li></ul>
<ul><li>Flat-files: </li></ul><ul><li>☺  Easy to install (Just grab it) </li></ul><ul><li>☻  Hard to upgrade </li></ul><ul...
<ul><li>Modules: </li></ul><ul><li>☺  Full-access to Perl OO goodness </li></ul><ul><li>☺  Avoid duplicate efforts of CPAN...
<ul><li>Nice-to-haves: </li></ul><ul><li>Order control </li></ul><ul><li>Temporarily disable plugins </li></ul><ul><li>Per...
Resources <ul><li>Class::Trigger </li></ul><ul><li>http:// search.cpan.org /dist/Class-Trigger/ </li></ul><ul><li>Module::...
Upcoming SlideShare
Loading in...5
×

Writing Pluggable Software

13,447

Published on

"How to write pluggable software" presented by Tatsuhiko Miyagawa at YAPC::Asia 2007 in Tokyo on April 5th 2007.

Published in: Technology
0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
13,447
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
8
Embeds 0
No embeds

No notes for slide

Writing Pluggable Software

  1. 1. Writing Pluggable Software Tatsuhiko Miyagawa [email_address] Six Apart, Ltd. / Shibuya Perl Mongers YAPC::Asia 2007 Tokyo
  2. 2. For non-JP attendees … <ul><li>If you find in the code, </li></ul><ul><li>Replace that with backslash. </li></ul><ul><li>(This is MS' fault) </li></ul>
  3. 3. <ul><li>Plaggable </li></ul><ul><li>Software </li></ul>
  4. 4. <ul><li>Plaggable </li></ul><ul><li>Software </li></ul>
  5. 5. <ul><li>Pl u ggable </li></ul><ul><li>Software </li></ul>
  6. 6. <ul><li>Agenda </li></ul>
  7. 7. <ul><li>#1 </li></ul><ul><li>How to make </li></ul><ul><li>your app pluggable </li></ul>
  8. 8. <ul><li>#2 </li></ul><ul><li>TMTOWTDP </li></ul><ul><li>There's More Than One Way To Deploy Plugins </li></ul><ul><li>Pros/Cons by examples </li></ul>
  9. 9. <ul><li>First-of-all: </li></ul><ul><li>Why pluggable? </li></ul>
  10. 10. <ul><li>Benefits </li></ul>
  11. 11. <ul><li>#1 </li></ul><ul><li>Keep the app design </li></ul><ul><li>and code simple </li></ul>
  12. 12. <ul><li>#2 </li></ul><ul><li>Let the app users </li></ul><ul><li>customize the behavior </li></ul><ul><li>(without hacking the internals) </li></ul>
  13. 13. <ul><li>#3 </li></ul><ul><li>It's fun to write plugins </li></ul><ul><li>for most hackers </li></ul><ul><li>(see: Plagger and Kwiki) </li></ul>
  14. 14. <ul><li>&quot;Can your app do XXX?&quot; </li></ul><ul><li>&quot;Yes, by plugins.&quot; </li></ul>
  15. 15. <ul><li>&quot;Your app has a bug in YYY&quot; </li></ul><ul><li>&quot;No, it's the bug in plugin YYY, </li></ul><ul><li>Not my fault.&quot; </li></ul><ul><li>(Chain Of Responsibilities) </li></ul>
  16. 16. <ul><li>Good Enough </li></ul><ul><li>Reasons, huh? </li></ul>
  17. 17. <ul><li>#1 </li></ul><ul><li>Make your app pluggable </li></ul>
  18. 18. <ul><li>Example </li></ul>
  19. 19. <ul><li>ack (App::Ack) </li></ul>
  20. 20. grep –r for programmers
  21. 21. <ul><li>Ack is a &quot;full-stack&quot; </li></ul><ul><li>software now. </li></ul>
  22. 22. <ul><li>By &quot;full-stack&quot; I mean: </li></ul><ul><li>Easy install </li></ul><ul><li>No configuration </li></ul><ul><li>No way to extend </li></ul>
  23. 23. <ul><li>Specifically: </li></ul><ul><li>These are hardcoded </li></ul><ul><li>Ignored directories </li></ul><ul><li>Filenames and types </li></ul>
  24. 24. Ignored Directories <ul><li>@ignore_dirs = qw( blib CVS RCS SCCS .svn </li></ul><ul><li>_darcs .git ); </li></ul>
  25. 25. Filenames and languages mapping <ul><li>%mappings = ( </li></ul><ul><li>asm => [qw( s S )], </li></ul><ul><li>binary => …, </li></ul><ul><li>cc => [qw( c h xs )], </li></ul><ul><li>cpp => [qw( cpp m h C H )], </li></ul><ul><li>csharp => [qw( cs )], </li></ul><ul><li>… </li></ul><ul><li>perl => [qw( pl pm pod tt ttml t )], </li></ul><ul><li>… </li></ul><ul><li>); </li></ul>
  26. 26. <ul><li>What if making </li></ul><ul><li>these pluggable? </li></ul>
  27. 27. <ul><li>DISCLAIMER </li></ul>
  28. 28. <ul><li>Don't get me wrong Andy, </li></ul><ul><li>I love ack the way it is… </li></ul><ul><li>Just thought it can be </li></ul><ul><li>a very good example for the tutorial. </li></ul>
  29. 29. <ul><li>Quickstart: </li></ul><ul><li>Class::Trigger Module::Pluggable </li></ul><ul><li>© Six Apart Ltd. Employees </li></ul>
  30. 30. Class::Trigger SYNOPSIS <ul><li>package Foo; </li></ul><ul><li>use Class::Trigger; </li></ul><ul><li>sub foo { </li></ul><ul><li>my $self = shift; </li></ul><ul><li>$self->call_trigger('before_foo'); </li></ul><ul><li># some code ... </li></ul><ul><li>$self->call_trigger('after_foo'); </li></ul><ul><li>} </li></ul><ul><li>package main; </li></ul><ul><li>Foo->add_trigger(before_foo => &sub1); </li></ul><ul><li>Foo->add_trigger(after_foo => &sub2); </li></ul>
  31. 31. <ul><li>Class::Trigger </li></ul><ul><li>Helps you to implement </li></ul><ul><li>Observer Pattern. </li></ul><ul><li>(Rails calls this Observer) </li></ul>
  32. 32. Module::Pluggable SYNOPSIS <ul><li>package MyClass; </li></ul><ul><li>use Module::Pluggable; </li></ul><ul><li>use MyClass; </li></ul><ul><li>my $mc = MyClass->new(); </li></ul><ul><li># returns the names of all plugins installed under MyClass::Plugin::* </li></ul><ul><li>my @plugins = $mc->plugins(); </li></ul><ul><li>package MyClass::Plugin::Foo; </li></ul><ul><li>sub new { … } </li></ul><ul><li>1; </li></ul>
  33. 33. Setup plugins in App::Ack <ul><li>package App::Ack; </li></ul><ul><li>use Class::Trigger; </li></ul><ul><li>use Module::Pluggable require => 1; </li></ul><ul><li>__PACKAGE__->plugins; </li></ul>
  34. 34. Setup plugins in App::Ack <ul><li>package App::Ack; </li></ul><ul><li>use Class::Trigger; </li></ul><ul><li>use Module::Pluggable require => 1 ; </li></ul><ul><li>__PACKAGE__->plugins; # &quot;requires&quot; modules </li></ul>
  35. 35. Ignored Directories (Before) <ul><li>@ignore_dirs = qw( blib CVS RCS SCCS .svn </li></ul><ul><li>_darcs .git ); </li></ul>
  36. 36. Ignored Directories (After) <ul><li># lib/App/Ack.pm </li></ul><ul><li>__PACKAGE__->call_trigger(' ignore_dirs.add ', @ignore_dirs); </li></ul>
  37. 37. Ignored Directories (plugins) <ul><li># lib/App/Ack/Plugin/IgnorePerlBuildDir.pm </li></ul><ul><li>package App::Ack::Plugin::IgnorePerlBuildDir; </li></ul><ul><li>App::Ack->add_trigger( </li></ul><ul><li>&quot; ignore_dirs.add &quot; => sub { </li></ul><ul><li>my($class, $ignore_dirs) = @_; </li></ul><ul><li>push @$ignore_dirs, qw( blib ); </li></ul><ul><li>}, </li></ul><ul><li>); </li></ul><ul><li>1; </li></ul>
  38. 38. Ignored Directories (plugins) <ul><li># lib/App/Ack/Plugin/IgnoreSourceControlDir.pm </li></ul><ul><li>package App::Ack::Plugin::IgnoreSourcdeControlDir; </li></ul><ul><li>App::Ack->add_trigger( </li></ul><ul><li>&quot; ignore_dirs.add &quot; => sub { </li></ul><ul><li>my($class, $ignore_dirs) = @_; </li></ul><ul><li>push @$ignore_dirs, qw( CVS RCS .svn </li></ul><ul><li>_darcs .git ); </li></ul><ul><li>}, </li></ul><ul><li>); </li></ul><ul><li>1; </li></ul>
  39. 39. Filenames and languages (before) <ul><li>%mappings = ( </li></ul><ul><li>asm => [qw( s S )], </li></ul><ul><li>binary => …, </li></ul><ul><li>cc => [qw( c h xs )], </li></ul><ul><li>cpp => [qw( cpp m h C H )], </li></ul><ul><li>csharp => [qw( cs )], </li></ul><ul><li>… </li></ul><ul><li>perl => [qw( pl pm pod tt ttml t )], </li></ul><ul><li>… </li></ul><ul><li>); </li></ul>
  40. 40. Filenames and languages (after) <ul><li># lib/App/Ack.pm </li></ul><ul><li>__PACKAGE__->call_trigger('mappings.add', mappings); </li></ul>
  41. 41. Filenames and languages (plugins) <ul><li>package App::Ack::Plugin::MappingCFamily; </li></ul><ul><li>use strict; </li></ul><ul><li>App::Ack->add_trigger( </li></ul><ul><li>&quot;mappings.add&quot; => sub { </li></ul><ul><li>my($class, $mappings) = @_; </li></ul><ul><li>$mappings->{asm} = [qw( s S )]; </li></ul><ul><li>$mappings->{cc} = [qw( c h xs )]; </li></ul><ul><li>$mappings->{cpp} = [qw( cpp m h C H )]; </li></ul><ul><li>$mappings->{csharp} = [qw( cs )]; </li></ul><ul><li>$mappings->{css} = [qw( css )]; </li></ul><ul><li>}, </li></ul><ul><li>); </li></ul><ul><li>1; </li></ul>
  42. 42. <ul><li>Works great </li></ul><ul><li>with few lines of code! </li></ul>
  43. 43. <ul><li>Now it's time to add </li></ul><ul><li>Some useful stuff. </li></ul>
  44. 44. <ul><li>Example Plugin: </li></ul><ul><li>Content Filter </li></ul>
  45. 45. <ul><li>sub _search { </li></ul><ul><li>my $fh = shift; </li></ul><ul><li>my $is_binary = shift; </li></ul><ul><li>my $filename = shift; </li></ul><ul><li>my $regex = shift; </li></ul><ul><li>my %opt = @_; </li></ul><ul><li>if ($is_binary) { </li></ul><ul><li>my $new_fh; </li></ul><ul><li>App::Ack->call_trigger('filter.binary', $filename, $new_fh); </li></ul><ul><li>if ($new_fh) { </li></ul><ul><li>return _search($new_fh, 0, $filename, $regex, @_); </li></ul><ul><li>} </li></ul><ul><li>} </li></ul>
  46. 46. <ul><li>Example: </li></ul><ul><li>Search PDF content </li></ul><ul><li>with ack </li></ul>
  47. 47. PDF filter plugin <ul><li>package App::Ack::Plugin::ExtractContentPDF; </li></ul><ul><li>use strict; </li></ul><ul><li>use CAM::PDF; </li></ul><ul><li>use File::Temp; </li></ul><ul><li>App::Ack->add_trigger( </li></ul><ul><li>'mappings.add' => sub { </li></ul><ul><li>my($class, $mappings) = @_; </li></ul><ul><li>$mappings->{pdf} = [qw(pdf)]; </li></ul><ul><li>}, </li></ul><ul><li>); </li></ul>
  48. 48. PDF filter plugin (cont.) <ul><li>App::Ack->add_trigger( </li></ul><ul><li>'filter.binary' => sub { </li></ul><ul><li>my($class, $filename, $fh_ref) = @_; </li></ul><ul><li>if ($filename =~ /.pdf$/) { </li></ul><ul><li>my $fh = File::Temp::tempfile; </li></ul><ul><li>my $doc = CAM::PDF->new($file); </li></ul><ul><li>my $text; </li></ul><ul><li>for my $page (1..$doc->numPages){ </li></ul><ul><li>$text .= $doc->getPageText($page); </li></ul><ul><li>} </li></ul><ul><li>print $fh $text; </li></ul><ul><li>seek $$fh, 0, 0; </li></ul><ul><li>$$fh_ref = $fh; </li></ul><ul><li>} </li></ul><ul><li>}, </li></ul><ul><li>); </li></ul>
  49. 49. PDF search <ul><li>> ack --type=pdf Audrey </li></ul><ul><li>yapcasia2007-pugs.pdf:3:Audrey Tang </li></ul>
  50. 50. <ul><li>Homework </li></ul><ul><li>Use File::Extract </li></ul><ul><li>To handle arbitrary media files </li></ul>
  51. 51. <ul><li>Homework 2: </li></ul><ul><li>Search non UTF-8 files </li></ul><ul><li>(hint: use Encode::Guess) </li></ul><ul><li>You'll need another hook. </li></ul>
  52. 52. Summary <ul><li>Class::Trigger </li></ul><ul><li>+ Module::Pluggable </li></ul><ul><li>= Pluggable app easy </li></ul>
  53. 53. <ul><li>#2 </li></ul><ul><li>TMTOWTDP </li></ul><ul><li>There's More Than One Way To Deploy Plugins </li></ul>
  54. 54. <ul><li>Module::Pluggable </li></ul><ul><li>+ Class::Trigger </li></ul><ul><li>= Simple and Nice </li></ul><ul><li>but has limitations </li></ul>
  55. 55. <ul><li>In Reality, </li></ul><ul><li>we need more control over how plugins behave </li></ul>
  56. 56. <ul><li>1) </li></ul><ul><li>The order of </li></ul><ul><li>plugin executions </li></ul>
  57. 57. <ul><li>2) </li></ul><ul><li>Per user configurations </li></ul><ul><li>for plugins </li></ul>
  58. 58. <ul><li>3) </li></ul><ul><li>Temporarily </li></ul><ul><li>Disable plugins </li></ul><ul><li>Should be easy </li></ul>
  59. 59. <ul><li>4) </li></ul><ul><li>How to install </li></ul><ul><li>& upgrade plugins </li></ul>
  60. 60. <ul><li>5) </li></ul><ul><li>Let plugins </li></ul><ul><li>have storage area </li></ul>
  61. 61. <ul><li>Etc, etc. </li></ul>
  62. 62. <ul><li>Examples: </li></ul><ul><li>Kwiki </li></ul><ul><li>Plagger </li></ul><ul><li>qpsmtpd </li></ul><ul><li>Movable Type </li></ul>
  63. 63. <ul><li>I won't talk about </li></ul><ul><li>Catalyst plugins </li></ul><ul><li>(and other framework thingy) </li></ul>
  64. 64. <ul><li>Because they're </li></ul><ul><li>NOT </li></ul><ul><li>&quot;plug-ins&quot; </li></ul>
  65. 65. <ul><li>Install plugins </li></ul><ul><li>And now you write </li></ul><ul><li>MORE CODE </li></ul>
  66. 66. <ul><li>95% of Catalyst plugins </li></ul><ul><li>Are NOT &quot;plugins&quot; </li></ul><ul><li>But &quot;components&quot; </li></ul><ul><li>95% of these statistics is made up by the speakers. </li></ul>
  67. 67. <ul><li>Kwiki 1.0 </li></ul>
  68. 68. Kwiki Plugin code <ul><li>package Kwiki::URLBL; </li></ul><ul><li>use Kwiki::Plugin -Base ; </li></ul><ul><li>use Kwiki::Installer -base; </li></ul><ul><li>const class_id => 'urlbl'; </li></ul><ul><li>const class_title => 'URL Blacklist DNS'; </li></ul><ul><li>const config_file => 'urlbl.yaml'; </li></ul><ul><li>sub register { </li></ul><ul><li>require URI::Find; </li></ul><ul><li>my $registry = shift; </li></ul><ul><li>$registry->add(hook => 'edit:save', pre => 'urlbl_hook'); </li></ul><ul><li>$registry->add(action => 'blacklisted_url'); </li></ul><ul><li>} </li></ul>
  69. 69. Kwiki Plugin (cont.) <ul><li>sub urlbl_hook { </li></ul><ul><li>my $hook = pop; </li></ul><ul><li>my $old_page = $self->hub->pages->new_page($self->pages->current->id); </li></ul><ul><li>my $this = $self->hub->urlbl; </li></ul><ul><li>my @old_urls = $this->get_urls($old_page->content); </li></ul><ul><li>my @urls = $this->get_urls($self->cgi->page_content); </li></ul><ul><li>my @new_urls = $this->get_new_urls(@old_urls, @urls); </li></ul><ul><li>if (@new_urls && $this->is_blocked(@new_urls)) { </li></ul><ul><li>$hook->cancel(); </li></ul><ul><li>return $self->redirect(&quot;action=blacklisted_url&quot;); </li></ul><ul><li>} </li></ul><ul><li>} </li></ul>
  70. 70. <ul><li>Magic implemented </li></ul><ul><li>in Spoon(::Hooks) </li></ul>
  71. 71. &quot;Install&quot; Kwiki Plugins <ul><li># order doesn't matter here (according to Ingy) </li></ul><ul><li>Kwiki::Display </li></ul><ul><li>Kwiki::Edit </li></ul><ul><li>Kwiki::Theme::Basic </li></ul><ul><li>Kwiki::Toolbar </li></ul><ul><li>Kwiki::Status </li></ul><ul><li>Kwiki::Widgets </li></ul><ul><li># Comment out (or entirely remove) to disable </li></ul><ul><li># Kwiki::UnnecessaryStuff </li></ul>
  72. 72. Kwiki plugin config <ul><li># in Kwiki::URLBL plugin </li></ul><ul><li>__config/urlbl.yaml__ </li></ul><ul><li>urlbl_dns: sc.surbl.org, bsb.spamlookup.net, rbl.bulkfeeds.jp </li></ul><ul><li># config.yaml </li></ul><ul><li>urlbl_dns: myowndns.example.org </li></ul>
  73. 73. Kwiki plugins are CPAN modules
  74. 74. Install and Upgrade plugins <ul><li>cpan> install Kwiki::SomeStuff </li></ul>
  75. 75. Using CPAN as a repository <ul><li>Pros #1: </li></ul><ul><li>reuse most of current CPAN infrastructure. </li></ul>
  76. 76. Using CPAN as a repository <ul><li>Pros #2: </li></ul><ul><li>Increasing # of modules </li></ul><ul><li>= good motivation </li></ul><ul><li>for Perl hackers </li></ul>
  77. 77. <ul><li>Cons #1: </li></ul><ul><li>Installing CPAN deps </li></ul><ul><li>could be a mess </li></ul><ul><li>(especially for Win32) </li></ul>
  78. 78. <ul><li>Cons #2: </li></ul><ul><li>Whenever Ingy releases </li></ul><ul><li>new Kwiki, lots of plugins just break. </li></ul>
  79. 79. Kwiki plugin storage <ul><li>return if </li></ul><ul><li>grep {$page->id} </li></ul><ul><li>@{$self-> config->cached_display_ignore }; </li></ul><ul><li>my $html = io->catfile( </li></ul><ul><li>$self-> plugin_directory ,$page->id </li></ul><ul><li>)->utf8; </li></ul>
  80. 80. <ul><li>Kwiki 2.0 </li></ul>
  81. 81. <ul><li>Same as Kwiki 1.0 </li></ul>
  82. 82. <ul><li>Except: </li></ul><ul><li>plugins are now </li></ul><ul><li>in SVN repository </li></ul>
  83. 84. Plagger plugin <ul><li>package Plagger::Plugin::Publish::iCal; </li></ul><ul><li>use strict; </li></ul><ul><li>use base qw( Plagger::Plugin ); </li></ul><ul><li>use Data::ICal; </li></ul><ul><li>use Data::ICal::Entry::Event; </li></ul><ul><li>use DateTime::Duration; </li></ul><ul><li>use DateTime::Format::ICal; </li></ul><ul><li>sub register { </li></ul><ul><li>my($self, $context) = @_; </li></ul><ul><li>$context-> register_hook ( </li></ul><ul><li>$self, </li></ul><ul><li>' publish.feed ' => &publish_feed, </li></ul><ul><li>' plugin.init ' => &plugin_init, </li></ul><ul><li>); </li></ul><ul><li>} </li></ul>
  84. 85. Plagger plugin (cont) <ul><li>sub plugin_init { </li></ul><ul><li>my($self, $context) = @_; </li></ul><ul><li>my $dir = $self->conf->{dir}; </li></ul><ul><li>unless (-e $dir && -d _) { </li></ul><ul><li>mkdir $dir, 0755 </li></ul><ul><li>or $context->error(&quot;Failed to mkdir $dir: $!&quot;); </li></ul><ul><li>} </li></ul><ul><li>} </li></ul>
  85. 86. Plagger plugin storage <ul><li>$self->conf->{invindex} ||= </li></ul><ul><li>$self-> cache->path_to ('invindex'); </li></ul>
  86. 87. Plagger plugin config <ul><li># The order matters in config.yaml </li></ul><ul><li># if they're in the same hooks </li></ul><ul><li>plugins: </li></ul><ul><li>- module: Subscription::Config </li></ul><ul><li>config: </li></ul><ul><li>feed: </li></ul><ul><li>- http://www.example.com/ </li></ul><ul><li>- module: Filter::DegradeYouTube </li></ul><ul><li>config: </li></ul><ul><li>dev_id: XYZXYZ </li></ul><ul><li>- module: Publish::Gmail </li></ul><ul><li>disable: 1 </li></ul>
  87. 88. Plugins Install & Upgrade <ul><li>cpan> notest install Plagger </li></ul><ul><li># or … </li></ul><ul><li>> svn co http://…/plagger/trunk plagger </li></ul><ul><li>> svn update </li></ul>
  88. 89. <ul><li>Plagger impl. </li></ul><ul><li>ripped off by </li></ul><ul><li>many apps now </li></ul>
  89. 90. <ul><li>qpsmtpd </li></ul>
  90. 91. <ul><li>mod_perl for SMTP </li></ul><ul><li>Runs with tcpserver, forkserver </li></ul><ul><li>or Danga::Socket standalone </li></ul>
  91. 92. Plugins: Flat files <ul><li>rock:/home/miyagawa/svn/qpsmtpd> ls -F plugins </li></ul><ul><li>async/ greylisting </li></ul><ul><li>auth/ hosts_allow </li></ul><ul><li>check_badmailfrom http_config </li></ul><ul><li>check_badmailfromto ident/ </li></ul><ul><li>check_badrcptto logging/ </li></ul><ul><li>check_badrcptto_patterns milter </li></ul><ul><li>check_basicheaders parse_addr_withhelo </li></ul><ul><li>check_earlytalker queue/ </li></ul><ul><li>check_loop quit_fortune </li></ul><ul><li>check_norelay rcpt_ok </li></ul><ul><li>check_relay relay_only </li></ul><ul><li>check_spamhelo require_resolvable_fromhost </li></ul><ul><li>content_log rhsbl </li></ul><ul><li>count_unrecognized_commands sender_permitted_from </li></ul><ul><li>dns_whitelist_soft spamassassin </li></ul><ul><li>dnsbl tls </li></ul><ul><li>domainkeys tls_cert* </li></ul><ul><li>dont_require_anglebrackets virus/ </li></ul>
  92. 93. qpsmtpd plugin <ul><li>sub hook_mail { </li></ul><ul><li>my ($self, $transaction, $sender, %param) = @_; </li></ul><ul><li>my @badmailfrom = $self->qp->config (&quot;badmailfrom&quot;) </li></ul><ul><li>or return (DECLINED); </li></ul><ul><li>for my $bad (@badmailfrom) { </li></ul><ul><li>my $reason = $bad; </li></ul><ul><li>$bad =~ s/^s*(S+).*/$1/; </li></ul><ul><li>next unless $bad; </li></ul><ul><li>$transaction->notes('badmailfrom', $reason) … </li></ul><ul><li>} </li></ul><ul><li>return (DECLINED); </li></ul><ul><li>} </li></ul>
  93. 94. <ul><li>Actually qpsmtpd </li></ul><ul><li>Plugins are &quot;compiled&quot; </li></ul><ul><li>to modules </li></ul>
  94. 95. <ul><li>my $eval = join(&quot; &quot;, </li></ul><ul><li>&quot;package $package;&quot;, </li></ul><ul><li>'use Qpsmtpd::Constants;', </li></ul><ul><li>&quot;require Qpsmtpd::Plugin;&quot;, </li></ul><ul><li>'use vars qw(@ISA);', </li></ul><ul><li>'use strict;', </li></ul><ul><li>'@ISA = qw(Qpsmtpd::Plugin);', </li></ul><ul><li>($test_mode ? 'use Test::More;' : ''), </li></ul><ul><li>&quot;sub plugin_name { qq[$plugin] }&quot;, </li></ul><ul><li>$line, </li></ul><ul><li>$sub, </li></ul><ul><li>&quot; &quot;, # last line comment without newline? </li></ul><ul><li>); </li></ul><ul><li>$eval =~ m/(.*)/s; </li></ul><ul><li>$eval = $1; </li></ul><ul><li>eval $eval; </li></ul><ul><li>die &quot;eval $@&quot; if $@; </li></ul>
  95. 96. qpsmtpd plugin config <ul><li>rock:/home/miyagawa/svn/qpsmtpd> ls config.sample/ </li></ul><ul><li>config.sample: </li></ul><ul><li>IP logging require_resolvable_fromhost </li></ul><ul><li>badhelo loglevel rhsbl_zones </li></ul><ul><li>badrcptto_patterns plugins size_threshold </li></ul><ul><li>dnsbl_zones rcpthosts tls_before_auth </li></ul><ul><li>invalid_resolvable_fromhost relayclients tls_ciphers </li></ul>
  96. 97. config/plugins <ul><li># content filters </li></ul><ul><li>virus/klez_filter </li></ul><ul><li># rejects mails with a SA score higher than 2 </li></ul><ul><li>spamassassin reject_threshold 20 </li></ul>
  97. 98. config/badhelo <ul><li># these domains never uses their domain when greeting us, so reject transactions </li></ul><ul><li>aol.com </li></ul><ul><li>yahoo.com </li></ul>
  98. 99. Install & Upgrade plugins <ul><li>Just use subversion </li></ul>
  99. 101. <ul><li>MT plugins are </li></ul><ul><li>flat-files </li></ul><ul><li>(or scripts that call modules) </li></ul>
  100. 102. MT plugin code <ul><li>package MT::Plugin::BanASCII; </li></ul><ul><li>our $Method = &quot;deny&quot;; </li></ul><ul><li>use MT; </li></ul><ul><li>use MT::Plugin; </li></ul><ul><li>my $plugin = MT::Plugin->new({ </li></ul><ul><li>name => &quot;BanASCII v$VERSION&quot;, </li></ul><ul><li>description => &quot;Deny or moderate ASCII or Latin-1 comment&quot;, </li></ul><ul><li>}); </li></ul><ul><li>MT->add_plugin($plugin); </li></ul><ul><li>MT->add_callback('CommentFilter', 2, $plugin, &handler); </li></ul>
  101. 103. MT plugin code (cont) <ul><li>sub init_app { </li></ul><ul><li>my $plugin = shift; </li></ul><ul><li>$plugin->SUPER::init_app(@_); </li></ul><ul><li>my($app) = @_; </li></ul><ul><li>return unless $app->isa('MT::App::CMS'); </li></ul><ul><li>$app-> add_itemset_action ({ </li></ul><ul><li>type => 'comment', </li></ul><ul><li>key => 'spam_submission_comment', </li></ul><ul><li>label => 'Report SPAM Comment(s)', </li></ul><ul><li>code => sub { </li></ul><ul><li>$plugin->submit_spams_action('MT::Comment', @_) }, </li></ul><ul><li>} </li></ul><ul><li>); </li></ul>
  102. 106. MT plugin storage <ul><li>require MT::PluginData; </li></ul><ul><li>my $data = MT::PluginData->load({ </li></ul><ul><li>plugin => 'sidebar-manager', </li></ul><ul><li>key => $blog_id }, </li></ul><ul><li>); </li></ul><ul><li>unless ($data) { </li></ul><ul><li>$data = MT::PluginData->new; </li></ul><ul><li>$data->plugin('sidebar-manager'); </li></ul><ul><li>$data->key($blog_id); </li></ul><ul><li>} </li></ul><ul><li>$data->data( $modulesets ); </li></ul><ul><li>$data->save or die $data->errstr; </li></ul>
  103. 107. Order control <ul><li>MT->add_callback('CMSPostEntrySave', 9 , $rightfields, &CMSPostEntrySave); </li></ul><ul><li>MT->add_callback('CMSPreSave_entry', 9 , $rightfields, &CMSPreSave_entry); </li></ul><ul><li>MT::Entry->add_callback('pre_remove', 9 , $rightfields, &entry_pre_remove); </li></ul>Defined in plugins. No Control on users end
  104. 108. Conclusion <ul><li>Flat-files </li></ul><ul><li>vs. </li></ul><ul><li>Modules </li></ul>
  105. 109. <ul><li>Flat-files: </li></ul><ul><li>☺ Easy to install (Just grab it) </li></ul><ul><li>☻ Hard to upgrade </li></ul><ul><li>OK for simple plugins </li></ul>
  106. 110. <ul><li>Modules: </li></ul><ul><li>☺ Full-access to Perl OO goodness </li></ul><ul><li>☺ Avoid duplicate efforts of CPAN </li></ul><ul><li>☻ Might be hard to resolve deps. </li></ul><ul><li>Subversion to the rescue </li></ul><ul><li>(could be a barrier for newbies) </li></ul>
  107. 111. <ul><li>Nice-to-haves: </li></ul><ul><li>Order control </li></ul><ul><li>Temporarily disable plugins </li></ul><ul><li>Per plugin config </li></ul><ul><li>Per plugin storage </li></ul>
  108. 112. Resources <ul><li>Class::Trigger </li></ul><ul><li>http:// search.cpan.org /dist/Class-Trigger/ </li></ul><ul><li>Module::Pluggable </li></ul><ul><li>http:// search.cpan.org /dist/Module-Pluggable/ </li></ul><ul><li>Ask Bjorn Hansen: Build Easily Extensible Perl Programs </li></ul><ul><li>http://conferences.oreillynet.com/cs/os2005/view/e_sess/6806 </li></ul><ul><li>qpsmtpd </li></ul><ul><li>http:// smtpd.develooper.com / </li></ul><ul><li>MT plugins </li></ul><ul><li>http:// www.sixapart.com/pronet/plugins / </li></ul><ul><li>Kwiki </li></ul><ul><li>http:// www.kwiki.org / </li></ul><ul><li>Plagger </li></ul><ul><li>http:// plagger.org / </li></ul>

×