• Save
Writing Pluggable Software
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Writing Pluggable Software

  • 21,498 views
Uploaded on

"How to write pluggable software" presented by Tatsuhiko Miyagawa at YAPC::Asia 2007 in Tokyo on April 5th 2007.

"How to write pluggable software" presented by Tatsuhiko Miyagawa at YAPC::Asia 2007 in Tokyo on April 5th 2007.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
21,498
On Slideshare
21,446
From Embeds
52
Number of Embeds
5

Actions

Shares
Downloads
0
Comments
0
Likes
7

Embeds 52

http://techvideos.humourbox.info 33
http://www.slideshare.net 13
http://www.linkedin.com 3
http://lj-toys.com 2
http://l.lj-toys.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Writing Pluggable Software Tatsuhiko Miyagawa [email_address] Six Apart, Ltd. / Shibuya Perl Mongers YAPC::Asia 2007 Tokyo
  • 2. For non-JP attendees …
    • If you find in the code,
    • Replace that with backslash.
    • (This is MS' fault)
  • 3.
    • Plaggable
    • Software
  • 4.
    • Plaggable
    • Software
  • 5.
    • Pl u ggable
    • Software
  • 6.
    • Agenda
  • 7.
    • #1
    • How to make
    • your app pluggable
  • 8.
    • #2
    • TMTOWTDP
    • There's More Than One Way To Deploy Plugins
    • Pros/Cons by examples
  • 9.
    • First-of-all:
    • Why pluggable?
  • 10.
    • Benefits
  • 11.
    • #1
    • Keep the app design
    • and code simple
  • 12.
    • #2
    • Let the app users
    • customize the behavior
    • (without hacking the internals)
  • 13.
    • #3
    • It's fun to write plugins
    • for most hackers
    • (see: Plagger and Kwiki)
  • 14.
    • "Can your app do XXX?"
    • "Yes, by plugins."
  • 15.
    • "Your app has a bug in YYY"
    • "No, it's the bug in plugin YYY,
    • Not my fault."
    • (Chain Of Responsibilities)
  • 16.
    • Good Enough
    • Reasons, huh?
  • 17.
    • #1
    • Make your app pluggable
  • 18.
    • Example
  • 19.
    • ack (App::Ack)
  • 20. grep –r for programmers
  • 21.
    • Ack is a "full-stack"
    • software now.
  • 22.
    • By "full-stack" I mean:
    • Easy install
    • No configuration
    • No way to extend
  • 23.
    • Specifically:
    • These are hardcoded
    • Ignored directories
    • Filenames and types
  • 24. Ignored Directories
    • @ignore_dirs = qw( blib CVS RCS SCCS .svn
    • _darcs .git );
  • 25. Filenames and languages mapping
    • %mappings = (
    • asm => [qw( s S )],
    • binary => …,
    • cc => [qw( c h xs )],
    • cpp => [qw( cpp m h C H )],
    • csharp => [qw( cs )],
    • perl => [qw( pl pm pod tt ttml t )],
    • );
  • 26.
    • What if making
    • these pluggable?
  • 27.
    • DISCLAIMER
  • 28.
    • Don't get me wrong Andy,
    • I love ack the way it is…
    • Just thought it can be
    • a very good example for the tutorial.
  • 29.
    • Quickstart:
    • Class::Trigger Module::Pluggable
    • © Six Apart Ltd. Employees
  • 30. Class::Trigger SYNOPSIS
    • package Foo;
    • use Class::Trigger;
    • sub foo {
    • my $self = shift;
    • $self->call_trigger('before_foo');
    • # some code ...
    • $self->call_trigger('after_foo');
    • }
    • package main;
    • Foo->add_trigger(before_foo => &sub1);
    • Foo->add_trigger(after_foo => &sub2);
  • 31.
    • Class::Trigger
    • Helps you to implement
    • Observer Pattern.
    • (Rails calls this Observer)
  • 32. Module::Pluggable SYNOPSIS
    • package MyClass;
    • use Module::Pluggable;
    • use MyClass;
    • my $mc = MyClass->new();
    • # returns the names of all plugins installed under MyClass::Plugin::*
    • my @plugins = $mc->plugins();
    • package MyClass::Plugin::Foo;
    • sub new { … }
    • 1;
  • 33. Setup plugins in App::Ack
    • package App::Ack;
    • use Class::Trigger;
    • use Module::Pluggable require => 1;
    • __PACKAGE__->plugins;
  • 34. Setup plugins in App::Ack
    • package App::Ack;
    • use Class::Trigger;
    • use Module::Pluggable require => 1 ;
    • __PACKAGE__->plugins; # "requires" modules
  • 35. Ignored Directories (Before)
    • @ignore_dirs = qw( blib CVS RCS SCCS .svn
    • _darcs .git );
  • 36. Ignored Directories (After)
    • # lib/App/Ack.pm
    • __PACKAGE__->call_trigger(' ignore_dirs.add ', @ignore_dirs);
  • 37. Ignored Directories (plugins)
    • # lib/App/Ack/Plugin/IgnorePerlBuildDir.pm
    • package App::Ack::Plugin::IgnorePerlBuildDir;
    • App::Ack->add_trigger(
    • " ignore_dirs.add " => sub {
    • my($class, $ignore_dirs) = @_;
    • push @$ignore_dirs, qw( blib );
    • },
    • );
    • 1;
  • 38. Ignored Directories (plugins)
    • # lib/App/Ack/Plugin/IgnoreSourceControlDir.pm
    • package App::Ack::Plugin::IgnoreSourcdeControlDir;
    • App::Ack->add_trigger(
    • " ignore_dirs.add " => sub {
    • my($class, $ignore_dirs) = @_;
    • push @$ignore_dirs, qw( CVS RCS .svn
    • _darcs .git );
    • },
    • );
    • 1;
  • 39. Filenames and languages (before)
    • %mappings = (
    • asm => [qw( s S )],
    • binary => …,
    • cc => [qw( c h xs )],
    • cpp => [qw( cpp m h C H )],
    • csharp => [qw( cs )],
    • perl => [qw( pl pm pod tt ttml t )],
    • );
  • 40. Filenames and languages (after)
    • # lib/App/Ack.pm
    • __PACKAGE__->call_trigger('mappings.add', mappings);
  • 41. Filenames and languages (plugins)
    • package App::Ack::Plugin::MappingCFamily;
    • use strict;
    • App::Ack->add_trigger(
    • "mappings.add" => sub {
    • my($class, $mappings) = @_;
    • $mappings->{asm} = [qw( s S )];
    • $mappings->{cc} = [qw( c h xs )];
    • $mappings->{cpp} = [qw( cpp m h C H )];
    • $mappings->{csharp} = [qw( cs )];
    • $mappings->{css} = [qw( css )];
    • },
    • );
    • 1;
  • 42.
    • Works great
    • with few lines of code!
  • 43.
    • Now it's time to add
    • Some useful stuff.
  • 44.
    • Example Plugin:
    • Content Filter
  • 45.
    • sub _search {
    • my $fh = shift;
    • my $is_binary = shift;
    • my $filename = shift;
    • my $regex = shift;
    • my %opt = @_;
    • if ($is_binary) {
    • my $new_fh;
    • App::Ack->call_trigger('filter.binary', $filename, $new_fh);
    • if ($new_fh) {
    • return _search($new_fh, 0, $filename, $regex, @_);
    • }
    • }
  • 46.
    • Example:
    • Search PDF content
    • with ack
  • 47. PDF filter plugin
    • package App::Ack::Plugin::ExtractContentPDF;
    • use strict;
    • use CAM::PDF;
    • use File::Temp;
    • App::Ack->add_trigger(
    • 'mappings.add' => sub {
    • my($class, $mappings) = @_;
    • $mappings->{pdf} = [qw(pdf)];
    • },
    • );
  • 48. PDF filter plugin (cont.)
    • App::Ack->add_trigger(
    • 'filter.binary' => sub {
    • my($class, $filename, $fh_ref) = @_;
    • if ($filename =~ /.pdf$/) {
    • my $fh = File::Temp::tempfile;
    • my $doc = CAM::PDF->new($file);
    • my $text;
    • for my $page (1..$doc->numPages){
    • $text .= $doc->getPageText($page);
    • }
    • print $fh $text;
    • seek $$fh, 0, 0;
    • $$fh_ref = $fh;
    • }
    • },
    • );
  • 49. PDF search
    • > ack --type=pdf Audrey
    • yapcasia2007-pugs.pdf:3:Audrey Tang
  • 50.
    • Homework
    • Use File::Extract
    • To handle arbitrary media files
  • 51.
    • Homework 2:
    • Search non UTF-8 files
    • (hint: use Encode::Guess)
    • You'll need another hook.
  • 52. Summary
    • Class::Trigger
    • + Module::Pluggable
    • = Pluggable app easy
  • 53.
    • #2
    • TMTOWTDP
    • There's More Than One Way To Deploy Plugins
  • 54.
    • Module::Pluggable
    • + Class::Trigger
    • = Simple and Nice
    • but has limitations
  • 55.
    • In Reality,
    • we need more control over how plugins behave
  • 56.
    • 1)
    • The order of
    • plugin executions
  • 57.
    • 2)
    • Per user configurations
    • for plugins
  • 58.
    • 3)
    • Temporarily
    • Disable plugins
    • Should be easy
  • 59.
    • 4)
    • How to install
    • & upgrade plugins
  • 60.
    • 5)
    • Let plugins
    • have storage area
  • 61.
    • Etc, etc.
  • 62.
    • Examples:
    • Kwiki
    • Plagger
    • qpsmtpd
    • Movable Type
  • 63.
    • I won't talk about
    • Catalyst plugins
    • (and other framework thingy)
  • 64.
    • Because they're
    • NOT
    • "plug-ins"
  • 65.
    • Install plugins
    • And now you write
    • MORE CODE
  • 66.
    • 95% of Catalyst plugins
    • Are NOT "plugins"
    • But "components"
    • 95% of these statistics is made up by the speakers.
  • 67.
    • Kwiki 1.0
  • 68. Kwiki Plugin code
    • package Kwiki::URLBL;
    • use Kwiki::Plugin -Base ;
    • use Kwiki::Installer -base;
    • const class_id => 'urlbl';
    • const class_title => 'URL Blacklist DNS';
    • const config_file => 'urlbl.yaml';
    • sub register {
    • require URI::Find;
    • my $registry = shift;
    • $registry->add(hook => 'edit:save', pre => 'urlbl_hook');
    • $registry->add(action => 'blacklisted_url');
    • }
  • 69. Kwiki Plugin (cont.)
    • sub urlbl_hook {
    • my $hook = pop;
    • my $old_page = $self->hub->pages->new_page($self->pages->current->id);
    • my $this = $self->hub->urlbl;
    • my @old_urls = $this->get_urls($old_page->content);
    • my @urls = $this->get_urls($self->cgi->page_content);
    • my @new_urls = $this->get_new_urls(@old_urls, @urls);
    • if (@new_urls && $this->is_blocked(@new_urls)) {
    • $hook->cancel();
    • return $self->redirect("action=blacklisted_url");
    • }
    • }
  • 70.
    • Magic implemented
    • in Spoon(::Hooks)
  • 71. "Install" Kwiki Plugins
    • # order doesn't matter here (according to Ingy)
    • Kwiki::Display
    • Kwiki::Edit
    • Kwiki::Theme::Basic
    • Kwiki::Toolbar
    • Kwiki::Status
    • Kwiki::Widgets
    • # Comment out (or entirely remove) to disable
    • # Kwiki::UnnecessaryStuff
  • 72. Kwiki plugin config
    • # in Kwiki::URLBL plugin
    • __config/urlbl.yaml__
    • urlbl_dns: sc.surbl.org, bsb.spamlookup.net, rbl.bulkfeeds.jp
    • # config.yaml
    • urlbl_dns: myowndns.example.org
  • 73. Kwiki plugins are CPAN modules
  • 74. Install and Upgrade plugins
    • cpan> install Kwiki::SomeStuff
  • 75. Using CPAN as a repository
    • Pros #1:
    • reuse most of current CPAN infrastructure.
  • 76. Using CPAN as a repository
    • Pros #2:
    • Increasing # of modules
    • = good motivation
    • for Perl hackers
  • 77.
    • Cons #1:
    • Installing CPAN deps
    • could be a mess
    • (especially for Win32)
  • 78.
    • Cons #2:
    • Whenever Ingy releases
    • new Kwiki, lots of plugins just break.
  • 79. Kwiki plugin storage
    • return if
    • grep {$page->id}
    • @{$self-> config->cached_display_ignore };
    • my $html = io->catfile(
    • $self-> plugin_directory ,$page->id
    • )->utf8;
  • 80.
    • Kwiki 2.0
  • 81.
    • Same as Kwiki 1.0
  • 82.
    • Except:
    • plugins are now
    • in SVN repository
  • 83.  
  • 84. Plagger plugin
    • package Plagger::Plugin::Publish::iCal;
    • use strict;
    • use base qw( Plagger::Plugin );
    • use Data::ICal;
    • use Data::ICal::Entry::Event;
    • use DateTime::Duration;
    • use DateTime::Format::ICal;
    • sub register {
    • my($self, $context) = @_;
    • $context-> register_hook (
    • $self,
    • ' publish.feed ' => &publish_feed,
    • ' plugin.init ' => &plugin_init,
    • );
    • }
  • 85. Plagger plugin (cont)
    • sub plugin_init {
    • my($self, $context) = @_;
    • my $dir = $self->conf->{dir};
    • unless (-e $dir && -d _) {
    • mkdir $dir, 0755
    • or $context->error("Failed to mkdir $dir: $!");
    • }
    • }
  • 86. Plagger plugin storage
    • $self->conf->{invindex} ||=
    • $self-> cache->path_to ('invindex');
  • 87. Plagger plugin config
    • # The order matters in config.yaml
    • # if they're in the same hooks
    • plugins:
    • - module: Subscription::Config
    • config:
    • feed:
    • - http://www.example.com/
    • - module: Filter::DegradeYouTube
    • config:
    • dev_id: XYZXYZ
    • - module: Publish::Gmail
    • disable: 1
  • 88. Plugins Install & Upgrade
    • cpan> notest install Plagger
    • # or …
    • > svn co http://…/plagger/trunk plagger
    • > svn update
  • 89.
    • Plagger impl.
    • ripped off by
    • many apps now
  • 90.
    • qpsmtpd
  • 91.
    • mod_perl for SMTP
    • Runs with tcpserver, forkserver
    • or Danga::Socket standalone
  • 92. Plugins: Flat files
    • rock:/home/miyagawa/svn/qpsmtpd> ls -F plugins
    • async/ greylisting
    • auth/ hosts_allow
    • check_badmailfrom http_config
    • check_badmailfromto ident/
    • check_badrcptto logging/
    • check_badrcptto_patterns milter
    • check_basicheaders parse_addr_withhelo
    • check_earlytalker queue/
    • check_loop quit_fortune
    • check_norelay rcpt_ok
    • check_relay relay_only
    • check_spamhelo require_resolvable_fromhost
    • content_log rhsbl
    • count_unrecognized_commands sender_permitted_from
    • dns_whitelist_soft spamassassin
    • dnsbl tls
    • domainkeys tls_cert*
    • dont_require_anglebrackets virus/
  • 93. qpsmtpd plugin
    • sub hook_mail {
    • my ($self, $transaction, $sender, %param) = @_;
    • my @badmailfrom = $self->qp->config ("badmailfrom")
    • or return (DECLINED);
    • for my $bad (@badmailfrom) {
    • my $reason = $bad;
    • $bad =~ s/^s*(S+).*/$1/;
    • next unless $bad;
    • $transaction->notes('badmailfrom', $reason) …
    • }
    • return (DECLINED);
    • }
  • 94.
    • Actually qpsmtpd
    • Plugins are "compiled"
    • to modules
  • 95.
    • my $eval = join(" ",
    • "package $package;",
    • 'use Qpsmtpd::Constants;',
    • "require Qpsmtpd::Plugin;",
    • 'use vars qw(@ISA);',
    • 'use strict;',
    • '@ISA = qw(Qpsmtpd::Plugin);',
    • ($test_mode ? 'use Test::More;' : ''),
    • "sub plugin_name { qq[$plugin] }",
    • $line,
    • $sub,
    • " ", # last line comment without newline?
    • );
    • $eval =~ m/(.*)/s;
    • $eval = $1;
    • eval $eval;
    • die "eval $@" if $@;
  • 96. qpsmtpd plugin config
    • rock:/home/miyagawa/svn/qpsmtpd> ls config.sample/
    • config.sample:
    • IP logging require_resolvable_fromhost
    • badhelo loglevel rhsbl_zones
    • badrcptto_patterns plugins size_threshold
    • dnsbl_zones rcpthosts tls_before_auth
    • invalid_resolvable_fromhost relayclients tls_ciphers
  • 97. config/plugins
    • # content filters
    • virus/klez_filter
    • # rejects mails with a SA score higher than 2
    • spamassassin reject_threshold 20
  • 98. config/badhelo
    • # these domains never uses their domain when greeting us, so reject transactions
    • aol.com
    • yahoo.com
  • 99. Install & Upgrade plugins
    • Just use subversion
  • 100.  
  • 101.
    • MT plugins are
    • flat-files
    • (or scripts that call modules)
  • 102. MT plugin code
    • package MT::Plugin::BanASCII;
    • our $Method = "deny";
    • use MT;
    • use MT::Plugin;
    • my $plugin = MT::Plugin->new({
    • name => "BanASCII v$VERSION",
    • description => "Deny or moderate ASCII or Latin-1 comment",
    • });
    • MT->add_plugin($plugin);
    • MT->add_callback('CommentFilter', 2, $plugin, &handler);
  • 103. MT plugin code (cont)
    • sub init_app {
    • my $plugin = shift;
    • $plugin->SUPER::init_app(@_);
    • my($app) = @_;
    • return unless $app->isa('MT::App::CMS');
    • $app-> add_itemset_action ({
    • type => 'comment',
    • key => 'spam_submission_comment',
    • label => 'Report SPAM Comment(s)',
    • code => sub {
    • $plugin->submit_spams_action('MT::Comment', @_) },
    • }
    • );
  • 104.  
  • 105.  
  • 106. MT plugin storage
    • require MT::PluginData;
    • my $data = MT::PluginData->load({
    • plugin => 'sidebar-manager',
    • key => $blog_id },
    • );
    • unless ($data) {
    • $data = MT::PluginData->new;
    • $data->plugin('sidebar-manager');
    • $data->key($blog_id);
    • }
    • $data->data( $modulesets );
    • $data->save or die $data->errstr;
  • 107. Order control
    • MT->add_callback('CMSPostEntrySave', 9 , $rightfields, &CMSPostEntrySave);
    • MT->add_callback('CMSPreSave_entry', 9 , $rightfields, &CMSPreSave_entry);
    • MT::Entry->add_callback('pre_remove', 9 , $rightfields, &entry_pre_remove);
    Defined in plugins. No Control on users end
  • 108. Conclusion
    • Flat-files
    • vs.
    • Modules
  • 109.
    • Flat-files:
    • ☺ Easy to install (Just grab it)
    • ☻ Hard to upgrade
    • OK for simple plugins
  • 110.
    • Modules:
    • ☺ Full-access to Perl OO goodness
    • ☺ Avoid duplicate efforts of CPAN
    • ☻ Might be hard to resolve deps.
    • Subversion to the rescue
    • (could be a barrier for newbies)
  • 111.
    • Nice-to-haves:
    • Order control
    • Temporarily disable plugins
    • Per plugin config
    • Per plugin storage
  • 112. Resources
    • Class::Trigger
    • http:// search.cpan.org /dist/Class-Trigger/
    • Module::Pluggable
    • http:// search.cpan.org /dist/Module-Pluggable/
    • Ask Bjorn Hansen: Build Easily Extensible Perl Programs
    • http://conferences.oreillynet.com/cs/os2005/view/e_sess/6806
    • qpsmtpd
    • http:// smtpd.develooper.com /
    • MT plugins
    • http:// www.sixapart.com/pronet/plugins /
    • Kwiki
    • http:// www.kwiki.org /
    • Plagger
    • http:// plagger.org /