• Save
Writing Pluggable Software
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Writing Pluggable Software

on

  • 21,406 views

"How to write pluggable software" presented by Tatsuhiko Miyagawa at YAPC::Asia 2007 in Tokyo on April 5th 2007.

"How to write pluggable software" presented by Tatsuhiko Miyagawa at YAPC::Asia 2007 in Tokyo on April 5th 2007.

Statistics

Views

Total Views
21,406
Views on SlideShare
21,354
Embed Views
52

Actions

Likes
7
Downloads
0
Comments
0

5 Embeds 52

http://techvideos.humourbox.info 33
http://www.slideshare.net 13
http://www.linkedin.com 3
http://lj-toys.com 2
http://l.lj-toys.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Writing Pluggable Software Presentation Transcript

  • 1. Writing Pluggable Software Tatsuhiko Miyagawa [email_address] Six Apart, Ltd. / Shibuya Perl Mongers YAPC::Asia 2007 Tokyo
  • 2. For non-JP attendees …
    • If you find in the code,
    • Replace that with backslash.
    • (This is MS' fault)
  • 3.
    • Plaggable
    • Software
  • 4.
    • Plaggable
    • Software
  • 5.
    • Pl u ggable
    • Software
  • 6.
    • Agenda
  • 7.
    • #1
    • How to make
    • your app pluggable
  • 8.
    • #2
    • TMTOWTDP
    • There's More Than One Way To Deploy Plugins
    • Pros/Cons by examples
  • 9.
    • First-of-all:
    • Why pluggable?
  • 10.
    • Benefits
  • 11.
    • #1
    • Keep the app design
    • and code simple
  • 12.
    • #2
    • Let the app users
    • customize the behavior
    • (without hacking the internals)
  • 13.
    • #3
    • It's fun to write plugins
    • for most hackers
    • (see: Plagger and Kwiki)
  • 14.
    • "Can your app do XXX?"
    • "Yes, by plugins."
  • 15.
    • "Your app has a bug in YYY"
    • "No, it's the bug in plugin YYY,
    • Not my fault."
    • (Chain Of Responsibilities)
  • 16.
    • Good Enough
    • Reasons, huh?
  • 17.
    • #1
    • Make your app pluggable
  • 18.
    • Example
  • 19.
    • ack (App::Ack)
  • 20. grep –r for programmers
  • 21.
    • Ack is a "full-stack"
    • software now.
  • 22.
    • By "full-stack" I mean:
    • Easy install
    • No configuration
    • No way to extend
  • 23.
    • Specifically:
    • These are hardcoded
    • Ignored directories
    • Filenames and types
  • 24. Ignored Directories
    • @ignore_dirs = qw( blib CVS RCS SCCS .svn
    • _darcs .git );
  • 25. Filenames and languages mapping
    • %mappings = (
    • asm => [qw( s S )],
    • binary => …,
    • cc => [qw( c h xs )],
    • cpp => [qw( cpp m h C H )],
    • csharp => [qw( cs )],
    • perl => [qw( pl pm pod tt ttml t )],
    • );
  • 26.
    • What if making
    • these pluggable?
  • 27.
    • DISCLAIMER
  • 28.
    • Don't get me wrong Andy,
    • I love ack the way it is…
    • Just thought it can be
    • a very good example for the tutorial.
  • 29.
    • Quickstart:
    • Class::Trigger Module::Pluggable
    • © Six Apart Ltd. Employees
  • 30. Class::Trigger SYNOPSIS
    • package Foo;
    • use Class::Trigger;
    • sub foo {
    • my $self = shift;
    • $self->call_trigger('before_foo');
    • # some code ...
    • $self->call_trigger('after_foo');
    • }
    • package main;
    • Foo->add_trigger(before_foo => &sub1);
    • Foo->add_trigger(after_foo => &sub2);
  • 31.
    • Class::Trigger
    • Helps you to implement
    • Observer Pattern.
    • (Rails calls this Observer)
  • 32. Module::Pluggable SYNOPSIS
    • package MyClass;
    • use Module::Pluggable;
    • use MyClass;
    • my $mc = MyClass->new();
    • # returns the names of all plugins installed under MyClass::Plugin::*
    • my @plugins = $mc->plugins();
    • package MyClass::Plugin::Foo;
    • sub new { … }
    • 1;
  • 33. Setup plugins in App::Ack
    • package App::Ack;
    • use Class::Trigger;
    • use Module::Pluggable require => 1;
    • __PACKAGE__->plugins;
  • 34. Setup plugins in App::Ack
    • package App::Ack;
    • use Class::Trigger;
    • use Module::Pluggable require => 1 ;
    • __PACKAGE__->plugins; # "requires" modules
  • 35. Ignored Directories (Before)
    • @ignore_dirs = qw( blib CVS RCS SCCS .svn
    • _darcs .git );
  • 36. Ignored Directories (After)
    • # lib/App/Ack.pm
    • __PACKAGE__->call_trigger(' ignore_dirs.add ', @ignore_dirs);
  • 37. Ignored Directories (plugins)
    • # lib/App/Ack/Plugin/IgnorePerlBuildDir.pm
    • package App::Ack::Plugin::IgnorePerlBuildDir;
    • App::Ack->add_trigger(
    • " ignore_dirs.add " => sub {
    • my($class, $ignore_dirs) = @_;
    • push @$ignore_dirs, qw( blib );
    • },
    • );
    • 1;
  • 38. Ignored Directories (plugins)
    • # lib/App/Ack/Plugin/IgnoreSourceControlDir.pm
    • package App::Ack::Plugin::IgnoreSourcdeControlDir;
    • App::Ack->add_trigger(
    • " ignore_dirs.add " => sub {
    • my($class, $ignore_dirs) = @_;
    • push @$ignore_dirs, qw( CVS RCS .svn
    • _darcs .git );
    • },
    • );
    • 1;
  • 39. Filenames and languages (before)
    • %mappings = (
    • asm => [qw( s S )],
    • binary => …,
    • cc => [qw( c h xs )],
    • cpp => [qw( cpp m h C H )],
    • csharp => [qw( cs )],
    • perl => [qw( pl pm pod tt ttml t )],
    • );
  • 40. Filenames and languages (after)
    • # lib/App/Ack.pm
    • __PACKAGE__->call_trigger('mappings.add', mappings);
  • 41. Filenames and languages (plugins)
    • package App::Ack::Plugin::MappingCFamily;
    • use strict;
    • App::Ack->add_trigger(
    • "mappings.add" => sub {
    • my($class, $mappings) = @_;
    • $mappings->{asm} = [qw( s S )];
    • $mappings->{cc} = [qw( c h xs )];
    • $mappings->{cpp} = [qw( cpp m h C H )];
    • $mappings->{csharp} = [qw( cs )];
    • $mappings->{css} = [qw( css )];
    • },
    • );
    • 1;
  • 42.
    • Works great
    • with few lines of code!
  • 43.
    • Now it's time to add
    • Some useful stuff.
  • 44.
    • Example Plugin:
    • Content Filter
  • 45.
    • sub _search {
    • my $fh = shift;
    • my $is_binary = shift;
    • my $filename = shift;
    • my $regex = shift;
    • my %opt = @_;
    • if ($is_binary) {
    • my $new_fh;
    • App::Ack->call_trigger('filter.binary', $filename, $new_fh);
    • if ($new_fh) {
    • return _search($new_fh, 0, $filename, $regex, @_);
    • }
    • }
  • 46.
    • Example:
    • Search PDF content
    • with ack
  • 47. PDF filter plugin
    • package App::Ack::Plugin::ExtractContentPDF;
    • use strict;
    • use CAM::PDF;
    • use File::Temp;
    • App::Ack->add_trigger(
    • 'mappings.add' => sub {
    • my($class, $mappings) = @_;
    • $mappings->{pdf} = [qw(pdf)];
    • },
    • );
  • 48. PDF filter plugin (cont.)
    • App::Ack->add_trigger(
    • 'filter.binary' => sub {
    • my($class, $filename, $fh_ref) = @_;
    • if ($filename =~ /.pdf$/) {
    • my $fh = File::Temp::tempfile;
    • my $doc = CAM::PDF->new($file);
    • my $text;
    • for my $page (1..$doc->numPages){
    • $text .= $doc->getPageText($page);
    • }
    • print $fh $text;
    • seek $$fh, 0, 0;
    • $$fh_ref = $fh;
    • }
    • },
    • );
  • 49. PDF search
    • > ack --type=pdf Audrey
    • yapcasia2007-pugs.pdf:3:Audrey Tang
  • 50.
    • Homework
    • Use File::Extract
    • To handle arbitrary media files
  • 51.
    • Homework 2:
    • Search non UTF-8 files
    • (hint: use Encode::Guess)
    • You'll need another hook.
  • 52. Summary
    • Class::Trigger
    • + Module::Pluggable
    • = Pluggable app easy
  • 53.
    • #2
    • TMTOWTDP
    • There's More Than One Way To Deploy Plugins
  • 54.
    • Module::Pluggable
    • + Class::Trigger
    • = Simple and Nice
    • but has limitations
  • 55.
    • In Reality,
    • we need more control over how plugins behave
  • 56.
    • 1)
    • The order of
    • plugin executions
  • 57.
    • 2)
    • Per user configurations
    • for plugins
  • 58.
    • 3)
    • Temporarily
    • Disable plugins
    • Should be easy
  • 59.
    • 4)
    • How to install
    • & upgrade plugins
  • 60.
    • 5)
    • Let plugins
    • have storage area
  • 61.
    • Etc, etc.
  • 62.
    • Examples:
    • Kwiki
    • Plagger
    • qpsmtpd
    • Movable Type
  • 63.
    • I won't talk about
    • Catalyst plugins
    • (and other framework thingy)
  • 64.
    • Because they're
    • NOT
    • "plug-ins"
  • 65.
    • Install plugins
    • And now you write
    • MORE CODE
  • 66.
    • 95% of Catalyst plugins
    • Are NOT "plugins"
    • But "components"
    • 95% of these statistics is made up by the speakers.
  • 67.
    • Kwiki 1.0
  • 68. Kwiki Plugin code
    • package Kwiki::URLBL;
    • use Kwiki::Plugin -Base ;
    • use Kwiki::Installer -base;
    • const class_id => 'urlbl';
    • const class_title => 'URL Blacklist DNS';
    • const config_file => 'urlbl.yaml';
    • sub register {
    • require URI::Find;
    • my $registry = shift;
    • $registry->add(hook => 'edit:save', pre => 'urlbl_hook');
    • $registry->add(action => 'blacklisted_url');
    • }
  • 69. Kwiki Plugin (cont.)
    • sub urlbl_hook {
    • my $hook = pop;
    • my $old_page = $self->hub->pages->new_page($self->pages->current->id);
    • my $this = $self->hub->urlbl;
    • my @old_urls = $this->get_urls($old_page->content);
    • my @urls = $this->get_urls($self->cgi->page_content);
    • my @new_urls = $this->get_new_urls(@old_urls, @urls);
    • if (@new_urls && $this->is_blocked(@new_urls)) {
    • $hook->cancel();
    • return $self->redirect("action=blacklisted_url");
    • }
    • }
  • 70.
    • Magic implemented
    • in Spoon(::Hooks)
  • 71. "Install" Kwiki Plugins
    • # order doesn't matter here (according to Ingy)
    • Kwiki::Display
    • Kwiki::Edit
    • Kwiki::Theme::Basic
    • Kwiki::Toolbar
    • Kwiki::Status
    • Kwiki::Widgets
    • # Comment out (or entirely remove) to disable
    • # Kwiki::UnnecessaryStuff
  • 72. Kwiki plugin config
    • # in Kwiki::URLBL plugin
    • __config/urlbl.yaml__
    • urlbl_dns: sc.surbl.org, bsb.spamlookup.net, rbl.bulkfeeds.jp
    • # config.yaml
    • urlbl_dns: myowndns.example.org
  • 73. Kwiki plugins are CPAN modules
  • 74. Install and Upgrade plugins
    • cpan> install Kwiki::SomeStuff
  • 75. Using CPAN as a repository
    • Pros #1:
    • reuse most of current CPAN infrastructure.
  • 76. Using CPAN as a repository
    • Pros #2:
    • Increasing # of modules
    • = good motivation
    • for Perl hackers
  • 77.
    • Cons #1:
    • Installing CPAN deps
    • could be a mess
    • (especially for Win32)
  • 78.
    • Cons #2:
    • Whenever Ingy releases
    • new Kwiki, lots of plugins just break.
  • 79. Kwiki plugin storage
    • return if
    • grep {$page->id}
    • @{$self-> config->cached_display_ignore };
    • my $html = io->catfile(
    • $self-> plugin_directory ,$page->id
    • )->utf8;
  • 80.
    • Kwiki 2.0
  • 81.
    • Same as Kwiki 1.0
  • 82.
    • Except:
    • plugins are now
    • in SVN repository
  • 83.  
  • 84. Plagger plugin
    • package Plagger::Plugin::Publish::iCal;
    • use strict;
    • use base qw( Plagger::Plugin );
    • use Data::ICal;
    • use Data::ICal::Entry::Event;
    • use DateTime::Duration;
    • use DateTime::Format::ICal;
    • sub register {
    • my($self, $context) = @_;
    • $context-> register_hook (
    • $self,
    • ' publish.feed ' => &publish_feed,
    • ' plugin.init ' => &plugin_init,
    • );
    • }
  • 85. Plagger plugin (cont)
    • sub plugin_init {
    • my($self, $context) = @_;
    • my $dir = $self->conf->{dir};
    • unless (-e $dir && -d _) {
    • mkdir $dir, 0755
    • or $context->error("Failed to mkdir $dir: $!");
    • }
    • }
  • 86. Plagger plugin storage
    • $self->conf->{invindex} ||=
    • $self-> cache->path_to ('invindex');
  • 87. Plagger plugin config
    • # The order matters in config.yaml
    • # if they're in the same hooks
    • plugins:
    • - module: Subscription::Config
    • config:
    • feed:
    • - http://www.example.com/
    • - module: Filter::DegradeYouTube
    • config:
    • dev_id: XYZXYZ
    • - module: Publish::Gmail
    • disable: 1
  • 88. Plugins Install & Upgrade
    • cpan> notest install Plagger
    • # or …
    • > svn co http://…/plagger/trunk plagger
    • > svn update
  • 89.
    • Plagger impl.
    • ripped off by
    • many apps now
  • 90.
    • qpsmtpd
  • 91.
    • mod_perl for SMTP
    • Runs with tcpserver, forkserver
    • or Danga::Socket standalone
  • 92. Plugins: Flat files
    • rock:/home/miyagawa/svn/qpsmtpd> ls -F plugins
    • async/ greylisting
    • auth/ hosts_allow
    • check_badmailfrom http_config
    • check_badmailfromto ident/
    • check_badrcptto logging/
    • check_badrcptto_patterns milter
    • check_basicheaders parse_addr_withhelo
    • check_earlytalker queue/
    • check_loop quit_fortune
    • check_norelay rcpt_ok
    • check_relay relay_only
    • check_spamhelo require_resolvable_fromhost
    • content_log rhsbl
    • count_unrecognized_commands sender_permitted_from
    • dns_whitelist_soft spamassassin
    • dnsbl tls
    • domainkeys tls_cert*
    • dont_require_anglebrackets virus/
  • 93. qpsmtpd plugin
    • sub hook_mail {
    • my ($self, $transaction, $sender, %param) = @_;
    • my @badmailfrom = $self->qp->config ("badmailfrom")
    • or return (DECLINED);
    • for my $bad (@badmailfrom) {
    • my $reason = $bad;
    • $bad =~ s/^s*(S+).*/$1/;
    • next unless $bad;
    • $transaction->notes('badmailfrom', $reason) …
    • }
    • return (DECLINED);
    • }
  • 94.
    • Actually qpsmtpd
    • Plugins are "compiled"
    • to modules
  • 95.
    • my $eval = join(" ",
    • "package $package;",
    • 'use Qpsmtpd::Constants;',
    • "require Qpsmtpd::Plugin;",
    • 'use vars qw(@ISA);',
    • 'use strict;',
    • '@ISA = qw(Qpsmtpd::Plugin);',
    • ($test_mode ? 'use Test::More;' : ''),
    • "sub plugin_name { qq[$plugin] }",
    • $line,
    • $sub,
    • " ", # last line comment without newline?
    • );
    • $eval =~ m/(.*)/s;
    • $eval = $1;
    • eval $eval;
    • die "eval $@" if $@;
  • 96. qpsmtpd plugin config
    • rock:/home/miyagawa/svn/qpsmtpd> ls config.sample/
    • config.sample:
    • IP logging require_resolvable_fromhost
    • badhelo loglevel rhsbl_zones
    • badrcptto_patterns plugins size_threshold
    • dnsbl_zones rcpthosts tls_before_auth
    • invalid_resolvable_fromhost relayclients tls_ciphers
  • 97. config/plugins
    • # content filters
    • virus/klez_filter
    • # rejects mails with a SA score higher than 2
    • spamassassin reject_threshold 20
  • 98. config/badhelo
    • # these domains never uses their domain when greeting us, so reject transactions
    • aol.com
    • yahoo.com
  • 99. Install & Upgrade plugins
    • Just use subversion
  • 100.  
  • 101.
    • MT plugins are
    • flat-files
    • (or scripts that call modules)
  • 102. MT plugin code
    • package MT::Plugin::BanASCII;
    • our $Method = "deny";
    • use MT;
    • use MT::Plugin;
    • my $plugin = MT::Plugin->new({
    • name => "BanASCII v$VERSION",
    • description => "Deny or moderate ASCII or Latin-1 comment",
    • });
    • MT->add_plugin($plugin);
    • MT->add_callback('CommentFilter', 2, $plugin, &handler);
  • 103. MT plugin code (cont)
    • sub init_app {
    • my $plugin = shift;
    • $plugin->SUPER::init_app(@_);
    • my($app) = @_;
    • return unless $app->isa('MT::App::CMS');
    • $app-> add_itemset_action ({
    • type => 'comment',
    • key => 'spam_submission_comment',
    • label => 'Report SPAM Comment(s)',
    • code => sub {
    • $plugin->submit_spams_action('MT::Comment', @_) },
    • }
    • );
  • 104.  
  • 105.  
  • 106. MT plugin storage
    • require MT::PluginData;
    • my $data = MT::PluginData->load({
    • plugin => 'sidebar-manager',
    • key => $blog_id },
    • );
    • unless ($data) {
    • $data = MT::PluginData->new;
    • $data->plugin('sidebar-manager');
    • $data->key($blog_id);
    • }
    • $data->data( $modulesets );
    • $data->save or die $data->errstr;
  • 107. Order control
    • MT->add_callback('CMSPostEntrySave', 9 , $rightfields, &CMSPostEntrySave);
    • MT->add_callback('CMSPreSave_entry', 9 , $rightfields, &CMSPreSave_entry);
    • MT::Entry->add_callback('pre_remove', 9 , $rightfields, &entry_pre_remove);
    Defined in plugins. No Control on users end
  • 108. Conclusion
    • Flat-files
    • vs.
    • Modules
  • 109.
    • Flat-files:
    • ☺ Easy to install (Just grab it)
    • ☻ Hard to upgrade
    • OK for simple plugins
  • 110.
    • Modules:
    • ☺ Full-access to Perl OO goodness
    • ☺ Avoid duplicate efforts of CPAN
    • ☻ Might be hard to resolve deps.
    • Subversion to the rescue
    • (could be a barrier for newbies)
  • 111.
    • Nice-to-haves:
    • Order control
    • Temporarily disable plugins
    • Per plugin config
    • Per plugin storage
  • 112. Resources
    • Class::Trigger
    • http:// search.cpan.org /dist/Class-Trigger/
    • Module::Pluggable
    • http:// search.cpan.org /dist/Module-Pluggable/
    • Ask Bjorn Hansen: Build Easily Extensible Perl Programs
    • http://conferences.oreillynet.com/cs/os2005/view/e_sess/6806
    • qpsmtpd
    • http:// smtpd.develooper.com /
    • MT plugins
    • http:// www.sixapart.com/pronet/plugins /
    • Kwiki
    • http:// www.kwiki.org /
    • Plagger
    • http:// plagger.org /