Friends, this is clean-up time and we're
            discounting all our silent, electric Ubiks
               by this much money. Yes' we're throwing
           away the bluebook. And remember: every Ubik
            on our lot has been used only as directed.


                              Phillip K. Dick. “Ubik”.




        Ubic
Polymorphic service manager
  github.com/berekuk/ubic
1-minute intro
Install

                     Debian/Ubuntu:
apt-add-repository ppa:berekuk/ubic && apt-get install ubic

                        FreeBSD:
      cd /usr/ports/sysutils/p5-Ubic && make install

                   Everywhere else:
            cpanm Ubic && ubic-admin setup
Write service config

  # cat >/etc/ubic/service/foo

  use Ubic::Service::SimpleDaemon;

  Ubic::Service::SimpleDaemon->new(
      bin => ‘sleep 1000’,
      stdout => ‘/var/log/foo.log’,
  );
Run it


# ubic start foo
Starting foo... started (pid 28152)
Flexible perl-based
 service manager
Flexible perl-based
   polymorphic
 service manager
APIs everywhere
I’ll talk about APIs
instead of features today

• Ubic.pm glue code
• services
• multiservices
• service loaders
Ubic.pm API
Ubic.pm API

Ubic->start(‘foo’);

Ubic->stop(‘foo’);

# and many other methods
Service state is different from
service status

# ubic status foo

foo    running (pid 29551)

# kill 29551

# ubic status foo

foo    not running

# ubic stop foo

Stopping foo... not running

# ubic status

foo    off
Watchdog adjusts service status
in accordance with service state.

# ubic status foo

foo    running (pid 29551)

# kill 29551

# ubic status foo

foo    not running

# ubic-watchdog foo

[Fri Dec 9 03:45:37 2011]	 foo status is 'not running',
restarting
Ubic is not a supervisor



There is no global supervisor process.
Ubic is more similar to init scripts than
to upstart or launchd in this aspect.
Service state is persistent


• state is stored on disk and kept even after
  reboot

• you don’t have to stop services on host
  shutdown and start them on reboot
  (I know some people want a proper
  shutdown; I’ll get to that later)
Ubic.pm API
            consumers

• /usr/bin/ubic — command-line frontend
• ubic.watchdog service — monitors everything
• ubic.ping — reports service status via http
Ideas for more features


• ubic-web — html app frontend
• ‘ubic shutdown’ command which stops
  everything but don’t change state
• parameterized commands
Services API
Service is an object
Services conform to simple API:

 use parent qw(Ubic::Service);

 sub start { ... }

 sub stop { ... }

 sub status { ... }
Start code
can get complicated
 use parent qw(Ubic::Service);

 sub start {
     # check if daemon is running
     # start daemon
     # sleep for some time
     # check status
     # sleep more
     # check status
     # ... etc
 }
So Ubic::Service::Skeleton
   provides shortcuts

  use parent qw(Ubic::Service::Skeleton);

  sub start_impl { ... }
  sub start_impl { ... }
  sub status_impl { ... }

  sub timeout_options
      { start => { step => 0.1, trials => 3 } }
  }
Services inherit from
      other services
                               Ubic::Service


Ubic::Multiservice     Ubic::Service::Skeleton


 Ubic::Service::SimpleDaemon                   Ubic::Service::InitScriptWrapper


                       Ubic::Service::ZooKeeper        Ubic::Service::MongoDB

                       Ubic::Service::Plack          Ubic::Service::Memcached



                       Ubic::Service::Starman
Custom status() methods
           Example from Ubic::Service::MongoDB

sub status_impl {

    my $self = shift;

    my $running = check_daemon($self->pidfile);

    return result('not running') unless $running; 

    my $status = MongoDB::Connection->new(...)->admin->run_command({ serverStatus => 1 });

    if (!$status->{ok} || $status->{ok} != 1) {

        return result('broken');

    } else {

        return result('running');

    }

}
SimpleDaemon is the
most popular service class
   Ubic::Service::SimpleDaemon->new(

        bin => ‘plackup /usr/share/foo/app.psgi’,

        stdout => ‘/var/log/foo.log’,

        stderr => ‘/var/log/foo.err.log’,

        ubic_log => ‘/var/log/foo.ubic.log’,

        user => ‘daemon’,

        group => ‘daemon’,

        cwd => ‘/’,

        env => { PERL5LIB => ‘/usr/share/foo/lib’ },

   };
Ideas for
    new service modules
• restart (report ‘broken’ status) on
  memory leaks
• or on code changes
• or once per N seconds
• or if service doesn’t answer http ping
  (parameterize with url, port and
  timeout)
More ideas for
   new service modules
• better mechanism for adding features
  (decorators/roles/middlewares)
• simple daemonizer which uses
  Proc::Daemon instead of Ubic::Daemon
• wrappers for other service managers
  (turn daemontools service to ubic
  service)
Multiservices API
All services belong to
the global service tree

# ubic status
foo     running (pid 1234)
bar     running (pid 1235)
ubic
     ubic.ping      running (pid 2182)
     ubic.update    running (pid 2181)
     ubic.watchdog running (pid 1996)
Files in service dir describe
   services with perl code

   # cat /etc/ubic/service/foo

   use Ubic::Service::SimpleDaemon;

   Ubic::Service::SimpleDaemon->new(
       bin => ‘sleep 1000’,
       stdout => ‘/var/log/foo.log’,
   );
Subdirectories
become multiservices
  # ls ~/ubic/service/ubic/
  ping     update   watchdog

  # ubic status ubic
  ubic
       ubic.ping     running (pid 2182)
       ubic.update   running (pid 2181)
       ubic.watchdog running (pid 1996)
Or you can implement
 multiservice in perl
 # cat /etc/ubic/service/psgi
 use parent qw(Ubic::Multiservice);
 use Ubic::Service::Plack;

 sub simple_service {
     my ($self, $name) = @_;
     return Ubic::Service::Plack->new(...);
 }

 sub service_names {
 my @configs = glob “/psgi_apps/*.psgi”;
     s{^/psgi_apps(.*).psgi$}{$1} for @configs;
     return @configs;
 }
Another use case:
create N services
use Ubic::Multiservice::Simple;
use Ubic::Service::SimpleDaemon;

return Ubic::Multiservice::Simple->new({
    map {
        "worker$_" =>
Ubic::Service::SimpleDaemon->new({
            bin => "/usr/bin/worker.pl",
        })
    } (1..10)
});
Ideas for more
          multiservices

• bind the whole /etc/init.d/ or /service/
  to ubic
• “stash” multiservice — create services
  from command-line without creating
  any configs
Service loaders
Configs with .ini
extension are different

  # cat /etc/ubic/service/foo.ini
  module = Ubic::Service::SimpleDaemon
  [options]
  bin = sleep 1000
  stdout = /var/log/foo.log
Configs with any
extension are different

• ini configs are loaded with
  Ubic::ServiceLoader::Ext::ini
• experimental yaml service loader is in
  separate repo (not sure if extra
  dependency is worth it)
Now we can reach to
  non-perl users
• Node.JS guys at Yandex use Ubic
• Java guys at Yandex use Ubic
• admins who don’t know Perl use Ubic
• all of them do it because they like it, not
  because I forced them :)
• tell your non-perl friends about Ubic
  today!
Ideas for more
        service loaders
• .bin — turn symlink to binary into
  service
• new [addons] section to ini config,
  which would wrap service object in
  listed decorators
• .xml, .json (I know we don’t need so
  many of different formats, but Node.JS
  people like to reinvent the wheel)
More ideas
There are no
cross-platform service managers


 • Let’s port Ubic to Windows!

 • several guys tried to help but disappeared

 • github.com/berekuk/Ubic/wiki/Windows
   is the current plan
Plugins

• add a new command to /usr/bin/ubic

• or add memory leaks monitoring to all
  services at once

• API: apply the given middleware to all
  services
The End
Questions? Patches?

github.com/berekuk/Ubic

#ubic at irc.perl.org

“Like” Ubic on metacpan: metacpan.org/release/Ubic
This is the slide #42

Ubic YAPC 2012

  • 1.
    Friends, this isclean-up time and we're discounting all our silent, electric Ubiks by this much money. Yes' we're throwing away the bluebook. And remember: every Ubik on our lot has been used only as directed. Phillip K. Dick. “Ubik”. Ubic Polymorphic service manager github.com/berekuk/ubic
  • 2.
  • 3.
    Install Debian/Ubuntu: apt-add-repository ppa:berekuk/ubic && apt-get install ubic FreeBSD: cd /usr/ports/sysutils/p5-Ubic && make install Everywhere else: cpanm Ubic && ubic-admin setup
  • 4.
    Write service config # cat >/etc/ubic/service/foo use Ubic::Service::SimpleDaemon; Ubic::Service::SimpleDaemon->new( bin => ‘sleep 1000’, stdout => ‘/var/log/foo.log’, );
  • 5.
    Run it # ubicstart foo Starting foo... started (pid 28152)
  • 6.
  • 7.
    Flexible perl-based polymorphic service manager
  • 8.
  • 9.
    I’ll talk aboutAPIs instead of features today • Ubic.pm glue code • services • multiservices • service loaders
  • 10.
  • 11.
  • 12.
    Service state isdifferent from service status # ubic status foo foo running (pid 29551) # kill 29551 # ubic status foo foo not running # ubic stop foo Stopping foo... not running # ubic status foo off
  • 13.
    Watchdog adjusts servicestatus in accordance with service state. # ubic status foo foo running (pid 29551) # kill 29551 # ubic status foo foo not running # ubic-watchdog foo [Fri Dec 9 03:45:37 2011] foo status is 'not running', restarting
  • 14.
    Ubic is nota supervisor There is no global supervisor process. Ubic is more similar to init scripts than to upstart or launchd in this aspect.
  • 15.
    Service state ispersistent • state is stored on disk and kept even after reboot • you don’t have to stop services on host shutdown and start them on reboot (I know some people want a proper shutdown; I’ll get to that later)
  • 16.
    Ubic.pm API consumers • /usr/bin/ubic — command-line frontend • ubic.watchdog service — monitors everything • ubic.ping — reports service status via http
  • 17.
    Ideas for morefeatures • ubic-web — html app frontend • ‘ubic shutdown’ command which stops everything but don’t change state • parameterized commands
  • 18.
  • 19.
    Service is anobject Services conform to simple API: use parent qw(Ubic::Service); sub start { ... } sub stop { ... } sub status { ... }
  • 20.
    Start code can getcomplicated use parent qw(Ubic::Service); sub start { # check if daemon is running # start daemon # sleep for some time # check status # sleep more # check status # ... etc }
  • 21.
    So Ubic::Service::Skeleton provides shortcuts use parent qw(Ubic::Service::Skeleton); sub start_impl { ... } sub start_impl { ... } sub status_impl { ... } sub timeout_options { start => { step => 0.1, trials => 3 } } }
  • 22.
    Services inherit from other services Ubic::Service Ubic::Multiservice Ubic::Service::Skeleton Ubic::Service::SimpleDaemon Ubic::Service::InitScriptWrapper Ubic::Service::ZooKeeper Ubic::Service::MongoDB Ubic::Service::Plack Ubic::Service::Memcached Ubic::Service::Starman
  • 23.
    Custom status() methods Example from Ubic::Service::MongoDB sub status_impl {     my $self = shift;     my $running = check_daemon($self->pidfile);     return result('not running') unless $running;      my $status = MongoDB::Connection->new(...)->admin->run_command({ serverStatus => 1 });     if (!$status->{ok} || $status->{ok} != 1) {         return result('broken');     } else {         return result('running');     } }
  • 24.
    SimpleDaemon is the mostpopular service class Ubic::Service::SimpleDaemon->new( bin => ‘plackup /usr/share/foo/app.psgi’, stdout => ‘/var/log/foo.log’, stderr => ‘/var/log/foo.err.log’, ubic_log => ‘/var/log/foo.ubic.log’, user => ‘daemon’, group => ‘daemon’, cwd => ‘/’, env => { PERL5LIB => ‘/usr/share/foo/lib’ }, };
  • 25.
    Ideas for new service modules • restart (report ‘broken’ status) on memory leaks • or on code changes • or once per N seconds • or if service doesn’t answer http ping (parameterize with url, port and timeout)
  • 26.
    More ideas for new service modules • better mechanism for adding features (decorators/roles/middlewares) • simple daemonizer which uses Proc::Daemon instead of Ubic::Daemon • wrappers for other service managers (turn daemontools service to ubic service)
  • 27.
  • 28.
    All services belongto the global service tree # ubic status foo running (pid 1234) bar running (pid 1235) ubic ubic.ping running (pid 2182) ubic.update running (pid 2181) ubic.watchdog running (pid 1996)
  • 29.
    Files in servicedir describe services with perl code # cat /etc/ubic/service/foo use Ubic::Service::SimpleDaemon; Ubic::Service::SimpleDaemon->new( bin => ‘sleep 1000’, stdout => ‘/var/log/foo.log’, );
  • 30.
    Subdirectories become multiservices # ls ~/ubic/service/ubic/ ping update watchdog # ubic status ubic ubic ubic.ping running (pid 2182) ubic.update running (pid 2181) ubic.watchdog running (pid 1996)
  • 31.
    Or you canimplement multiservice in perl # cat /etc/ubic/service/psgi use parent qw(Ubic::Multiservice); use Ubic::Service::Plack; sub simple_service { my ($self, $name) = @_; return Ubic::Service::Plack->new(...); } sub service_names { my @configs = glob “/psgi_apps/*.psgi”; s{^/psgi_apps(.*).psgi$}{$1} for @configs; return @configs; }
  • 32.
    Another use case: createN services use Ubic::Multiservice::Simple; use Ubic::Service::SimpleDaemon; return Ubic::Multiservice::Simple->new({     map {         "worker$_" => Ubic::Service::SimpleDaemon->new({             bin => "/usr/bin/worker.pl",         })     } (1..10) });
  • 33.
    Ideas for more multiservices • bind the whole /etc/init.d/ or /service/ to ubic • “stash” multiservice — create services from command-line without creating any configs
  • 34.
  • 35.
    Configs with .ini extensionare different # cat /etc/ubic/service/foo.ini module = Ubic::Service::SimpleDaemon [options] bin = sleep 1000 stdout = /var/log/foo.log
  • 36.
    Configs with any extensionare different • ini configs are loaded with Ubic::ServiceLoader::Ext::ini • experimental yaml service loader is in separate repo (not sure if extra dependency is worth it)
  • 37.
    Now we canreach to non-perl users • Node.JS guys at Yandex use Ubic • Java guys at Yandex use Ubic • admins who don’t know Perl use Ubic • all of them do it because they like it, not because I forced them :) • tell your non-perl friends about Ubic today!
  • 38.
    Ideas for more service loaders • .bin — turn symlink to binary into service • new [addons] section to ini config, which would wrap service object in listed decorators • .xml, .json (I know we don’t need so many of different formats, but Node.JS people like to reinvent the wheel)
  • 39.
  • 40.
    There are no cross-platformservice managers • Let’s port Ubic to Windows! • several guys tried to help but disappeared • github.com/berekuk/Ubic/wiki/Windows is the current plan
  • 41.
    Plugins • add anew command to /usr/bin/ubic • or add memory leaks monitoring to all services at once • API: apply the given middleware to all services
  • 42.
  • 43.
    Questions? Patches? github.com/berekuk/Ubic #ubic atirc.perl.org “Like” Ubic on metacpan: metacpan.org/release/Ubic
  • 44.
    This is theslide #42