• Like
  • Save
Assurer - a pluggable server testing/monitoring framework
Upcoming SlideShare
Loading in...5
×
 

Assurer - a pluggable server testing/monitoring framework

on

  • 1,662 views

 

Statistics

Views

Total Views
1,662
Views on SlideShare
1,545
Embed Views
117

Actions

Likes
0
Downloads
2
Comments
0

1 Embed 117

http://coderwall.com 117

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Assurer - a pluggable server testing/monitoring framework Assurer - a pluggable server testing/monitoring framework Presentation Transcript

    • Assurer – A server testing/monitoring framework Gosuke Miyashita http://mizzy.org/
    • Myself
      • Known as Chocoboy Miyashita somewhere
      • Working in paperboy&co.
      • Mainly developed public/private web service APIs with perl and catalyt
      • ... but now also woking on servers and networks and so on
      • Today’s my son’s entrance ceremony of an elementary school
    •  
    • What’s Assurer ?
      • A server testing/monitoring framework like Plagger
        • Plugin mechanism
        • YAML configurable
        • assets
        • Test by Test::Base
      • Originally developed for one time test of servers that are newly built
      • Monitoring seems like testing, so decided to cover monitoring
    • Where the name Assurer derived from ?
      • Pronounce “A-shu-ra”
      • Plagger like tools ended with “er”
        • Archer by tokuhirom
        • Observer by Hatena (not in public)
        • Dishuber by Yappo
        • Precure by Kan Fushihara
      • Testing is also called “Quality Assurance”
      MISS
    • Image of exec Assurer #0
    • Image of exec Assurer #1
    • Image of Asura Baster
    • Image of the real Asura
    • Image of fake Asura
    • Execution phases of Assurer
    • Execution phases of Assurer
      • Test phase
        • Execute test
      • Nofity phase
        • Notify test results
      • Format phase
        • Format test results
      • Publish phase
        • Output formatted test results
    • Relation of each phases Test Publish Format Nofity
    • Details of each phases
    • Test phase
    • Test phase
      • Execute test
      • Plugins on this phase:
        • Test::HTTP
        • Test::SMTP
        • Test::DBI
        • Test::Ping
        • etc
    • Config of test phase
      • test:
      • - module: HTTP
      • config:
      • host: www.mizzy.org
      • content: It works!
    • Notify phase
    • Notify フェーズ
      • Notify test results
      • Plugin on this phase:
        • Notify::IRC
    • Config of notify phase
      • notify:
      • - module: IRC
      • config:
      • daemon_port: 9991
      • nickname: assurerbot
      • server_host: chat.freenode.net
      • server_port: 6667
      • server_channels:
      • - #assurer
      • - #assurer-jp
      • charset: iso-2022-jp
      • announce: notice
    • Notify::IRC example
    • Format phase
    • Format phase
      • Format test results
      • Plugins on this phase:
        • Format::Text
        • Fomat::HTML
    • Config of format phase
      • format:
      • - module: HTML
      • config:
      • css: /css/style.css
    • Publish phase
    • Publish phase
      • Output formatted results
      • Plugins on this phase:
        • Publish::Term
        • Publish::Mail
        • Publish::File
    • Config of publish phase
      • publish:
      • - module: Mail
      • config:
      • subject: Test results from Assurer
      • to: someone@example.com
      • from: root@example.com
      publish: - module: Mail config: subject: Test results from Assurer to: someone@example.com from: root@example.com
    • Format::Text + Publish::Term
    • Format::HTML + Publish::File
    • Format::Text + Publish::Mail
    • Summary for now
      • Assurer is a serverPlagger
      • Test, Notify, Format and Publish phases
      • Easily change tests and outputs with plugins
    • Extra functions
    • Hosts and Roles
    • Hosts and roles
      • Write target hosts and roles in one place on config.yaml
    • Config without hosts and roles test: - module: HTTP config: host: www0.mizzy.org - module: HTTP config: host: www1.mizzy.org
    • Config with hosts and roles test: - module: HTTP role: web hosts: web: - www0.mizzy.org - www1.mizzy.org
    • Filter
    • Filter
      • Filtering test results on format and notify phase
      • Filtering formatted results on publish phase
    • Filter on format phase
      • Only OK results are formatted as text format
      format: - module: Text filter: module: Status status: ok
    • Filter on notify phase
      • Only NOK results are notified to IRC
      notify: - module: IRC filter: module: Status status: not ok
    • Filter on publish phase
      • Text formatted results are displayed on a terminal,html formatted results are sent as a mail
      format: - module: Text - module: HTML pusblish: - module: Term filter: module: Type type: text/plain - module: Mail filter: module: Type type: text/html
    • Config validation
    • Config validation
      • Kwalify – YAML schema validator http://www.kuwata-lab.com/kwalify /
      • Validate common config
      • Validate config of each plugin
      • Kwalify schemas for each plugin put under assets/kwalify/plugins
    • Mistake on common config test s : # test is right - module: Test::HTTP $ ./assurer.pl -c config.yaml Assurer::ConfigLoader [fatal] - [/] Expected required key `test‘ - [/tests] Unexpected key `tests' at line 46
    • Mistake on plugin config test: - module: HTTP config: content s : XXX # content is right $ ./assurer.pl -c config.yaml Assurer::ConfigLoader [fatal] Config error in Test::HTTP - [/contents] Unexpected key `contents' at line 66
    • Shell mode
    • Shell mode
      • Execute shell command on hosts in config.yaml
    • Shell mode $ ./assurer.pl --shell assurer> uptime [www.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17 [svn.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17 [ftp.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17
    • Exec on hosts belong to specified role $ ./assurer.pl --shell --role=web assurer> uptime [www.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17
    • Special command !on assurer> !on app1.foo.com app2.foo.com do uptime # Exec on app1.foo.com and app2.foo.com only assurer> !on /.*.foo.com/ do uptime # Exec on hosts matched .*.foo.com only
    • Special command !with assurer> !with web db do uptime # exec on hosts belong to web role and db role assurer> !with /web|mail/ do uptime # exec on hosts belong to role match web or mail
    • Special command !test assurer> !test HTTP Assurer::Plugin::Test::HTTP [info] Testing HTTP on www0.mizzy.org ... ok 1 - HTTP status code of http://mizzy.org:80 is 200 not ok 2 – Content of http://mizzy.org:80 matches 'It works!‘ assurer>
    • Special command !test assurer> !test HTTP on app1.foo.com # test http on app1.foo.com assurer> !test HTTP on /.*.foo.com/ # can use regexp assurer> !test HTTP with web # test http on hosts belong to web role assurer> !test HTTP with /web|mail/ # can use regexp
    • Parallel testing
    • Pallarel testing
      • Multiple test processes on test phase
      • Default is 5
      • Can change number processes working at once
      $ assurer.pl --para=20
    • Distributed testing
    • Distributed testing localhost test0.mizzy.org www0.mizzy.org www1.mizzy.org www2.mizzy.org test1.mizzy.org test2.mizzy.org 1. ssh assurer_test.pl 3. Return result 2. Exec test
    • Config for distributed testing exec_on: - host: test0.mizzy.org priority: 3 - host: test1.mizzy.org priority: 2 - host: test2.mizzy.org priority: 1
    • Auto config generation
    • Auto config generation
      • Scan hosts in template.yaml by Nmap::Scanner
      • Detect services working on the hosts and generete config.yaml automatically
      • config templates for each plugin put under assets/discover (http.yaml, mysql.yaml, etc)
      # assurer.pl --discover -c template.yaml
    • Integragte Test::WWW::Mechanize
    • Integrate Test::WWW::Mechanize
      • Can write tests using Test::WWW::Mechanize
      • Put .pl files under assets/plugins/Test-WWW-Mechanize
      • Pre-set $mech, $host on execution
    • asset file for Test::WWW::Mechanize # $host and $mech are set by Assure core $mech ->get_ok("http:// $host ", "got htttp:// $host "); $mech ->content_contains('It works!', "Content matches 'It works!'");
    • Inside Assurer
    • Assurer::Test module
    • Assurer::Test module
      • Export Test::More like methods
      • is,() like(), ok()
      • Output test results formatted as TAP (Test Analyze Protocol)
    • Inside test plugins
    • Inside test plugins package Assurer::Plugin::Test::HTTP; use base qw( Assurer::Plugin::Test ); use Assurer::Test; sub register { my $self = shift; $self->register_tests( qw/ status content server / ); } sub status { my ( $self, $context, $args ) = @_; ... 中略 is( $res->code, '200', 'HTTP status code of $self->{url} is 200' ); }
    • Internal mechanism in test phase
    • Internal mechanism in test phase assurer.pl Assurer::bootstrap() Assurer::Dispatch::run() call exec directly or by ssh exec tests return test results call generate result objects next phase assurer_test.pl assurer_test.pl assurer_test.pl assurer_test.pl assurer_test.pl Assurer::Result Assurer::Result Assurer::Result Assurer::Result Assurer::Result
    • Assurer::Dispatch::run()
      • Exec assurer_test.pl for each test
      • Pallarelize by POE::Wheel::Run
      • Number of processed controlled by POE::Component::JobQueue
      • Pass config for plugin and context object serialized by YAML and encoded by Base64
    • assurer_test.pl assurer_test.pl -– config = LS0tCmNvbmZpZzoKICBjb250ZW50OiBJ dCB3b3JrcyEKICBob3N0OiB3d3cubWl6enkub3JnCm1vZHVsZTogSFRUUApuYW1lOiBIVFRQIHRlc3QKcm9sZTogd2ViCg== - - context =LS0tICEhcGVybC9oYXNoOkFzc3VyZXIKYmFzZV9kaXI6IC9 ob21lL21peWEvc3ZrL0Fzc3VyZXIKY29uZmlnOgogIGZvcm1hdDoKICAgIC0gY29uZmlnOiAmMSB7fQogICAgICBtb2R1bGU6IFRleHQKICBnbG9iYWw6CiAgICBob3N0OiB+CiAgICBpbnRlcnZhbDogMwogICAgbG9nOgogICAgICBsZXZlbDogZGVidWcKICAgIG5vX2RpYWc6IDAKICAgIHJldHJ5OiAzCiAgaG9zdHM6CiAgICBmdHA6CiAgICAgIC0gZnRwLm1penp5Lm9yZwogICAgc3ZuOgogICAgICAtIHN2bi5taXp6eS5vcmcKICAgIHdlYjoKICAgICAgLSB3d3cubWl6enkub3JnCiAgICAgIC0gc3ZuLm1penp5Lm9yZwogICAgICAtIHRyYWMubWl6enkub3JnCiAgbm90aWZ5OgogICAgLSBjb25maWc6CiAgICAgICAgYW5ub3VuY2U6IG5vdGljZQogICAgICAgIGNoYXJzZXQ6IGlzby0yMDIyLWpwCiAgICAgICAgZGFlbW9uX3BvcnQ6IDk5OTEKICAgICAgICBuaWNrbmFtZTogYXNzdXJlcmJvdAog ... cont ...
    • Why POE::Wheel::Run ?
      • Would like to exec tests as other processes
        • Need non-blocking I/O with single process and POE
        • Can easily distribute to remote servers (exec directly on local, exec by ssh on remote )
      • Communication with test processes
        • Can communicate easily with test processes by STDIN, STDOUT and STDERR
    • Roadmap
    • Roadmap
      • Integrate with Nagios/NRPE (Test::Nagios)
      • Enhance notify phase
      • Store Plugin (store test results in database)
      • Integrate with Archer, puppet (share host lists)
      • Test of Assurer itsself
    • Web site and IRC channels
    • Web sites and IRC channels
      • http://assurer.jp/trac
      • http://mizzy.org/
      • #assurer@chat.freenode.net
      • #assurer-ja@chat.freenode.net
    • Thanks