Assurer - a pluggable server testing/monitoring framework

Assurer – A server testing/monitoring framework Gosuke Miyashita http://mizzy.org/

Myself
Known as Chocoboy Miyashita somewhere
Working in paperboy&co.
Mainly developed public/private web service APIs with perl and catalyt
... but now also woking on servers and networks and so on
Today’s my son’s entrance ceremony of an elementary school

What’s Assurer ?
A server testing/monitoring framework like Plagger
Plugin mechanism
YAML configurable
assets
Test by Test::Base
Originally developed for one time test of servers that are newly built
Monitoring seems like testing, so decided to cover monitoring

Where the name Assurer derived from ?
Pronounce “A-shu-ra”
Plagger like tools ended with “er”
Archer by tokuhirom
Observer by Hatena (not in public)
Dishuber by Yappo
Precure by Kan Fushihara
Testing is also called “Quality Assurance”
MISS

Image of exec Assurer #0

Image of exec Assurer #1

Image of Asura Baster

Image of the real Asura

Image of fake Asura

Execution phases of Assurer

Execution phases of Assurer
Test phase
Execute test
Nofity phase
Notify test results
Format phase
Format test results
Publish phase
Output formatted test results

Relation of each phases Test Publish Format Nofity

Details of each phases

Test phase

Test phase
Execute test
Plugins on this phase:
Test::HTTP
Test::SMTP
Test::DBI
Test::Ping
etc

Config of test phase
test:
- module: HTTP
config:
host: www.mizzy.org
content: It works!

Notify phase

Notify フェーズ
Notify test results
Plugin on this phase:
Notify::IRC

Config of notify phase
notify:
- module: IRC
config:
daemon_port: 9991
nickname: assurerbot
server_host: chat.freenode.net
server_port: 6667
server_channels:
- #assurer
- #assurer-jp
charset: iso-2022-jp
announce: notice

Notify::IRC example

Format phase

Format phase
Format test results
Format::Text
Fomat::HTML

Config of format phase
format:
- module: HTML
config:
css: /css/style.css

Publish phase

Publish phase
Output formatted results
Publish::Term
Publish::Mail
Publish::File

Config of publish phase
publish:
- module: Mail
config:
subject: Test results from Assurer
to: someone@example.com
from: root@example.com
publish: - module: Mail config: subject: Test results from Assurer to: someone@example.com from: root@example.com

Format::Text + Publish::Term

Format::HTML + Publish::File

Format::Text + Publish::Mail

Summary for now
Assurer is a serverPlagger
Test, Notify, Format and Publish phases
Easily change tests and outputs with plugins

Extra functions

Hosts and Roles

Hosts and roles
Write target hosts and roles in one place on config.yaml

Config without hosts and roles test: - module: HTTP config: host: www0.mizzy.org - module: HTTP config: host: www1.mizzy.org

Config with hosts and roles test: - module: HTTP role: web hosts: web: - www0.mizzy.org - www1.mizzy.org

Filter

Filter
Filtering test results on format and notify phase
Filtering formatted results on publish phase

Filter on format phase
Only OK results are formatted as text format
format: - module: Text filter: module: Status status: ok

Filter on notify phase
Only NOK results are notified to IRC
notify: - module: IRC filter: module: Status status: not ok

Filter on publish phase
Text formatted results are displayed on a terminal,html formatted results are sent as a mail
format: - module: Text - module: HTML pusblish: - module: Term filter: module: Type type: text/plain - module: Mail filter: module: Type type: text/html

Config validation

Config validation
Kwalify – YAML schema validator http://www.kuwata-lab.com/kwalify /
Validate common config
Validate config of each plugin
Kwalify schemas for each plugin put under assets/kwalify/plugins

Mistake on common config test s : # test is right - module: Test::HTTP $ ./assurer.pl -c config.yaml Assurer::ConfigLoader [fatal] - [/] Expected required key `test‘ - [/tests] Unexpected key `tests' at line 46

Mistake on plugin config test: - module: HTTP config: content s : XXX # content is right $ ./assurer.pl -c config.yaml Assurer::ConfigLoader [fatal] Config error in Test::HTTP - [/contents] Unexpected key `contents' at line 66

Shell mode

Shell mode
Execute shell command on hosts in config.yaml

Shell mode $ ./assurer.pl --shell assurer> uptime [www.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17 [svn.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17 [ftp.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17

Exec on hosts belong to specified role $ ./assurer.pl --shell --role=web assurer> uptime [www.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17

Special command !on assurer> !on app1.foo.com app2.foo.com do uptime # Exec on app1.foo.com and app2.foo.com only assurer> !on /.*.foo.com/ do uptime # Exec on hosts matched .*.foo.com only

Special command !with assurer> !with web db do uptime # exec on hosts belong to web role and db role assurer> !with /web|mail/ do uptime # exec on hosts belong to role match web or mail

Special command !test assurer> !test HTTP Assurer::Plugin::Test::HTTP [info] Testing HTTP on www0.mizzy.org ... ok 1 - HTTP status code of http://mizzy.org:80 is 200 not ok 2 – Content of http://mizzy.org:80 matches 'It works!‘ assurer>

Special command !test assurer> !test HTTP on app1.foo.com # test http on app1.foo.com assurer> !test HTTP on /.*.foo.com/ # can use regexp assurer> !test HTTP with web # test http on hosts belong to web role assurer> !test HTTP with /web|mail/ # can use regexp

Parallel testing

Pallarel testing
Multiple test processes on test phase
Default is 5
Can change number processes working at once
$ assurer.pl --para=20

Distributed testing

Distributed testing localhost test0.mizzy.org www0.mizzy.org www1.mizzy.org www2.mizzy.org test1.mizzy.org test2.mizzy.org 1. ssh assurer_test.pl 3. Return result 2. Exec test

Config for distributed testing exec_on: - host: test0.mizzy.org priority: 3 - host: test1.mizzy.org priority: 2 - host: test2.mizzy.org priority: 1

Auto config generation

Auto config generation
Scan hosts in template.yaml by Nmap::Scanner
Detect services working on the hosts and generete config.yaml automatically
config templates for each plugin put under assets/discover (http.yaml, mysql.yaml, etc)
# assurer.pl --discover -c template.yaml

Integragte Test::WWW::Mechanize

Integrate Test::WWW::Mechanize
Can write tests using Test::WWW::Mechanize
Put .pl files under assets/plugins/Test-WWW-Mechanize
Pre-set $mech, $host on execution

asset file for Test::WWW::Mechanize # $host and $mech are set by Assure core $mech ->get_ok("http:// $host ", "got htttp:// $host "); $mech ->content_contains('It works!', "Content matches 'It works!'");

Inside Assurer

Assurer::Test module

Assurer::Test module
Export Test::More like methods
is,() like(), ok()
Output test results formatted as TAP (Test Analyze Protocol)

Inside test plugins

Inside test plugins package Assurer::Plugin::Test::HTTP; use base qw( Assurer::Plugin::Test ); use Assurer::Test; sub register { my $self = shift; $self->register_tests( qw/ status content server / ); } sub status { my ( $self, $context, $args ) = @_; ... 中略 is( $res->code, '200', 'HTTP status code of $self->{url} is 200' ); }

Internal mechanism in test phase

Internal mechanism in test phase assurer.pl Assurer::bootstrap() Assurer::Dispatch::run() call exec directly or by ssh exec tests return test results call generate result objects next phase assurer_test.pl assurer_test.pl assurer_test.pl assurer_test.pl assurer_test.pl Assurer::Result Assurer::Result Assurer::Result Assurer::Result Assurer::Result

Assurer::Dispatch::run()
Exec assurer_test.pl for each test
Pallarelize by POE::Wheel::Run
Number of processed controlled by POE::Component::JobQueue
Pass config for plugin and context object serialized by YAML and encoded by Base64

assurer_test.pl assurer_test.pl -– config = LS0tCmNvbmZpZzoKICBjb250ZW50OiBJ dCB3b3JrcyEKICBob3N0OiB3d3cubWl6enkub3JnCm1vZHVsZTogSFRUUApuYW1lOiBIVFRQIHRlc3QKcm9sZTogd2ViCg== - - context =LS0tICEhcGVybC9oYXNoOkFzc3VyZXIKYmFzZV9kaXI6IC9 ob21lL21peWEvc3ZrL0Fzc3VyZXIKY29uZmlnOgogIGZvcm1hdDoKICAgIC0gY29uZmlnOiAmMSB7fQogICAgICBtb2R1bGU6IFRleHQKICBnbG9iYWw6CiAgICBob3N0OiB+CiAgICBpbnRlcnZhbDogMwogICAgbG9nOgogICAgICBsZXZlbDogZGVidWcKICAgIG5vX2RpYWc6IDAKICAgIHJldHJ5OiAzCiAgaG9zdHM6CiAgICBmdHA6CiAgICAgIC0gZnRwLm1penp5Lm9yZwogICAgc3ZuOgogICAgICAtIHN2bi5taXp6eS5vcmcKICAgIHdlYjoKICAgICAgLSB3d3cubWl6enkub3JnCiAgICAgIC0gc3ZuLm1penp5Lm9yZwogICAgICAtIHRyYWMubWl6enkub3JnCiAgbm90aWZ5OgogICAgLSBjb25maWc6CiAgICAgICAgYW5ub3VuY2U6IG5vdGljZQogICAgICAgIGNoYXJzZXQ6IGlzby0yMDIyLWpwCiAgICAgICAgZGFlbW9uX3BvcnQ6IDk5OTEKICAgICAgICBuaWNrbmFtZTogYXNzdXJlcmJvdAog ... cont ...

Why POE::Wheel::Run ?
Would like to exec tests as other processes
Need non-blocking I/O with single process and POE
Can easily distribute to remote servers (exec directly on local, exec by ssh on remote ）
Communication with test processes
Can communicate easily with test processes by STDIN, STDOUT and STDERR

Roadmap

Roadmap
Integrate with Nagios/NRPE (Test::Nagios)
Enhance notify phase
Store Plugin (store test results in database)
Integrate with Archer, puppet (share host lists)
Test of Assurer itsself

Web site and IRC channels

Web sites and IRC channels
http://assurer.jp/trac
http://mizzy.org/
#assurer@chat.freenode.net
#assurer-ja@chat.freenode.net

Thanks

Assurer - a pluggable server testing/monitoring framework

by Gosuke Miyashita on Aug 04, 2011

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 115

Statistics

Assurer - a pluggable server testing/monitoring framework — Presentation Transcript