Your SlideShare is downloading. ×
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Assurer - a pluggable server testing/monitoring framework

1,379

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,379
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Assurer – A server testing/monitoring framework Gosuke Miyashita http://mizzy.org/
  • 2. Myself
    • Known as Chocoboy Miyashita somewhere
    • Working in paperboy&co.
    • Mainly developed public/private web service APIs with perl and catalyt
    • ... but now also woking on servers and networks and so on
    • Today’s my son’s entrance ceremony of an elementary school
  • 3.  
  • 4. What’s Assurer ?
    • A server testing/monitoring framework like Plagger
      • Plugin mechanism
      • YAML configurable
      • assets
      • Test by Test::Base
    • Originally developed for one time test of servers that are newly built
    • Monitoring seems like testing, so decided to cover monitoring
  • 5. Where the name Assurer derived from ?
    • Pronounce “A-shu-ra”
    • Plagger like tools ended with “er”
      • Archer by tokuhirom
      • Observer by Hatena (not in public)
      • Dishuber by Yappo
      • Precure by Kan Fushihara
    • Testing is also called “Quality Assurance”
    MISS
  • 6. Image of exec Assurer #0
  • 7. Image of exec Assurer #1
  • 8. Image of Asura Baster
  • 9. Image of the real Asura
  • 10. Image of fake Asura
  • 11. Execution phases of Assurer
  • 12. Execution phases of Assurer
    • Test phase
      • Execute test
    • Nofity phase
      • Notify test results
    • Format phase
      • Format test results
    • Publish phase
      • Output formatted test results
  • 13. Relation of each phases Test Publish Format Nofity
  • 14. Details of each phases
  • 15. Test phase
  • 16. Test phase
    • Execute test
    • Plugins on this phase:
      • Test::HTTP
      • Test::SMTP
      • Test::DBI
      • Test::Ping
      • etc
  • 17. Config of test phase
    • test:
    • - module: HTTP
    • config:
    • host: www.mizzy.org
    • content: It works!
  • 18. Notify phase
  • 19. Notify フェーズ
    • Notify test results
    • Plugin on this phase:
      • Notify::IRC
  • 20. Config of notify phase
    • notify:
    • - module: IRC
    • config:
    • daemon_port: 9991
    • nickname: assurerbot
    • server_host: chat.freenode.net
    • server_port: 6667
    • server_channels:
    • - #assurer
    • - #assurer-jp
    • charset: iso-2022-jp
    • announce: notice
  • 21. Notify::IRC example
  • 22. Format phase
  • 23. Format phase
    • Format test results
    • Plugins on this phase:
      • Format::Text
      • Fomat::HTML
  • 24. Config of format phase
    • format:
    • - module: HTML
    • config:
    • css: /css/style.css
  • 25. Publish phase
  • 26. Publish phase
    • Output formatted results
    • Plugins on this phase:
      • Publish::Term
      • Publish::Mail
      • Publish::File
  • 27. Config of publish phase
    • publish:
    • - module: Mail
    • config:
    • subject: Test results from Assurer
    • to: someone@example.com
    • from: root@example.com
    publish: - module: Mail config: subject: Test results from Assurer to: someone@example.com from: root@example.com
  • 28. Format::Text + Publish::Term
  • 29. Format::HTML + Publish::File
  • 30. Format::Text + Publish::Mail
  • 31. Summary for now
    • Assurer is a serverPlagger
    • Test, Notify, Format and Publish phases
    • Easily change tests and outputs with plugins
  • 32. Extra functions
  • 33. Hosts and Roles
  • 34. Hosts and roles
    • Write target hosts and roles in one place on config.yaml
  • 35. Config without hosts and roles test: - module: HTTP config: host: www0.mizzy.org - module: HTTP config: host: www1.mizzy.org
  • 36. Config with hosts and roles test: - module: HTTP role: web hosts: web: - www0.mizzy.org - www1.mizzy.org
  • 37. Filter
  • 38. Filter
    • Filtering test results on format and notify phase
    • Filtering formatted results on publish phase
  • 39. Filter on format phase
    • Only OK results are formatted as text format
    format: - module: Text filter: module: Status status: ok
  • 40. Filter on notify phase
    • Only NOK results are notified to IRC
    notify: - module: IRC filter: module: Status status: not ok
  • 41. Filter on publish phase
    • Text formatted results are displayed on a terminal,html formatted results are sent as a mail
    format: - module: Text - module: HTML pusblish: - module: Term filter: module: Type type: text/plain - module: Mail filter: module: Type type: text/html
  • 42. Config validation
  • 43. Config validation
    • Kwalify – YAML schema validator http://www.kuwata-lab.com/kwalify /
    • Validate common config
    • Validate config of each plugin
    • Kwalify schemas for each plugin put under assets/kwalify/plugins
  • 44. Mistake on common config test s : # test is right - module: Test::HTTP $ ./assurer.pl -c config.yaml Assurer::ConfigLoader [fatal] - [/] Expected required key `test‘ - [/tests] Unexpected key `tests' at line 46
  • 45. Mistake on plugin config test: - module: HTTP config: content s : XXX # content is right $ ./assurer.pl -c config.yaml Assurer::ConfigLoader [fatal] Config error in Test::HTTP - [/contents] Unexpected key `contents' at line 66
  • 46. Shell mode
  • 47. Shell mode
    • Execute shell command on hosts in config.yaml
  • 48. Shell mode $ ./assurer.pl --shell assurer> uptime [www.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17 [svn.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17 [ftp.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17
  • 49. Exec on hosts belong to specified role $ ./assurer.pl --shell --role=web assurer> uptime [www.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17
  • 50. Special command !on assurer> !on app1.foo.com app2.foo.com do uptime # Exec on app1.foo.com and app2.foo.com only assurer> !on /.*.foo.com/ do uptime # Exec on hosts matched .*.foo.com only
  • 51. Special command !with assurer> !with web db do uptime # exec on hosts belong to web role and db role assurer> !with /web|mail/ do uptime # exec on hosts belong to role match web or mail
  • 52. Special command !test assurer> !test HTTP Assurer::Plugin::Test::HTTP [info] Testing HTTP on www0.mizzy.org ... ok 1 - HTTP status code of http://mizzy.org:80 is 200 not ok 2 – Content of http://mizzy.org:80 matches 'It works!‘ assurer>
  • 53. Special command !test assurer> !test HTTP on app1.foo.com # test http on app1.foo.com assurer> !test HTTP on /.*.foo.com/ # can use regexp assurer> !test HTTP with web # test http on hosts belong to web role assurer> !test HTTP with /web|mail/ # can use regexp
  • 54. Parallel testing
  • 55. Pallarel testing
    • Multiple test processes on test phase
    • Default is 5
    • Can change number processes working at once
    $ assurer.pl --para=20
  • 56. Distributed testing
  • 57. Distributed testing localhost test0.mizzy.org www0.mizzy.org www1.mizzy.org www2.mizzy.org test1.mizzy.org test2.mizzy.org 1. ssh assurer_test.pl 3. Return result 2. Exec test
  • 58. Config for distributed testing exec_on: - host: test0.mizzy.org priority: 3 - host: test1.mizzy.org priority: 2 - host: test2.mizzy.org priority: 1
  • 59. Auto config generation
  • 60. Auto config generation
    • Scan hosts in template.yaml by Nmap::Scanner
    • Detect services working on the hosts and generete config.yaml automatically
    • config templates for each plugin put under assets/discover (http.yaml, mysql.yaml, etc)
    # assurer.pl --discover -c template.yaml
  • 61. Integragte Test::WWW::Mechanize
  • 62. Integrate Test::WWW::Mechanize
    • Can write tests using Test::WWW::Mechanize
    • Put .pl files under assets/plugins/Test-WWW-Mechanize
    • Pre-set $mech, $host on execution
  • 63. asset file for Test::WWW::Mechanize # $host and $mech are set by Assure core $mech ->get_ok("http:// $host ", "got htttp:// $host "); $mech ->content_contains('It works!', "Content matches 'It works!'");
  • 64. Inside Assurer
  • 65. Assurer::Test module
  • 66. Assurer::Test module
    • Export Test::More like methods
    • is,() like(), ok()
    • Output test results formatted as TAP (Test Analyze Protocol)
  • 67. Inside test plugins
  • 68. Inside test plugins package Assurer::Plugin::Test::HTTP; use base qw( Assurer::Plugin::Test ); use Assurer::Test; sub register { my $self = shift; $self->register_tests( qw/ status content server / ); } sub status { my ( $self, $context, $args ) = @_; ... 中略 is( $res->code, '200', 'HTTP status code of $self->{url} is 200' ); }
  • 69. Internal mechanism in test phase
  • 70. Internal mechanism in test phase assurer.pl Assurer::bootstrap() Assurer::Dispatch::run() call exec directly or by ssh exec tests return test results call generate result objects next phase assurer_test.pl assurer_test.pl assurer_test.pl assurer_test.pl assurer_test.pl Assurer::Result Assurer::Result Assurer::Result Assurer::Result Assurer::Result
  • 71. Assurer::Dispatch::run()
    • Exec assurer_test.pl for each test
    • Pallarelize by POE::Wheel::Run
    • Number of processed controlled by POE::Component::JobQueue
    • Pass config for plugin and context object serialized by YAML and encoded by Base64
  • 72. assurer_test.pl assurer_test.pl -– config = LS0tCmNvbmZpZzoKICBjb250ZW50OiBJ dCB3b3JrcyEKICBob3N0OiB3d3cubWl6enkub3JnCm1vZHVsZTogSFRUUApuYW1lOiBIVFRQIHRlc3QKcm9sZTogd2ViCg== - - context =LS0tICEhcGVybC9oYXNoOkFzc3VyZXIKYmFzZV9kaXI6IC9 ob21lL21peWEvc3ZrL0Fzc3VyZXIKY29uZmlnOgogIGZvcm1hdDoKICAgIC0gY29uZmlnOiAmMSB7fQogICAgICBtb2R1bGU6IFRleHQKICBnbG9iYWw6CiAgICBob3N0OiB+CiAgICBpbnRlcnZhbDogMwogICAgbG9nOgogICAgICBsZXZlbDogZGVidWcKICAgIG5vX2RpYWc6IDAKICAgIHJldHJ5OiAzCiAgaG9zdHM6CiAgICBmdHA6CiAgICAgIC0gZnRwLm1penp5Lm9yZwogICAgc3ZuOgogICAgICAtIHN2bi5taXp6eS5vcmcKICAgIHdlYjoKICAgICAgLSB3d3cubWl6enkub3JnCiAgICAgIC0gc3ZuLm1penp5Lm9yZwogICAgICAtIHRyYWMubWl6enkub3JnCiAgbm90aWZ5OgogICAgLSBjb25maWc6CiAgICAgICAgYW5ub3VuY2U6IG5vdGljZQogICAgICAgIGNoYXJzZXQ6IGlzby0yMDIyLWpwCiAgICAgICAgZGFlbW9uX3BvcnQ6IDk5OTEKICAgICAgICBuaWNrbmFtZTogYXNzdXJlcmJvdAog ... cont ...
  • 73. Why POE::Wheel::Run ?
    • Would like to exec tests as other processes
      • Need non-blocking I/O with single process and POE
      • Can easily distribute to remote servers (exec directly on local, exec by ssh on remote )
    • Communication with test processes
      • Can communicate easily with test processes by STDIN, STDOUT and STDERR
  • 74. Roadmap
  • 75. Roadmap
    • Integrate with Nagios/NRPE (Test::Nagios)
    • Enhance notify phase
    • Store Plugin (store test results in database)
    • Integrate with Archer, puppet (share host lists)
    • Test of Assurer itsself
  • 76. Web site and IRC channels
  • 77. Web sites and IRC channels
    • http://assurer.jp/trac
    • http://mizzy.org/
    • #assurer@chat.freenode.net
    • #assurer-ja@chat.freenode.net
  • 78. Thanks

×