Your SlideShare is downloading. ×
0
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Assurer - a pluggable server testing/monitoring framework
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Assurer - a pluggable server testing/monitoring framework

1,404

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,404
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Assurer – A server testing/monitoring framework Gosuke Miyashita http://mizzy.org/
  • 2. Myself <ul><li>Known as Chocoboy Miyashita somewhere </li></ul><ul><li>Working in paperboy&co. </li></ul><ul><li>Mainly developed public/private web service APIs with perl and catalyt </li></ul><ul><li>... but now also woking on servers and networks and so on </li></ul><ul><li>Today’s my son’s entrance ceremony of an elementary school </li></ul>
  • 3.  
  • 4. What’s Assurer ? <ul><li>A server testing/monitoring framework like Plagger </li></ul><ul><ul><li>Plugin mechanism </li></ul></ul><ul><ul><li>YAML configurable </li></ul></ul><ul><ul><li>assets </li></ul></ul><ul><ul><li>Test by Test::Base </li></ul></ul><ul><li>Originally developed for one time test of servers that are newly built </li></ul><ul><li>Monitoring seems like testing, so decided to cover monitoring </li></ul>
  • 5. Where the name Assurer derived from ? <ul><li>Pronounce “A-shu-ra” </li></ul><ul><li>Plagger like tools ended with “er” </li></ul><ul><ul><li>Archer by tokuhirom </li></ul></ul><ul><ul><li>Observer by Hatena (not in public) </li></ul></ul><ul><ul><li>Dishuber by Yappo </li></ul></ul><ul><ul><li>Precure by Kan Fushihara </li></ul></ul><ul><li>Testing is also called “Quality Assurance” </li></ul>MISS
  • 6. Image of exec Assurer #0
  • 7. Image of exec Assurer #1
  • 8. Image of Asura Baster
  • 9. Image of the real Asura
  • 10. Image of fake Asura
  • 11. Execution phases of Assurer
  • 12. Execution phases of Assurer <ul><li>Test phase </li></ul><ul><ul><li>Execute test </li></ul></ul><ul><li>Nofity phase </li></ul><ul><ul><li>Notify test results </li></ul></ul><ul><li>Format phase </li></ul><ul><ul><li>Format test results </li></ul></ul><ul><li>Publish phase </li></ul><ul><ul><li>Output formatted test results </li></ul></ul>
  • 13. Relation of each phases Test Publish Format Nofity
  • 14. Details of each phases
  • 15. Test phase
  • 16. Test phase <ul><li>Execute test </li></ul><ul><li>Plugins on this phase: </li></ul><ul><ul><li>Test::HTTP </li></ul></ul><ul><ul><li>Test::SMTP </li></ul></ul><ul><ul><li>Test::DBI </li></ul></ul><ul><ul><li>Test::Ping </li></ul></ul><ul><ul><li>etc </li></ul></ul>
  • 17. Config of test phase <ul><li>test: </li></ul><ul><li>- module: HTTP </li></ul><ul><li>config: </li></ul><ul><li>host: www.mizzy.org </li></ul><ul><li>content: It works! </li></ul>
  • 18. Notify phase
  • 19. Notify フェーズ <ul><li>Notify test results </li></ul><ul><li>Plugin on this phase: </li></ul><ul><ul><li>Notify::IRC </li></ul></ul>
  • 20. Config of notify phase <ul><li>notify: </li></ul><ul><li>- module: IRC </li></ul><ul><li>config: </li></ul><ul><li>daemon_port: 9991 </li></ul><ul><li>nickname: assurerbot </li></ul><ul><li>server_host: chat.freenode.net </li></ul><ul><li>server_port: 6667 </li></ul><ul><li>server_channels: </li></ul><ul><li>- #assurer </li></ul><ul><li>- #assurer-jp </li></ul><ul><li>charset: iso-2022-jp </li></ul><ul><li>announce: notice </li></ul>
  • 21. Notify::IRC example
  • 22. Format phase
  • 23. Format phase <ul><li>Format test results </li></ul><ul><li>Plugins on this phase: </li></ul><ul><ul><li>Format::Text </li></ul></ul><ul><ul><li>Fomat::HTML </li></ul></ul>
  • 24. Config of format phase <ul><li>format: </li></ul><ul><li>- module: HTML </li></ul><ul><li>config: </li></ul><ul><li>css: /css/style.css </li></ul>
  • 25. Publish phase
  • 26. Publish phase <ul><li>Output formatted results </li></ul><ul><li>Plugins on this phase: </li></ul><ul><ul><li>Publish::Term </li></ul></ul><ul><ul><li>Publish::Mail </li></ul></ul><ul><ul><li>Publish::File </li></ul></ul>
  • 27. Config of publish phase <ul><li>publish: </li></ul><ul><li>- module: Mail </li></ul><ul><li>config: </li></ul><ul><li>subject: Test results from Assurer </li></ul><ul><li>to: someone@example.com </li></ul><ul><li>from: root@example.com </li></ul>publish: - module: Mail config: subject: Test results from Assurer to: someone@example.com from: root@example.com
  • 28. Format::Text + Publish::Term
  • 29. Format::HTML + Publish::File
  • 30. Format::Text + Publish::Mail
  • 31. Summary for now <ul><li>Assurer is a serverPlagger </li></ul><ul><li>Test, Notify, Format and Publish phases </li></ul><ul><li>Easily change tests and outputs with plugins </li></ul>
  • 32. Extra functions
  • 33. Hosts and Roles
  • 34. Hosts and roles <ul><li>Write target hosts and roles in one place on config.yaml </li></ul>
  • 35. Config without hosts and roles test: - module: HTTP config: host: www0.mizzy.org - module: HTTP config: host: www1.mizzy.org
  • 36. Config with hosts and roles test: - module: HTTP role: web hosts: web: - www0.mizzy.org - www1.mizzy.org
  • 37. Filter
  • 38. Filter <ul><li>Filtering test results on format and notify phase </li></ul><ul><li>Filtering formatted results on publish phase </li></ul>
  • 39. Filter on format phase <ul><li>Only OK results are formatted as text format </li></ul>format: - module: Text filter: module: Status status: ok
  • 40. Filter on notify phase <ul><li>Only NOK results are notified to IRC </li></ul>notify: - module: IRC filter: module: Status status: not ok
  • 41. Filter on publish phase <ul><li>Text formatted results are displayed on a terminal,html formatted results are sent as a mail </li></ul>format: - module: Text - module: HTML pusblish: - module: Term filter: module: Type type: text/plain - module: Mail filter: module: Type type: text/html
  • 42. Config validation
  • 43. Config validation <ul><li>Kwalify – YAML schema validator http://www.kuwata-lab.com/kwalify / </li></ul><ul><li>Validate common config </li></ul><ul><li>Validate config of each plugin </li></ul><ul><li>Kwalify schemas for each plugin put under assets/kwalify/plugins </li></ul>
  • 44. Mistake on common config test s : # test is right - module: Test::HTTP $ ./assurer.pl -c config.yaml Assurer::ConfigLoader [fatal] - [/] Expected required key `test‘ - [/tests] Unexpected key `tests' at line 46
  • 45. Mistake on plugin config test: - module: HTTP config: content s : XXX # content is right $ ./assurer.pl -c config.yaml Assurer::ConfigLoader [fatal] Config error in Test::HTTP - [/contents] Unexpected key `contents' at line 66
  • 46. Shell mode
  • 47. Shell mode <ul><li>Execute shell command on hosts in config.yaml </li></ul>
  • 48. Shell mode $ ./assurer.pl --shell assurer> uptime [www.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17 [svn.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17 [ftp.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17
  • 49. Exec on hosts belong to specified role $ ./assurer.pl --shell --role=web assurer> uptime [www.mizzy.org] 21:48:24 up 5 days, 23:08, 2 users, load average: 0.24, 0.16, 0.17
  • 50. Special command !on assurer> !on app1.foo.com app2.foo.com do uptime # Exec on app1.foo.com and app2.foo.com only assurer> !on /.*.foo.com/ do uptime # Exec on hosts matched .*.foo.com only
  • 51. Special command !with assurer> !with web db do uptime # exec on hosts belong to web role and db role assurer> !with /web|mail/ do uptime # exec on hosts belong to role match web or mail
  • 52. Special command !test assurer> !test HTTP Assurer::Plugin::Test::HTTP [info] Testing HTTP on www0.mizzy.org ... ok 1 - HTTP status code of http://mizzy.org:80 is 200 not ok 2 – Content of http://mizzy.org:80 matches 'It works!‘ assurer>
  • 53. Special command !test assurer> !test HTTP on app1.foo.com # test http on app1.foo.com assurer> !test HTTP on /.*.foo.com/ # can use regexp assurer> !test HTTP with web # test http on hosts belong to web role assurer> !test HTTP with /web|mail/ # can use regexp
  • 54. Parallel testing
  • 55. Pallarel testing <ul><li>Multiple test processes on test phase </li></ul><ul><li>Default is 5 </li></ul><ul><li>Can change number processes working at once </li></ul>$ assurer.pl --para=20
  • 56. Distributed testing
  • 57. Distributed testing localhost test0.mizzy.org www0.mizzy.org www1.mizzy.org www2.mizzy.org test1.mizzy.org test2.mizzy.org 1. ssh assurer_test.pl 3. Return result 2. Exec test
  • 58. Config for distributed testing exec_on: - host: test0.mizzy.org priority: 3 - host: test1.mizzy.org priority: 2 - host: test2.mizzy.org priority: 1
  • 59. Auto config generation
  • 60. Auto config generation <ul><li>Scan hosts in template.yaml by Nmap::Scanner </li></ul><ul><li>Detect services working on the hosts and generete config.yaml automatically </li></ul><ul><li>config templates for each plugin put under assets/discover (http.yaml, mysql.yaml, etc) </li></ul># assurer.pl --discover -c template.yaml
  • 61. Integragte Test::WWW::Mechanize
  • 62. Integrate Test::WWW::Mechanize <ul><li>Can write tests using Test::WWW::Mechanize </li></ul><ul><li>Put .pl files under assets/plugins/Test-WWW-Mechanize </li></ul><ul><li>Pre-set $mech, $host on execution </li></ul>
  • 63. asset file for Test::WWW::Mechanize # $host and $mech are set by Assure core $mech ->get_ok(&quot;http:// $host &quot;, &quot;got htttp:// $host &quot;); $mech ->content_contains('It works!', &quot;Content matches 'It works!'&quot;);
  • 64. Inside Assurer
  • 65. Assurer::Test module
  • 66. Assurer::Test module <ul><li>Export Test::More like methods </li></ul><ul><li>is,() like(), ok() </li></ul><ul><li>Output test results formatted as TAP (Test Analyze Protocol) </li></ul>
  • 67. Inside test plugins
  • 68. Inside test plugins package Assurer::Plugin::Test::HTTP; use base qw( Assurer::Plugin::Test ); use Assurer::Test; sub register { my $self = shift; $self->register_tests( qw/ status content server / ); } sub status { my ( $self, $context, $args ) = @_; ... 中略 is( $res->code, '200', 'HTTP status code of $self->{url} is 200' ); }
  • 69. Internal mechanism in test phase
  • 70. Internal mechanism in test phase assurer.pl Assurer::bootstrap() Assurer::Dispatch::run() call exec directly or by ssh exec tests return test results call generate result objects next phase assurer_test.pl assurer_test.pl assurer_test.pl assurer_test.pl assurer_test.pl Assurer::Result Assurer::Result Assurer::Result Assurer::Result Assurer::Result
  • 71. Assurer::Dispatch::run() <ul><li>Exec assurer_test.pl for each test </li></ul><ul><li>Pallarelize by POE::Wheel::Run </li></ul><ul><li>Number of processed controlled by POE::Component::JobQueue </li></ul><ul><li>Pass config for plugin and context object serialized by YAML and encoded by Base64 </li></ul>
  • 72. assurer_test.pl assurer_test.pl -– config = LS0tCmNvbmZpZzoKICBjb250ZW50OiBJ dCB3b3JrcyEKICBob3N0OiB3d3cubWl6enkub3JnCm1vZHVsZTogSFRUUApuYW1lOiBIVFRQIHRlc3QKcm9sZTogd2ViCg== - - context =LS0tICEhcGVybC9oYXNoOkFzc3VyZXIKYmFzZV9kaXI6IC9 ob21lL21peWEvc3ZrL0Fzc3VyZXIKY29uZmlnOgogIGZvcm1hdDoKICAgIC0gY29uZmlnOiAmMSB7fQogICAgICBtb2R1bGU6IFRleHQKICBnbG9iYWw6CiAgICBob3N0OiB+CiAgICBpbnRlcnZhbDogMwogICAgbG9nOgogICAgICBsZXZlbDogZGVidWcKICAgIG5vX2RpYWc6IDAKICAgIHJldHJ5OiAzCiAgaG9zdHM6CiAgICBmdHA6CiAgICAgIC0gZnRwLm1penp5Lm9yZwogICAgc3ZuOgogICAgICAtIHN2bi5taXp6eS5vcmcKICAgIHdlYjoKICAgICAgLSB3d3cubWl6enkub3JnCiAgICAgIC0gc3ZuLm1penp5Lm9yZwogICAgICAtIHRyYWMubWl6enkub3JnCiAgbm90aWZ5OgogICAgLSBjb25maWc6CiAgICAgICAgYW5ub3VuY2U6IG5vdGljZQogICAgICAgIGNoYXJzZXQ6IGlzby0yMDIyLWpwCiAgICAgICAgZGFlbW9uX3BvcnQ6IDk5OTEKICAgICAgICBuaWNrbmFtZTogYXNzdXJlcmJvdAog ... cont ...
  • 73. Why POE::Wheel::Run ? <ul><li>Would like to exec tests as other processes </li></ul><ul><ul><li>Need non-blocking I/O with single process and POE </li></ul></ul><ul><ul><li>Can easily distribute to remote servers (exec directly on local, exec by ssh on remote ) </li></ul></ul><ul><li>Communication with test processes </li></ul><ul><ul><li>Can communicate easily with test processes by STDIN, STDOUT and STDERR </li></ul></ul>
  • 74. Roadmap
  • 75. Roadmap <ul><li>Integrate with Nagios/NRPE (Test::Nagios) </li></ul><ul><li>Enhance notify phase </li></ul><ul><li>Store Plugin (store test results in database) </li></ul><ul><li>Integrate with Archer, puppet (share host lists) </li></ul><ul><li>Test of Assurer itsself </li></ul>
  • 76. Web site and IRC channels
  • 77. Web sites and IRC channels <ul><li>http://assurer.jp/trac </li></ul><ul><li>http://mizzy.org/ </li></ul><ul><li>#assurer@chat.freenode.net </li></ul><ul><li>#assurer-ja@chat.freenode.net </li></ul>
  • 78. Thanks

×