Smolder And Buildbot

7,371 views
7,257 views

Published on

Using Buildbot and Smolder together to play to both of their strengths

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,371
On SlideShare
0
From Embeds
0
Number of Embeds
15
Actions
Shares
0
Downloads
17
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Smolder And Buildbot

  1. 1. What is Smolder? ● Will it automate my build/test cycle? – No. ● Will it run my test suite? – No. ● Will it checkout my code from svn, run my build and then run my test suite? – Hello! Smolder takes the information about a test run and makes it pretty. Michael Peters, Plus Three LP (http://plusthree.com)
  2. 2. History of Smolder ● Plus Three, LP ● Long running test suites for multiple projects – 5,460 tests in 28 minutes (510K HTML report) – 18,000+ tests in 50 minutes ● Stole lots of ideas from the Pugs smoke server. ● Added lot – Comments, tags, invalidation – Pretty graphs – Multiple projects (public and private) – Atom data feeds Michael Peters, Plus Three LP (http://plusthree.com)
  3. 3. TAP ● Test Anything Protocol ● Created by Test::Building, Test::More, etc ● Consumed by Test::Harness, prove, etc 1..5 ok 1 – foo is fine ok 2 – testing bar value not ok 3 – The object isa Frobnicate # this is some useful # comment thing not ok 4 # SKIP Foo::Bar is not instal led not ok 5 # TODO future feature Michael Peters, Plus Three LP (http://plusthree.com)
  4. 4. TAP ● How do you create it? use Test::More (tests => 5) ok($foo, 'foo is fine'); is($bar, 'right', 'testing bar value'); isa_ok($bar, 'Frobnicate'); diag(“this is some usefulncomment thing”); SKIP: {   skip('Foo::Bar is not installed' 1);   ok($foo­>bar, 'has a bar'); } TODO: {   local $TODO = 'future feature';   ok($foo­>feature, 'some feature'); } Michael Peters, Plus Three LP (http://plusthree.com)
  5. 5. TAP ● Started in Perl-Land ● We have since created colonies – C – PHP – Python – Java – JavaScript – Parrot – Postgres – .... more Michael Peters, Plus Three LP (http://plusthree.com)
  6. 6. TAP Archives ● A single file (.tar or .tar.gz) ● Multiple recorded TAP streams (files) ● Some meta information (META.yml) – Stream (test) order – Start/Stop times – Future expansion (server info, log files, etc) ● Easy to create with TAP::Harness::Archive – prove -a Michael Peters, Plus Three LP (http://plusthree.com)
  7. 7. Continuous Integration Cycle ● Automated builds – Simple bash script #!/bin/bash if svn update | grep 'Updated to' then make build && make test else echo quot;No updatesquot; fi ● Get fancier – SmokeRunner::Multi – Custom harness written with Test::Harness – BuildBot Michael Peters, Plus Three LP (http://plusthree.com)
  8. 8. Buildbot ● Most projects don't have multi-platform, multi- architecture needs – Anything written in pure Perl – Most $work applications ● Some projects do – Languages (Perl, Python) – VMs (Parrot) – System Tools (SCM, etc) Michael Peters, Plus Three LP (http://plusthree.com)
  9. 9. What is Buildbot? ● Will it automate my build/test cycle? – Yes. ● Will it run my test suite? – Yes. ● Will it checkout my code from svn, run my build and then run my test suite? – Absolutely! Michael Peters, Plus Three LP (http://plusthree.com)
  10. 10. Other Buildbot Spiffyness ● Master configuration, multiple slaves do the work – Lots of architectures and platforms ● Arbitrary build commands – Not a language specific system – Doesn't care if you use make, Build.PL, ant, maven ● Portable – It's Python and works in lots of places despite all the whitespace ● Very configurable Michael Peters, Plus Three LP (http://plusthree.com)
  11. 11. So Why Do I Need Smolder? ● Buildbot gives just a single pass/fail for each run ● All other details are just dumped to the screen ● No good way to organize your test runs ● .... Michael Peters, Plus Three LP (http://plusthree.com)
  12. 12. So Why Do I Need Smolder? ● It's UGLY! Michael Peters, Plus Three LP (http://plusthree.com)
  13. 13. So Why Do I Need Smolder? ● Oooh-la-la! Michael Peters, Plus Three LP (http://plusthree.com)
  14. 14. So Why Do I Need Smolder? Buildbot => Smolder Michael Peters, Plus Three LP (http://plusthree.com)
  15. 15. So Why Do I Need Smolder? Buildbot => Smolder => Michael Peters, Plus Three LP (http://plusthree.com)
  16. 16. So Why Do I Need Smolder? ● Information is more organized – Access to the raw TAP Archives – Search interface and Graphs ● Let's you answer questions like – When was the last time our test suite passed on platform X? – Who broke the test suite? – How many tests have we added for this release? – How did checkin X affect the speed of the test suite? Michael Peters, Plus Three LP (http://plusthree.com)
  17. 17. Buildbot Talking to Smolder ● Remember Buildbot allows arbitrary build commands ● So your project just needs a command that can run the suite, create a TAP archive and send it to a Smolder server – prove + smolder_smoke_signal – make smoke_tests ● Buildbot doesn't know that it's being cheated on! Michael Peters, Plus Three LP (http://plusthree.com)
  18. 18. Scary Live Demos! ● Parrot ● Smolder Michael Peters, Plus Three LP (http://plusthree.com)
  19. 19. Smolder's Future ● Test Coverage ● Tracking individual test assertions ● Better visual display of diagnostics (new TAP extensions) ● More and better ways to search old reports Michael Peters, Plus Three LP (http://plusthree.com)

×