Smolder And Buildbot

  • 4,177 views
Uploaded on

Using Buildbot and Smolder together to play to both of their strengths

Using Buildbot and Smolder together to play to both of their strengths

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
4,177
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
16
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. What is Smolder? ● Will it automate my build/test cycle? – No. ● Will it run my test suite? – No. ● Will it checkout my code from svn, run my build and then run my test suite? – Hello! Smolder takes the information about a test run and makes it pretty. Michael Peters, Plus Three LP (http://plusthree.com)
  • 2. History of Smolder ● Plus Three, LP ● Long running test suites for multiple projects – 5,460 tests in 28 minutes (510K HTML report) – 18,000+ tests in 50 minutes ● Stole lots of ideas from the Pugs smoke server. ● Added lot – Comments, tags, invalidation – Pretty graphs – Multiple projects (public and private) – Atom data feeds Michael Peters, Plus Three LP (http://plusthree.com)
  • 3. TAP ● Test Anything Protocol ● Created by Test::Building, Test::More, etc ● Consumed by Test::Harness, prove, etc 1..5 ok 1 – foo is fine ok 2 – testing bar value not ok 3 – The object isa Frobnicate # this is some useful # comment thing not ok 4 # SKIP Foo::Bar is not instal led not ok 5 # TODO future feature Michael Peters, Plus Three LP (http://plusthree.com)
  • 4. TAP ● How do you create it? use Test::More (tests => 5) ok($foo, 'foo is fine'); is($bar, 'right', 'testing bar value'); isa_ok($bar, 'Frobnicate'); diag(“this is some usefulncomment thing”); SKIP: {   skip('Foo::Bar is not installed' 1);   ok($foo­>bar, 'has a bar'); } TODO: {   local $TODO = 'future feature';   ok($foo­>feature, 'some feature'); } Michael Peters, Plus Three LP (http://plusthree.com)
  • 5. TAP ● Started in Perl-Land ● We have since created colonies – C – PHP – Python – Java – JavaScript – Parrot – Postgres – .... more Michael Peters, Plus Three LP (http://plusthree.com)
  • 6. TAP Archives ● A single file (.tar or .tar.gz) ● Multiple recorded TAP streams (files) ● Some meta information (META.yml) – Stream (test) order – Start/Stop times – Future expansion (server info, log files, etc) ● Easy to create with TAP::Harness::Archive – prove -a Michael Peters, Plus Three LP (http://plusthree.com)
  • 7. Continuous Integration Cycle ● Automated builds – Simple bash script #!/bin/bash if svn update | grep 'Updated to' then make build && make test else echo quot;No updatesquot; fi ● Get fancier – SmokeRunner::Multi – Custom harness written with Test::Harness – BuildBot Michael Peters, Plus Three LP (http://plusthree.com)
  • 8. Buildbot ● Most projects don't have multi-platform, multi- architecture needs – Anything written in pure Perl – Most $work applications ● Some projects do – Languages (Perl, Python) – VMs (Parrot) – System Tools (SCM, etc) Michael Peters, Plus Three LP (http://plusthree.com)
  • 9. What is Buildbot? ● Will it automate my build/test cycle? – Yes. ● Will it run my test suite? – Yes. ● Will it checkout my code from svn, run my build and then run my test suite? – Absolutely! Michael Peters, Plus Three LP (http://plusthree.com)
  • 10. Other Buildbot Spiffyness ● Master configuration, multiple slaves do the work – Lots of architectures and platforms ● Arbitrary build commands – Not a language specific system – Doesn't care if you use make, Build.PL, ant, maven ● Portable – It's Python and works in lots of places despite all the whitespace ● Very configurable Michael Peters, Plus Three LP (http://plusthree.com)
  • 11. So Why Do I Need Smolder? ● Buildbot gives just a single pass/fail for each run ● All other details are just dumped to the screen ● No good way to organize your test runs ● .... Michael Peters, Plus Three LP (http://plusthree.com)
  • 12. So Why Do I Need Smolder? ● It's UGLY! Michael Peters, Plus Three LP (http://plusthree.com)
  • 13. So Why Do I Need Smolder? ● Oooh-la-la! Michael Peters, Plus Three LP (http://plusthree.com)
  • 14. So Why Do I Need Smolder? Buildbot => Smolder Michael Peters, Plus Three LP (http://plusthree.com)
  • 15. So Why Do I Need Smolder? Buildbot => Smolder => Michael Peters, Plus Three LP (http://plusthree.com)
  • 16. So Why Do I Need Smolder? ● Information is more organized – Access to the raw TAP Archives – Search interface and Graphs ● Let's you answer questions like – When was the last time our test suite passed on platform X? – Who broke the test suite? – How many tests have we added for this release? – How did checkin X affect the speed of the test suite? Michael Peters, Plus Three LP (http://plusthree.com)
  • 17. Buildbot Talking to Smolder ● Remember Buildbot allows arbitrary build commands ● So your project just needs a command that can run the suite, create a TAP archive and send it to a Smolder server – prove + smolder_smoke_signal – make smoke_tests ● Buildbot doesn't know that it's being cheated on! Michael Peters, Plus Three LP (http://plusthree.com)
  • 18. Scary Live Demos! ● Parrot ● Smolder Michael Peters, Plus Three LP (http://plusthree.com)
  • 19. Smolder's Future ● Test Coverage ● Tracking individual test assertions ● Better visual display of diagnostics (new TAP extensions) ● More and better ways to search old reports Michael Peters, Plus Three LP (http://plusthree.com)