What is Smolder?
●
    Will it automate my build/test cycle?
    –   No.
●
    Will it run my test suite?
    –   No.
●
  ...
History of Smolder
●
    Plus Three, LP
●
    Long running test suites for multiple projects
    –   5,460 tests in 28 min...
TAP
●
    Test Anything Protocol
●
    Created by Test::Building, Test::More, etc
●
    Consumed by Test::Harness, prove, ...
TAP
●
    How do you create it?
      use Test::More (tests => 5)
      ok($foo, 'foo is fine');
      is($bar, 'right', '...
TAP
●
    Started in Perl-Land
●
    We have since created colonies
    –   C
    –   PHP
    –   Python
    –   Java
    ...
TAP Archives
●
    A single file (.tar or .tar.gz)
●
    Multiple recorded TAP streams (files)
●
    Some meta information...
Continuous Integration Cycle
●
    Automated builds
    –   Simple bash script
        #!/bin/bash
        if svn update |...
Buildbot
●
    Most projects don't have multi-platform, multi-
    architecture needs
    –   Anything written in pure Per...
What is Buildbot?
●
    Will it automate my build/test cycle?
    –   Yes.
●
    Will it run my test suite?
    –   Yes.
●...
Other Buildbot Spiffyness
●
    Master configuration, multiple slaves do the
    work
    –   Lots of architectures and pl...
So Why Do I Need Smolder?
●
    Buildbot gives just a single pass/fail for each
    run
●
    All other details are just d...
So Why Do I Need Smolder?
●
    It's UGLY!




Michael Peters, Plus Three LP (http://plusthree.com)
So Why Do I Need Smolder?
●
    Oooh-la-la!




Michael Peters, Plus Three LP (http://plusthree.com)
So Why Do I Need Smolder?

   Buildbot                             =>             Smolder




Michael Peters, Plus Three L...
So Why Do I Need Smolder?

   Buildbot                             =>             Smolder


                              ...
So Why Do I Need Smolder?
●
    Information is more organized
    –   Access to the raw TAP Archives
    –   Search interf...
Buildbot Talking to Smolder
●
    Remember Buildbot allows arbitrary build
    commands
●
    So your project just needs a...
Scary Live Demos!
●
    Parrot
●
    Smolder




Michael Peters, Plus Three LP (http://plusthree.com)
Smolder's Future
●
    Test Coverage
●
    Tracking individual test assertions
●
    Better visual display of diagnostics ...
Smolder And Buildbot
Upcoming SlideShare
Loading in...5
×

Smolder And Buildbot

6,145

Published on

Using Buildbot and Smolder together to play to both of their strengths

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
6,145
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
17
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Smolder And Buildbot"

  1. 1. What is Smolder? ● Will it automate my build/test cycle? – No. ● Will it run my test suite? – No. ● Will it checkout my code from svn, run my build and then run my test suite? – Hello! Smolder takes the information about a test run and makes it pretty. Michael Peters, Plus Three LP (http://plusthree.com)
  2. 2. History of Smolder ● Plus Three, LP ● Long running test suites for multiple projects – 5,460 tests in 28 minutes (510K HTML report) – 18,000+ tests in 50 minutes ● Stole lots of ideas from the Pugs smoke server. ● Added lot – Comments, tags, invalidation – Pretty graphs – Multiple projects (public and private) – Atom data feeds Michael Peters, Plus Three LP (http://plusthree.com)
  3. 3. TAP ● Test Anything Protocol ● Created by Test::Building, Test::More, etc ● Consumed by Test::Harness, prove, etc 1..5 ok 1 – foo is fine ok 2 – testing bar value not ok 3 – The object isa Frobnicate # this is some useful # comment thing not ok 4 # SKIP Foo::Bar is not instal led not ok 5 # TODO future feature Michael Peters, Plus Three LP (http://plusthree.com)
  4. 4. TAP ● How do you create it? use Test::More (tests => 5) ok($foo, 'foo is fine'); is($bar, 'right', 'testing bar value'); isa_ok($bar, 'Frobnicate'); diag(“this is some usefulncomment thing”); SKIP: {   skip('Foo::Bar is not installed' 1);   ok($foo­>bar, 'has a bar'); } TODO: {   local $TODO = 'future feature';   ok($foo­>feature, 'some feature'); } Michael Peters, Plus Three LP (http://plusthree.com)
  5. 5. TAP ● Started in Perl-Land ● We have since created colonies – C – PHP – Python – Java – JavaScript – Parrot – Postgres – .... more Michael Peters, Plus Three LP (http://plusthree.com)
  6. 6. TAP Archives ● A single file (.tar or .tar.gz) ● Multiple recorded TAP streams (files) ● Some meta information (META.yml) – Stream (test) order – Start/Stop times – Future expansion (server info, log files, etc) ● Easy to create with TAP::Harness::Archive – prove -a Michael Peters, Plus Three LP (http://plusthree.com)
  7. 7. Continuous Integration Cycle ● Automated builds – Simple bash script #!/bin/bash if svn update | grep 'Updated to' then make build && make test else echo quot;No updatesquot; fi ● Get fancier – SmokeRunner::Multi – Custom harness written with Test::Harness – BuildBot Michael Peters, Plus Three LP (http://plusthree.com)
  8. 8. Buildbot ● Most projects don't have multi-platform, multi- architecture needs – Anything written in pure Perl – Most $work applications ● Some projects do – Languages (Perl, Python) – VMs (Parrot) – System Tools (SCM, etc) Michael Peters, Plus Three LP (http://plusthree.com)
  9. 9. What is Buildbot? ● Will it automate my build/test cycle? – Yes. ● Will it run my test suite? – Yes. ● Will it checkout my code from svn, run my build and then run my test suite? – Absolutely! Michael Peters, Plus Three LP (http://plusthree.com)
  10. 10. Other Buildbot Spiffyness ● Master configuration, multiple slaves do the work – Lots of architectures and platforms ● Arbitrary build commands – Not a language specific system – Doesn't care if you use make, Build.PL, ant, maven ● Portable – It's Python and works in lots of places despite all the whitespace ● Very configurable Michael Peters, Plus Three LP (http://plusthree.com)
  11. 11. So Why Do I Need Smolder? ● Buildbot gives just a single pass/fail for each run ● All other details are just dumped to the screen ● No good way to organize your test runs ● .... Michael Peters, Plus Three LP (http://plusthree.com)
  12. 12. So Why Do I Need Smolder? ● It's UGLY! Michael Peters, Plus Three LP (http://plusthree.com)
  13. 13. So Why Do I Need Smolder? ● Oooh-la-la! Michael Peters, Plus Three LP (http://plusthree.com)
  14. 14. So Why Do I Need Smolder? Buildbot => Smolder Michael Peters, Plus Three LP (http://plusthree.com)
  15. 15. So Why Do I Need Smolder? Buildbot => Smolder => Michael Peters, Plus Three LP (http://plusthree.com)
  16. 16. So Why Do I Need Smolder? ● Information is more organized – Access to the raw TAP Archives – Search interface and Graphs ● Let's you answer questions like – When was the last time our test suite passed on platform X? – Who broke the test suite? – How many tests have we added for this release? – How did checkin X affect the speed of the test suite? Michael Peters, Plus Three LP (http://plusthree.com)
  17. 17. Buildbot Talking to Smolder ● Remember Buildbot allows arbitrary build commands ● So your project just needs a command that can run the suite, create a TAP archive and send it to a Smolder server – prove + smolder_smoke_signal – make smoke_tests ● Buildbot doesn't know that it's being cheated on! Michael Peters, Plus Three LP (http://plusthree.com)
  18. 18. Scary Live Demos! ● Parrot ● Smolder Michael Peters, Plus Three LP (http://plusthree.com)
  19. 19. Smolder's Future ● Test Coverage ● Tracking individual test assertions ● Better visual display of diagnostics (new TAP extensions) ● More and better ways to search old reports Michael Peters, Plus Three LP (http://plusthree.com)
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×