Your SlideShare is downloading. ×
Smolder And Buildbot
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Smolder And Buildbot

4,951
views

Published on

Using Buildbot and Smolder together to play to both of their strengths

Using Buildbot and Smolder together to play to both of their strengths

Published in: Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,951
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
16
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. What is Smolder? ● Will it automate my build/test cycle? – No. ● Will it run my test suite? – No. ● Will it checkout my code from svn, run my build and then run my test suite? – Hello! Smolder takes the information about a test run and makes it pretty. Michael Peters, Plus Three LP (http://plusthree.com)
  • 2. History of Smolder ● Plus Three, LP ● Long running test suites for multiple projects – 5,460 tests in 28 minutes (510K HTML report) – 18,000+ tests in 50 minutes ● Stole lots of ideas from the Pugs smoke server. ● Added lot – Comments, tags, invalidation – Pretty graphs – Multiple projects (public and private) – Atom data feeds Michael Peters, Plus Three LP (http://plusthree.com)
  • 3. TAP ● Test Anything Protocol ● Created by Test::Building, Test::More, etc ● Consumed by Test::Harness, prove, etc 1..5 ok 1 – foo is fine ok 2 – testing bar value not ok 3 – The object isa Frobnicate # this is some useful # comment thing not ok 4 # SKIP Foo::Bar is not instal led not ok 5 # TODO future feature Michael Peters, Plus Three LP (http://plusthree.com)
  • 4. TAP ● How do you create it? use Test::More (tests => 5) ok($foo, 'foo is fine'); is($bar, 'right', 'testing bar value'); isa_ok($bar, 'Frobnicate'); diag(“this is some usefulncomment thing”); SKIP: {   skip('Foo::Bar is not installed' 1);   ok($foo­>bar, 'has a bar'); } TODO: {   local $TODO = 'future feature';   ok($foo­>feature, 'some feature'); } Michael Peters, Plus Three LP (http://plusthree.com)
  • 5. TAP ● Started in Perl-Land ● We have since created colonies – C – PHP – Python – Java – JavaScript – Parrot – Postgres – .... more Michael Peters, Plus Three LP (http://plusthree.com)
  • 6. TAP Archives ● A single file (.tar or .tar.gz) ● Multiple recorded TAP streams (files) ● Some meta information (META.yml) – Stream (test) order – Start/Stop times – Future expansion (server info, log files, etc) ● Easy to create with TAP::Harness::Archive – prove -a Michael Peters, Plus Three LP (http://plusthree.com)
  • 7. Continuous Integration Cycle ● Automated builds – Simple bash script #!/bin/bash if svn update | grep 'Updated to' then make build && make test else echo quot;No updatesquot; fi ● Get fancier – SmokeRunner::Multi – Custom harness written with Test::Harness – BuildBot Michael Peters, Plus Three LP (http://plusthree.com)
  • 8. Buildbot ● Most projects don't have multi-platform, multi- architecture needs – Anything written in pure Perl – Most $work applications ● Some projects do – Languages (Perl, Python) – VMs (Parrot) – System Tools (SCM, etc) Michael Peters, Plus Three LP (http://plusthree.com)
  • 9. What is Buildbot? ● Will it automate my build/test cycle? – Yes. ● Will it run my test suite? – Yes. ● Will it checkout my code from svn, run my build and then run my test suite? – Absolutely! Michael Peters, Plus Three LP (http://plusthree.com)
  • 10. Other Buildbot Spiffyness ● Master configuration, multiple slaves do the work – Lots of architectures and platforms ● Arbitrary build commands – Not a language specific system – Doesn't care if you use make, Build.PL, ant, maven ● Portable – It's Python and works in lots of places despite all the whitespace ● Very configurable Michael Peters, Plus Three LP (http://plusthree.com)
  • 11. So Why Do I Need Smolder? ● Buildbot gives just a single pass/fail for each run ● All other details are just dumped to the screen ● No good way to organize your test runs ● .... Michael Peters, Plus Three LP (http://plusthree.com)
  • 12. So Why Do I Need Smolder? ● It's UGLY! Michael Peters, Plus Three LP (http://plusthree.com)
  • 13. So Why Do I Need Smolder? ● Oooh-la-la! Michael Peters, Plus Three LP (http://plusthree.com)
  • 14. So Why Do I Need Smolder? Buildbot => Smolder Michael Peters, Plus Three LP (http://plusthree.com)
  • 15. So Why Do I Need Smolder? Buildbot => Smolder => Michael Peters, Plus Three LP (http://plusthree.com)
  • 16. So Why Do I Need Smolder? ● Information is more organized – Access to the raw TAP Archives – Search interface and Graphs ● Let's you answer questions like – When was the last time our test suite passed on platform X? – Who broke the test suite? – How many tests have we added for this release? – How did checkin X affect the speed of the test suite? Michael Peters, Plus Three LP (http://plusthree.com)
  • 17. Buildbot Talking to Smolder ● Remember Buildbot allows arbitrary build commands ● So your project just needs a command that can run the suite, create a TAP archive and send it to a Smolder server – prove + smolder_smoke_signal – make smoke_tests ● Buildbot doesn't know that it's being cheated on! Michael Peters, Plus Three LP (http://plusthree.com)
  • 18. Scary Live Demos! ● Parrot ● Smolder Michael Peters, Plus Three LP (http://plusthree.com)
  • 19. Smolder's Future ● Test Coverage ● Tracking individual test assertions ● Better visual display of diagnostics (new TAP extensions) ● More and better ways to search old reports Michael Peters, Plus Three LP (http://plusthree.com)