Turbo Charged Test Suites
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Turbo Charged Test Suites

  • 5,236 views
Uploaded on

Techniques for Perl programmers to speed up test suite and developer performance.

Techniques for Perl programmers to speed up test suite and developer performance.

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • It's worth noting that the slides regarding rebuilding tables and avoiding transactions assume that your test suite is constrained to run in a single process. If you parallelize your test suite, transactions are recommended.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
5,236
On Slideshare
4,048
From Embeds
1,188
Number of Embeds
16

Actions

Shares
Downloads
34
Comments
1
Likes
4

Embeds 1,188

http://blogs.perl.org 1,055
http://feedly.com 83
http://digg.com 10
http://www.linkedin.com 8
http://newsblur.com 6
http://www.tuicool.com 5
http://www.feedspot.com 4
http://cloud.feedly.com 3
http://news.int80.biz 3
http://www.newsblur.com 2
http://www.inoreader.com 2
http://www.slideshare.net 2
https://www.linkedin.com 2
http://localhost 1
http://www.docshut.com 1
http://inoreader.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Turbo Charged Test Suites Tuesday, 7 October 2008
  • 2. Curtis “Ovid” Poe • CPAN Author • “Perl Hacks” Contributor • Perl Foundation Member Tuesday, 7 October 2008
  • 3. Test Suite Performance • Computer • Developer Tuesday, 7 October 2008
  • 4. Faster Tests Tuesday, 7 October 2008
  • 5. Considerations • Why speed things up? • Trade-offs Tuesday, 7 October 2008
  • 6. Before You Start • Set a goal • No tests may fail • No non-test output Tuesday, 7 October 2008
  • 7. Databases • Don’t drop them • Don’t fake it with transactions Tuesday, 7 October 2008
  • 8. Databases • Static tables -- country_codes • Dynamic tables -- orders Tuesday, 7 October 2008
  • 9. CREATE TABLE changed_table ( changed_table_id INT(11) NOT NULL AUTO_INCREMENT PRIMARY KEY, table_name VARCHAR(30) NOT NULL, is_static INT NOT NULL DEFAULT 0, inserts INT NOT NULL DEFAULT 0, updates INT NOT NULL DEFAULT 0, deletes INT NOT NULL DEFAULT 0 ) -- then insert table records Tuesday, 7 October 2008
  • 10. foreach my $action (qw/insert update delete/) { $dbh->do(<<quot; END_SQLquot;); CREATE TRIGGER tr_${action}_$table BEFORE $action ON $table FOR EACH ROW UPDATE changed_table SET ${action}s = ${action}s + 1 WHERE table_name = '$table'; END_SQL } Tuesday, 7 October 2008
  • 11. db.disable_foreign_keys() for table in changed_tables() table.truncate() if table.is_static() table.rebuild() db.enable_foreign_keys() Tuesday, 7 October 2008
  • 12. Databases • Before: 80 minutes • After: 22 minutes Tuesday, 7 October 2008
  • 13. Aggregation • Only load things once • Dangers of shared state Tuesday, 7 October 2008
  • 14. use My::YAML::Test; My::YAML::Test->run; # t/api/customer.t # t/api/customer.yml Tuesday, 7 October 2008
  • 15. use My::YAML::Test; foreach (yaml_tests()) { diag “running $_”; My::YAML::Test->run($_); } sub yaml_tests { return @ARGV if @ARGV; # or return all YAML files ... } Tuesday, 7 October 2008
  • 16. YAML Aggregation • Before: 22 minutes • After: 16 minutes Tuesday, 7 October 2008
  • 17. use Test::Aggregate; my $tests = Test::Aggregate->new({ dirs => ‘aggtests’, }); $tests->run; Tuesday, 7 October 2008
  • 18. use Test::Aggregate; my $tests = Test::Aggregate->new({ dirs => 'aggtests', dump => $dump, set_filenames => 1, shuffle => 1, startup => &startup, shutdown => &shutdown, }); Tuesday, 7 October 2008
  • 19. Generic Aggregation • Before: 16 minutes • After: 12 minutes Tuesday, 7 October 2008
  • 20. Better Aggregation • use Test::Class Tuesday, 7 October 2008
  • 21. OK to Fail is OK • POD tests • Perl::Critic • Anything “non-functional” Tuesday, 7 October 2008
  • 22. OK to Fail is OK • Move them to xt/ • But they must be run! Tuesday, 7 October 2008
  • 23. Lessons Learned • Not dropping your database is tricky • Aggregating tests finds bugs in tests • Aggregating tests finds bugs in code • *CORE::GLOBAL:: is evil • Don’t touch UNIVERSAL:: Tuesday, 7 October 2008
  • 24. Faster Programmers Tuesday, 7 October 2008
  • 25. Custom Test Modules Tuesday, 7 October 2008
  • 26. use Test::More tests => 13; use Test::Exception; use Test::XML; use Test::JSON; use Test::Differences; Tuesday, 7 October 2008
  • 27. package Our::Test::More; use Test::Builder::Module; our ( @ISA, @EXPORT ); use Test::More; use Test::Exception; BEGIN { @ISA = qw(Test::Builder::Module); @EXPORT = ( @Test::More::EXPORT, @Test::Exception::EXPORT, ); } 1; Tuesday, 7 October 2008
  • 28. package My::Custom::Tests; use Test::Kit ( 'Test::More', 'Test::XML', 'Test::Differences', '+explain', ); 1; Tuesday, 7 October 2008
  • 29. Popularity Contest Test::More 44461 Test 8937 Test::Exception 1397 Test::Simple 731 Test::Base 316 Test::Builder::Tester 193 Test::NoWarnings 174 Test::Differences 146 Test::MockObject 139 Test::Deep 127 Tuesday, 7 October 2008
  • 30. use Test::Most tests => 4, 'die'; ok 1, 'one is true'; is 2, 2, '... and two is two'; eq_or_diff [3], [4], “... but three ain’t four”; throws_ok { $foo/0 } qr/Illegal division by zero/, 'and no-constant folding with vars'; Tuesday, 7 October 2008
  • 31. Test From Your Editor Tuesday, 7 October 2008
  • 32. “ in your .vimrc function! PerlMappings() noremap K :!perldoc <cword> <bar><bar> perldoc -f <cword><cr> endfunction function! PerlTestMappings() noremap <buffer> ,t :!prove -vl %<CR> endfunction “ remember My::Test::YAML? function! YAMLTestMappings() noremap <buffer> ,t :!prove -vl t/yaml.t :: %<CR> endfunction au! FileType perl :call PerlMappings() au! FileType yaml :call YAMLTestMappings() au! BufRead,BufNewFile *.t :call PerlTestMappings() Tuesday, 7 October 2008
  • 33. vim $(ack -l --perl 'api/v1/episode' t/) map <leader>tb :call RunTestsInBuffers()<cr> function! RunTestsInBuffers() let i = 1 let tests = '' while (i <= bufnr(quot;$quot;)) let filename = bufname(i) if match(filename, '.t$') > -1 let tests = tests . ' quot;' . filename . 'quot;' endif let i = i+1 endwhile if !strlen(tests) echo quot;No tests found in buffersquot; else execute ':!prove ' . tests endif endfunction Tuesday, 7 October 2008
  • 34. Advanced “prove” Tuesday, 7 October 2008
  • 35. Test-Harness $ prove -l t --state=hot,fast,save --timer [20:57:19] t/yamlish-output.........ok 40 ms [20:57:20] t/console................ok 45 ms [20:57:20] t/utils..................ok 48 ms <snip> [20:57:23] t/harness................ok 300 ms [20:57:23] t/process................ok 1020 ms [20:57:24] t/prove..................ok 1017 ms [20:57:25] t/regression.............ok 4217 ms [20:57:29] All tests successful. Files=32, Tests=10326, 10 wallclock secs ( 1.25 usr 0.24 sys + 5.60 cusr 1.83 csys = 8.92 CPU) Result: PASS Tuesday, 7 October 2008
  • 36. Test-Harness $ prove -l t --state=hot,slow,save -j 9 t/callbacks.............. ok t/nofork................. ok t/proverc................ ok <snip> t/yamlish................ ok t/prove.................. ok t/regression............. ok All tests successful. Files=32, Tests=10326, 6 wallclock secs ( 1.34 usr 0.24 sys + 5.63 cusr 1.83 csys = 9.04 CPU) Result: PASS Tuesday, 7 October 2008
  • 37. # slow running tests from App::Prove::State # http://use.perl.org/~Ovid/journal/35831 Generation 18 Number of test programs: 58 Total runtime approximately 17 minutes 35 seconds Five slowest tests: 482.7 seconds -> t/acceptance.t 234.4 seconds -> t/aggregate.t 96.3 seconds -> t/standards/strict.t 66.6 seconds -> t/unit/db/migrations.t 56.7 seconds -> t/unit/piptest/pprove/testdb.t Tuesday, 7 October 2008
  • 38. Devel::CoverX::Covered Tuesday, 7 October 2008
  • 39. Devel::CoverX::Covered • What tests cover this file? • What files are covered by this test? • (Soon) What tests cover this line? • (Soon) What tests cover this subroutine? • ... and more ... Tuesday, 7 October 2008
  • 40. function! PerlMappings() noremap <buffer> ,cv :call Coverage()<cr> endfunction function! PerlTestMappings() noremap <buffer> ,t :!prove -vl --norc %<CR> endfunction function! Coverage() let file = bufname('%') if match(filename, '.t$') > -1 execute '!covered by --test_file=quot;'.file.'quot;' else execute '!covered covering --source_file=quot;'.file.'quot;' end endfunction au! FileType perl :call PerlMappings() au! BufRead,BufNewFile *.t :call PerlTestMappings() Tuesday, 7 October 2008
  • 41. Questions? Tuesday, 7 October 2008