Turbo Charged Test Suites

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

0 comments

Post a comment

    Post a comment
    Embed Video
    Edit your comment Cancel

    1 Favorite & 1 Group

    Turbo Charged Test Suites - Presentation Transcript

    1. Turbo Charged Test Suites Tuesday, 7 October 2008
    2. Curtis “Ovid” Poe • CPAN Author • “Perl Hacks” Contributor • Perl Foundation Member Tuesday, 7 October 2008
    3. Test Suite Performance • Computer • Developer Tuesday, 7 October 2008
    4. Faster Tests Tuesday, 7 October 2008
    5. Considerations • Why speed things up? • Trade-offs Tuesday, 7 October 2008
    6. Before You Start • Set a goal • No tests may fail • No non-test output Tuesday, 7 October 2008
    7. Databases • Don’t drop them • Don’t fake it with transactions Tuesday, 7 October 2008
    8. Databases • Static tables -- country_codes • Dynamic tables -- orders Tuesday, 7 October 2008
    9. CREATE TABLE changed_table ( changed_table_id INT(11) NOT NULL AUTO_INCREMENT PRIMARY KEY, table_name VARCHAR(30) NOT NULL, is_static INT NOT NULL DEFAULT 0, inserts INT NOT NULL DEFAULT 0, updates INT NOT NULL DEFAULT 0, deletes INT NOT NULL DEFAULT 0 ) -- then insert table records Tuesday, 7 October 2008
    10. foreach my $action (qw/insert update delete/) { $dbh->do(<<\" END_SQL\"); CREATE TRIGGER tr_${action}_$table BEFORE $action ON $table FOR EACH ROW UPDATE changed_table SET ${action}s = ${action}s + 1 WHERE table_name = '$table'; END_SQL } Tuesday, 7 October 2008
    11. db.disable_foreign_keys() for table in changed_tables() table.truncate() if table.is_static() table.rebuild() db.enable_foreign_keys() Tuesday, 7 October 2008
    12. Databases • Before: 80 minutes • After: 22 minutes Tuesday, 7 October 2008
    13. Aggregation • Only load things once • Dangers of shared state Tuesday, 7 October 2008
    14. use My::YAML::Test; My::YAML::Test->run; # t/api/customer.t # t/api/customer.yml Tuesday, 7 October 2008
    15. use My::YAML::Test; foreach (yaml_tests()) { diag “running $_”; My::YAML::Test->run($_); } sub yaml_tests { return @ARGV if @ARGV; # or return all YAML files ... } Tuesday, 7 October 2008
    16. YAML Aggregation • Before: 22 minutes • After: 16 minutes Tuesday, 7 October 2008
    17. use Test::Aggregate; my $tests = Test::Aggregate->new({ dirs => ‘aggtests’, }); $tests->run; Tuesday, 7 October 2008
    18. use Test::Aggregate; my $tests = Test::Aggregate->new({ dirs => 'aggtests', dump => $dump, set_filenames => 1, shuffle => 1, startup => \\&startup, shutdown => \\&shutdown, }); Tuesday, 7 October 2008
    19. Generic Aggregation • Before: 16 minutes • After: 12 minutes Tuesday, 7 October 2008
    20. Better Aggregation • use Test::Class Tuesday, 7 October 2008
    21. OK to Fail is OK • POD tests • Perl::Critic • Anything “non-functional” Tuesday, 7 October 2008
    22. OK to Fail is OK • Move them to xt/ • But they must be run! Tuesday, 7 October 2008
    23. Lessons Learned • Not dropping your database is tricky • Aggregating tests finds bugs in tests • Aggregating tests finds bugs in code • *CORE::GLOBAL:: is evil • Don’t touch UNIVERSAL:: Tuesday, 7 October 2008
    24. Faster Programmers Tuesday, 7 October 2008
    25. Custom Test Modules Tuesday, 7 October 2008
    26. use Test::More tests => 13; use Test::Exception; use Test::XML; use Test::JSON; use Test::Differences; Tuesday, 7 October 2008
    27. package Our::Test::More; use Test::Builder::Module; our ( @ISA, @EXPORT ); use Test::More; use Test::Exception; BEGIN { @ISA = qw(Test::Builder::Module); @EXPORT = ( @Test::More::EXPORT, @Test::Exception::EXPORT, ); } 1; Tuesday, 7 October 2008
    28. package My::Custom::Tests; use Test::Kit ( 'Test::More', 'Test::XML', 'Test::Differences', '+explain', ); 1; Tuesday, 7 October 2008
    29. Popularity Contest Test::More 44461 Test 8937 Test::Exception 1397 Test::Simple 731 Test::Base 316 Test::Builder::Tester 193 Test::NoWarnings 174 Test::Differences 146 Test::MockObject 139 Test::Deep 127 Tuesday, 7 October 2008
    30. use Test::Most tests => 4, 'die'; ok 1, 'one is true'; is 2, 2, '... and two is two'; eq_or_diff [3], [4], “... but three ain’t four”; throws_ok { $foo/0 } qr/Illegal division by zero/, 'and no-constant folding with vars'; Tuesday, 7 October 2008
    31. Test From Your Editor Tuesday, 7 October 2008
    32. “ in your .vimrc function! PerlMappings() noremap K :!perldoc <cword> <bar><bar> \\ perldoc -f <cword><cr> endfunction function! PerlTestMappings() noremap <buffer> ,t :!prove -vl %<CR> endfunction “ remember My::Test::YAML? function! YAMLTestMappings() noremap <buffer> ,t :!prove -vl t/yaml.t :: %<CR> endfunction au! FileType perl :call PerlMappings() au! FileType yaml :call YAMLTestMappings() au! BufRead,BufNewFile *.t :call PerlTestMappings() Tuesday, 7 October 2008
    33. vim $(ack -l --perl 'api/v1/episode' t/) map <leader>tb :call RunTestsInBuffers()<cr> function! RunTestsInBuffers() let i = 1 let tests = '' while (i <= bufnr(\"$\")) let filename = bufname(i) if match(filename, '\\.t$') > -1 let tests = tests . ' \"' . filename . '\"' endif let i = i+1 endwhile if !strlen(tests) echo \"No tests found in buffers\" else execute ':!prove ' . tests endif endfunction Tuesday, 7 October 2008
    34. Advanced “prove” Tuesday, 7 October 2008
    35. Test-Harness $ prove -l t --state=hot,fast,save --timer [20:57:19] t/yamlish-output.........ok 40 ms [20:57:20] t/console................ok 45 ms [20:57:20] t/utils..................ok 48 ms <snip> [20:57:23] t/harness................ok 300 ms [20:57:23] t/process................ok 1020 ms [20:57:24] t/prove..................ok 1017 ms [20:57:25] t/regression.............ok 4217 ms [20:57:29] All tests successful. Files=32, Tests=10326, 10 wallclock secs ( 1.25 usr 0.24 sys + 5.60 cusr 1.83 csys = 8.92 CPU) Result: PASS Tuesday, 7 October 2008
    36. Test-Harness $ prove -l t --state=hot,slow,save -j 9 t/callbacks.............. ok t/nofork................. ok t/proverc................ ok <snip> t/yamlish................ ok t/prove.................. ok t/regression............. ok All tests successful. Files=32, Tests=10326, 6 wallclock secs ( 1.34 usr 0.24 sys + 5.63 cusr 1.83 csys = 9.04 CPU) Result: PASS Tuesday, 7 October 2008
    37. # slow running tests from App::Prove::State # http://use.perl.org/~Ovid/journal/35831 Generation 18 Number of test programs: 58 Total runtime approximately 17 minutes 35 seconds Five slowest tests: 482.7 seconds -> t/acceptance.t 234.4 seconds -> t/aggregate.t 96.3 seconds -> t/standards/strict.t 66.6 seconds -> t/unit/db/migrations.t 56.7 seconds -> t/unit/piptest/pprove/testdb.t Tuesday, 7 October 2008
    38. Devel::CoverX::Covered Tuesday, 7 October 2008
    39. Devel::CoverX::Covered • What tests cover this file? • What files are covered by this test? • (Soon) What tests cover this line? • (Soon) What tests cover this subroutine? • ... and more ... Tuesday, 7 October 2008
    40. function! PerlMappings() noremap <buffer> ,cv :call Coverage()<cr> endfunction function! PerlTestMappings() noremap <buffer> ,t :!prove -vl --norc %<CR> endfunction function! Coverage() let file = bufname('%') if match(filename, '\\.t$') > -1 execute '!covered by --test_file=\"'.file.'\"' else execute '!covered covering --source_file=\"'.file.'\"' end endfunction au! FileType perl :call PerlMappings() au! BufRead,BufNewFile *.t :call PerlTestMappings() Tuesday, 7 October 2008
    41. Questions? Tuesday, 7 October 2008

    + Curtis PoeCurtis Poe, 2 years ago

    custom

    1545 views, 1 favs, 0 embeds more stats

    Techniques for Perl programmers to speed up test su more

    More info about this document

    © All Rights Reserved

    Go to text version

    • Total Views 1545
      • 1545 on SlideShare
      • 0 from embeds
    • Comments 0
    • Favorites 1
    • Downloads 14
    Most viewed embeds

    more

    All embeds

    less

    Flagged as inappropriate Flag as inappropriate
    Flag as inappropriate

    Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

    Cancel
    File a copyright complaint
    Having problems? Go to our helpdesk?

    Categories

    Tags

    Groups / Events