YAPC::ASIA 2012What is wrongon Test::More?Test::Moreが抱える問題点とその解決策makoto kuwata <kwa@kuwata-lab.com>http://www.kuwata-lab.c...
Agenda✦ Testing   without spec 仕様を書かずにテストしてる✦ Not   structured tests テストが構造化されてない✦ Needs     test plan 事前にテストプラン (=テスト数) を...
Section 1:Testing without spec仕様を書かずにテストしている             copyright(c) 2012 kuwata-lab.com all rights reserved.
Point    Write your test    according to spec,    not your code.    テストは、コードではなく仕様をもとに書け。        copyright(c) 2012 kuwata-...
Sample: Test::More (Perl)use	 Test::More	 tests	 =>	 4;is	 f(0),	 0;is	 f(1),	 1;is	 f(2),	 1;is	 f(3),	 2;               ...
Sample: RSpec (Ruby)describe	 f()	 	 it	 "calculates	 fibonacchi	 sequence"	 do	 	 	 	 f(0).should	 ==	 0	 	 	 	 f(1).shou...
Sample: unittest (Python)import	 unittestclass	 FooTest(unittest.TestCase):	 	 def	 test_calculates_fiboacchi_seq(self):	 ...
Goal of test✦ Test::More: Does that code run correctly? そのコードは意図した通りに動くか?✦ RSpec: Does that code satisfy the spec? そのコードは仕...
Difference between Test::More and RSpec##	 Test::Morelike	 $html,	 qr<h3>Hello</h3>;                                Higher...
Solution: spec()                                                  https://gist.github.com/3797929sub	 spec	 {	 	 my	 ($tex...
Spec FirstStep 1. Write specificationsspec	 "...specification1...";spec	 "...specification2..."; 	 	 	 #=>	 not	 ok	 1	 -	 .....
Solution: spec()                                                  https://gist.github.com/3797939sub	 spec	 {	 	 my	 ($tex...
Meaning of output linesAs is - ok when assertion passed, not ok when        failed       アサーションが成功したらok、失敗したらnot okok	 1	 ...
Spec : Assertion = 1 : N✦A specification can contain some assertions 1つの仕様に複数のアサーションを書いてよいspec	 "returns	 pair	 of	 integer...
Spec : Assertion = 1 : NAs is                    Output lines per assertion                               アサーションごとに出力行ok	 ...
Solution: OK()                                                  https://gist.github.com/3797954sub	 OK	 {	 	 my	 ($expr)	 ...
Solution: spec()                                                            https://gist.github.com/3797954my	 $spec	 =	 u...
Conclusion of this section✦ Write   test based on spec, not on code テストは、コードに対してではなく、仕様に対して書く✦ Spec   specified? instead o...
Section 2:Not structured testsテストが構造化されてない             copyright(c) 2012 kuwata-lab.com all rights reserved.
PointTest should have structure.Because spec has structure.テストには構造がある。なぜなら仕様に構造があるから。        copyright(c) 2012 kuwata-lab....
Sample: Specification document  クラス:Calendar  メソッド:isLeapYear(int year): boolean  動作詳細:   ・100で割り切れる場合、    ・400で割り切れる場合はtru...
Test code as spec documentIs your test code availableas spec document?そのコードは仕様書としてほんとに利用できるの?           copyright(c) 2012 ...
Sample: Test::Moreuse	 Test::More	 tests	 =>	 2;use	 Foo;$foo	 =	 Foo->new();is(Foo->new()->meth1(),	 11);is(Foo->new()->m...
Sample: RSpec                               Test target (class, method, ...)require	 rspec            テスト対象 (クラス、メソッド、…)de...
Sample: RSpec                                     Test condition or situationrequire	 rspec                     条件や状況descr...
Sample: RSpecrequire	 rspecdescribe	 Foo	 do	 	 describe	 #bar()	 do	 	 	 	 context	 when	 arg	 is	 provided	 do	 	 	 	 	 ...
Sample: unittest (Python)import	 unittestclass	 FooTest(unitteset.TestCase):	 	 def	 test_bar_1(self):	 	 	 	 """returns	 ...
Sample: Test::Unit2 (Ruby)require	 test/unitrequire	 fooclass	 FooTest	 <	 Test::Unit::TestCase	 	 class	 MethTest	 <	 sel...
Sample: subtest() (Test::More)use	 Test::More	 tests=>1;subtest	 "package	 Foo",	 sub	 {	 	 plan	 tests=>1;	 	 subtest	 "s...
Sample: subtest() (Test::More)$	 perl	 homhom.t1..1	 	 	 	 1..1	 	 	 	 	 	 	 	 1..2	 	 	 	 	 	 	 	 ok	 1	 	 	 	 	 	 	 	 ok...
Sample: subtest() (Test::More)$	 perl	 homhom.t1..1	 	 	 	 1..1	 	 	 	 	 	 	 	 1..2	 	 	 	 	 	 	 	 ok	 1	 	 	 	 	 	 	 	 ok...
Sample: subtest() (Test::More)$	 perl	 homhom.t1..1	 	 	 	 1..1	 	 	 	 	 	 	 	 1..2	 	 	 	 	 	 	 	 ok	 1	 	 	 	 	 	 	 	 ok...
Solution: subtest() alternatives                                        Test target                                       ...
Solution: subtest() alternatives$	 perl	 homhom.t1..2#	 *	 package	 Foo#	 	 	 *	 sub	 meth1()#	 	 	 	 	 -	 when	 arg	 is	 ...
Solution: subtest() alternatives                                                   https://gist.github.com/3797976my	 $dep...
Solution: subtest() alternatives                                                  https://gist.github.com/3797976my	 $dept...
Conclution of this section✦ Testhas structure, because spec has structure.  テストには構造がある。なぜなら仕様に構造があるから。✦ xUnit     focuses ...
Section 3:Needs test plan事前にテストプラン (=テスト数) を必要とする             copyright(c) 2012 kuwata-lab.com all rights reserved.
Sample: Test::Moreuse	 Test::More	 tests=>2;is(1+1,	 2);is(1-1,	 0);$	 perl	 homhom.t1..2ok	 1ok	 2               copyrigh...
Sample: subtest()use	 Test::More	 tests=>1;subtest	 package	 Foo,	 sub	 {	 	 plan	 tests=>1;	 	 subtest	 sub	 meth(),	 sub...
Pros of test plan✦ Detect   unexpected test finishing  テストの異常終了が検知できる • Compare number of (ok + not ok) with test   plan   ...
Cons of test plan✦ Too   messy to keep currect value  正しい値に更新し続けるのが面倒すぎる • Update test plan when you add assertions   asse...
Sample: done_testing()use	 Test::More	 tests=>2;is(1+1,	 2);is(1-1,	 0);done_testing();$	 perl	 homhom.tok	 1ok	 21..2    ...
done_testing() and subtest()use	 Test::More	 tests=>1;subtest	 "package	 Foo",	 sub	 {	 	 subtest	 "sub	 meth1()",	 sub	 {...
Solution: subtest() alternatives$	 perl	 homhom.pl	 	 	 	 	 	 	 	 ok	 1	 	 	 	 	 	 	 	 ok	 2	 	 	 	 	 	 	 	 1..2	 	 	 	 ok...
Pros of done_testing()✦ No   need to speicfy test plan!  テスト数を指定しなくていい!               copyright(c) 2012 kuwata-lab.com all...
Cons of done_testing()✦ Need   to expand TAP spec  TAP仕様に拡張が必要 • Simplicity of TAP has gone   もはやTAPの簡易性は損なわれた✦ Prove   pr...
Off Topic: Doubt about TAP✦ If   TAP accepts test plan after running tests,   テスト数がわかるのがテスト終了後でいいなら  • End of test indicat...
The root cause of problem✦ Impossibleto count number of tests before running tests  テスト数を事前に数えられない✦ To   be  あるべき姿 • Step1...
Solution: Intermediate data structureAs is:is	 1+1,	 2;                                                           ok	 1is	...
Cons of intermediate data structure✦ Easyto count and filter tests before running tests  テスト実行前にテストを数えたりフィルタするのが簡単にできる✦ No ...
Sample: xUnitclass	 FooTest(TestCase):	 	 def	 test1(self):	 ...	 	 def	 test2(self):	 ...##	 build	 intermediate	 data	 s...
Solution: topic() and spec()                                                   https://gist.github.com/3798000topic	 class...
Solution: topic()                                                 https://gist.github.com/3798000my	 $NODES	 =	 [];sub	 to...
Solution: spec()                                                 https://gist.github.com/3798000my	 $NODES	 =	 [];sub	 spe...
Solution: _count_specs()                                                    https://gist.github.com/3798000sub	 _count_spe...
Solution: run_all()                                                          https://gist.github.com/3798000sub	 run_all	 ...
Conclustion in this seciton✦ Dont   count tests manually. Use computer.  テスト数を手動で数えるのはやめてコンピュータにさせよう✦ Noneed to expand TAP...
Section 4:No fixture featureフィクスチャ機能がない             copyright(c) 2012 kuwata-lab.com all rights reserved.
What is fixture?"A test fixture (also known as a test context) is theset of preconditions or state needed to run a test.The ...
Fixture method✦ xUnit • setUp() / tearDown()✦ RSpec • before() / after()✦ Test::More • (nothing!)                 copyrigh...
Fault of setUp/tearDownAll tests in a class must share a setUp/tearDown.	 	 sub	 setUp	 {	 my	 ($self)	 =	 @_;	 	 	 	 $sel...
Fault of setUp/tearDownSeparate TestCase class?package	 FooTestCase;sub	 setUp	 {	 ...	 }sub	 testFoo	 {	 ...	 }package	 B...
Another approach on fixtureDefine fixture method for each test data.sub	 fx_man	 {	 	 return	 User->new(gender=>M);	 }sub	 fx...
Cons of the approachspec	 "returns	 length	 of	 file",	 sub	 {	 	 my	 $file	 =	 fx_file("homhom");	 	 OK(file_length($file...
Solution: at_end()sub	 fx_file	 {	 	 my	 ($content)	 =	 (@_);	 	 $file	 =	 "test-".rand().".txt";	 	 write_file($file,	 $c...
Solution: at_end()spec	 "returns	 length	 of	 file",	 sub	 {	 	 my	 $file	 =	 fx_file("homhom");	 	 OK(file_length($file)	...
Solution: at_end()                                                 https://gist.github.com/3798046our	 @_CLOSURES	 =	 ();s...
Solution: _run_spec()                                                 https://gist.github.com/3798046sub	 _run_spec	 {	 	 ...
Conclusion in this section✦ No   fixture feature in Test::More  Test::Moreにはfixture機能がない✦ SetUp()/tearDown()                ...
Section 5:Hard to distinguish assertionsどれがアサーションなのか分かりにくい             copyright(c) 2012 kuwata-lab.com all rights reserved.
Assertions in xUnitConsistent naming rule for assertion methods.アサーションメソッドに一貫した命名規則がある   Easy to distinguish assertions in...
Assertions in Test::MoreNo naming rule for assertion functions.アサーションメソッドに一貫した命名規則がない   Hard to distinguish assertions in ...
Solution: AssertionObject class                                                 https://gist.github.com/3798075Collect ass...
Solution: OK()                                                  https://gist.github.com/3798075Change OK() which returns A...
Solution: Operator overload                                                   https://gist.github.com/3798075package	 Asse...
ok() vs. OK()ok(2 == 1);not	 ok	 1#	 	 	 Failed	 test	 at	 homhom.t	 line	 4.OK(2) == 1;not	 ok	 1#	 	 	 AssertionFailed	 ...
is() vs. OK()is() :is("1.0",	 1.0);	 	 	 	 	 #=>	 not	 okOK() : OK	 ("1.0")	 eq	 1.0;	 	 	 #=>	 not	 ok OK	 ("1.0")	 ==	 1...
Test::More vs. OK()Test::More                                         OK()ok(1+1	 ==	 2);                                 ...
Conclusion in this section✦ Noconsistent naming rule of assertion functions in Test::Unit  Test::Moreにはアサーション関数に一貫した命名規則がな...
One More Thing...
Oktest.pm - a new style testing library               http://search.cpan.org/~kwatch/Oktest/lib/Oktest.pmuse	 strict;use	 ...
おしまい
Upcoming SlideShare
Loading in...5
×

What is wrong on Test::More? / Test::Moreが抱える問題点とその解決策

3,891

Published on

What is wrong on and how to improve Test::More.
Test::Moreの何が問題でどう解決すればいいか。

Published in: Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,891
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

What is wrong on Test::More? / Test::Moreが抱える問題点とその解決策

  1. 1. YAPC::ASIA 2012What is wrongon Test::More?Test::Moreが抱える問題点とその解決策makoto kuwata <kwa@kuwata-lab.com>http://www.kuwata-lab.com/2012-09-27 (Fri) copyright(c) 2012 kuwata-lab.com all rights reserved.
  2. 2. Agenda✦ Testing without spec 仕様を書かずにテストしてる✦ Not structured tests テストが構造化されてない✦ Needs test plan 事前にテストプラン (=テスト数) を必要とする✦ No fixture feature フィクスチャ機能がない✦ Hard to distinguish assertions どれがアサーションなのか分かりにくい copyright(c) 2012 kuwata-lab.com all rights reserved.
  3. 3. Section 1:Testing without spec仕様を書かずにテストしている copyright(c) 2012 kuwata-lab.com all rights reserved.
  4. 4. Point Write your test according to spec, not your code. テストは、コードではなく仕様をもとに書け。 copyright(c) 2012 kuwata-lab.com all rights reserved.
  5. 5. Sample: Test::More (Perl)use Test::More tests => 4;is f(0), 0;is f(1), 1;is f(2), 1;is f(3), 2; copyright(c) 2012 kuwata-lab.com all rights reserved.
  6. 6. Sample: RSpec (Ruby)describe f() it "calculates fibonacchi sequence" do f(0).should == 0 f(1).should == 1 f(2).should == 1 f(3).should == 2 endend copyright(c) 2012 kuwata-lab.com all rights reserved.
  7. 7. Sample: unittest (Python)import unittestclass FooTest(unittest.TestCase): def test_calculates_fiboacchi_seq(self): """Calculates Fibonacchi sequence""" self.assertEqual(0, f(0)) self.assertEqual(1, f(1)) self.assertEqual(1, f(2)) self.assertEqual(2, f(3)) copyright(c) 2012 kuwata-lab.com all rights reserved.
  8. 8. Goal of test✦ Test::More: Does that code run correctly? そのコードは意図した通りに動くか?✦ RSpec: Does that code satisfy the spec? そのコードは仕様を満たしているか? copyright(c) 2012 kuwata-lab.com all rights reserved.
  9. 9. Difference between Test::More and RSpec## Test::Morelike $html, qr<h3>Hello</h3>; Higher-level information より高水準の情報## RSpecit "contains section title" do html.should =~ %r<h3>Hello</h3>end copyright(c) 2012 kuwata-lab.com all rights reserved.
  10. 10. Solution: spec() https://gist.github.com/3797929sub spec { my ($text, $block) = @_; $block->();}## usagespec "page contains section title", sub { ok(render() =~ qr`<h1>Hello</h1>`);}; copyright(c) 2012 kuwata-lab.com all rights reserved.
  11. 11. Spec FirstStep 1. Write specificationsspec "...specification1...";spec "...specification2..."; #=> not ok 1 - ...spec... # TODO #=> not ok 2 - ...spec... # TODOStep 2. Add assertions according to specspec "...specification...", sub { assertion1; assertion2;}; copyright(c) 2012 kuwata-lab.com all rights reserved.
  12. 12. Solution: spec() https://gist.github.com/3797939sub spec { my ($text, $block) = @_; return $block->() if $block; TODO: { local $TODO = ": not implemented yet"; ok(undef); }} copyright(c) 2012 kuwata-lab.com all rights reserved.
  13. 13. Meaning of output linesAs is - ok when assertion passed, not ok when failed アサーションが成功したらok、失敗したらnot okok 1 - assertion1not ok 2 - assertion2To be - ok when spec satisfied, not ok when not コードが仕様を満たしたらok、満たさなければnot okok 1 - specification1not ok 2 - specification2 copyright(c) 2012 kuwata-lab.com all rights reserved.
  14. 14. Spec : Assertion = 1 : N✦A specification can contain some assertions 1つの仕様に複数のアサーションを書いてよいspec "returns pair of integer", sub { is scalar(@$ret), 2; like $ret->[0], qr/^d+$/; like $ret->[1], qr/^d+$/;}; copyright(c) 2012 kuwata-lab.com all rights reserved.
  15. 15. Spec : Assertion = 1 : NAs is Output lines per assertion アサーションごとに出力行ok 1ok 2ok 3To beok 1 - returns pair of integer Output lines per spec 仕様ごとに出力行 copyright(c) 2012 kuwata-lab.com all rights reserved.
  16. 16. Solution: OK() https://gist.github.com/3797954sub OK { my ($expr) = @_; unless ($expr) { my ($pkg, $file, $lineno) = caller(); die "AssertionFailed" ." at $file line $lineno.n"; }} copyright(c) 2012 kuwata-lab.com all rights reserved.
  17. 17. Solution: spec() https://gist.github.com/3797954my $spec = undef;my $num = 0;sub spec { my ($text, $block) = @_; $spec = $text; $num++; eval { $block->(); }; my $err = $@; if (! $err) { print "ok $num - $textn"; } else { print "not ok $num - $textn"; $err =~ s/^/# /mg; $err .= "n" if $err !~ /nz/; print STDERR $err; }} copyright(c) 2012 kuwata-lab.com all rights reserved.
  18. 18. Conclusion of this section✦ Write test based on spec, not on code テストは、コードに対してではなく、仕様に対して書く✦ Spec specified? instead of Run correctly? 「正しく動作するか?」ではなく「仕様を満たしているか?」✦ Spec first, assertion second 仕様を先に書いて、そのあとにアサーションを書く✦ Spec : Assertion = 1 : N 1つの仕様が複数のアサーションを含んでよい✦ Output line per spec, not assertion 出力行のok / not okはアサーション単位ではなく仕様単位に出す copyright(c) 2012 kuwata-lab.com all rights reserved.
  19. 19. Section 2:Not structured testsテストが構造化されてない copyright(c) 2012 kuwata-lab.com all rights reserved.
  20. 20. PointTest should have structure.Because spec has structure.テストには構造がある。なぜなら仕様に構造があるから。 copyright(c) 2012 kuwata-lab.com all rights reserved.
  21. 21. Sample: Specification document クラス:Calendar メソッド:isLeapYear(int year): boolean 動作詳細: ・100で割り切れる場合、 ・400で割り切れる場合はtrueを返す ・それ以外はfalseを返す ・4で割り切れる場合はtrueを返す ・それ以外はfalseを返す copyright(c) 2012 kuwata-lab.com all rights reserved.
  22. 22. Test code as spec documentIs your test code availableas spec document?そのコードは仕様書としてほんとに利用できるの? copyright(c) 2012 kuwata-lab.com all rights reserved.
  23. 23. Sample: Test::Moreuse Test::More tests => 2;use Foo;$foo = Foo->new();is(Foo->new()->meth1(), 11);is(Foo->new()->meth2(), 12); copyright(c) 2012 kuwata-lab.com all rights reserved.
  24. 24. Sample: RSpec Test target (class, method, ...)require rspec テスト対象 (クラス、メソッド、…)describe Foo do describe #bar() do context when arg is provided do it "returns length" do Foo.new.methd1([0,1,2]).should == 3 Foo.new.methd1([]).should == 0 end end endend copyright(c) 2012 kuwata-lab.com all rights reserved.
  25. 25. Sample: RSpec Test condition or situationrequire rspec 条件や状況describe Foo do describe #bar() do context when arg is provided do it "returns length" do Foo.new.methd1([0,1,2]).should == 3 Foo.new.methd1([]).should == 0 end end endend copyright(c) 2012 kuwata-lab.com all rights reserved.
  26. 26. Sample: RSpecrequire rspecdescribe Foo do describe #bar() do context when arg is provided do it "returns length" do Foo.new.methd1([0,1,2]).should == 3 Foo.new.methd1([]).should == 0 end end end Specification 仕様end copyright(c) 2012 kuwata-lab.com all rights reserved.
  27. 27. Sample: unittest (Python)import unittestclass FooTest(unitteset.TestCase): def test_bar_1(self): """returns length of arg passed""" self.assertequal(3, Foo().bar([1,2,3])) def test_bar_2(self): """returns 0 when arg is not passed""" self.assertEqual(0, Foo().bar()) copyright(c) 2012 kuwata-lab.com all rights reserved.
  28. 28. Sample: Test::Unit2 (Ruby)require test/unitrequire fooclass FooTest < Test::Unit::TestCase class MethTest < self def test_returns_length_of_arg n = Foo.new.bar([1,2,3]) assert_equal 3, n end endend ref: http://www.clear-code.com/blog/2012/4/25.html copyright(c) 2012 kuwata-lab.com all rights reserved.
  29. 29. Sample: subtest() (Test::More)use Test::More tests=>1;subtest "package Foo", sub { plan tests=>1; subtest "sub bar()", sub { plan tests=>2; ok (1+1 == 2); ok (1-1 == 0); };}; copyright(c) 2012 kuwata-lab.com all rights reserved.
  30. 30. Sample: subtest() (Test::More)$ perl homhom.t1..1 1..1 1..2 ok 1 ok 2 ok 1 - sub bar()ok 1 - package Foo copyright(c) 2012 kuwata-lab.com all rights reserved.
  31. 31. Sample: subtest() (Test::More)$ perl homhom.t1..1 1..1 1..2 ok 1 ok 2 ok 1 - sub bar()ok 1 - package Foo copyright(c) 2012 kuwata-lab.com all rights reserved.
  32. 32. Sample: subtest() (Test::More)$ perl homhom.t1..1 1..1 1..2 ok 1 ok 2 ok 1 - sub bar()ok 1 - package Foo copyright(c) 2012 kuwata-lab.com all rights reserved.
  33. 33. Solution: subtest() alternatives Test target テスト対象print "1..2n";topic package Foo, sub { topic sub meth1(), sub { case_when arg is passed, sub { spec "1+1 should be 2", sub { OK(1+1 == 2); }; spec "1-1 should be 0", sub { OK(1-1 == 0); }; }; Condition or situation }; 条件や状況}; copyright(c) 2012 kuwata-lab.com all rights reserved.
  34. 34. Solution: subtest() alternatives$ perl homhom.t1..2# * package Foo# * sub meth1()# - when arg is passedok 1 - 1+1 should be 2ok 2 - 1-1 should be 0 copyright(c) 2012 kuwata-lab.com all rights reserved.
  35. 35. Solution: subtest() alternatives https://gist.github.com/3797976my $depth = 0;sub topic { my ($name, $block) = @_; my $indent = x $depth; print "# $indent* $namen"; $depth++; $block->(); $depth--;} copyright(c) 2012 kuwata-lab.com all rights reserved.
  36. 36. Solution: subtest() alternatives https://gist.github.com/3797976my $depth = 0;sub case_when { my ($condition, $block) = @_; my $indent = x $depth; print "# $indent- $conditionn"; $depth++; $block->(); $depth--;} copyright(c) 2012 kuwata-lab.com all rights reserved.
  37. 37. Conclution of this section✦ Testhas structure, because spec has structure. テストには構造がある。なぜなら仕様に構造があるから。✦ xUnit focuses on test automation, RSpec focuses on test structure. xUnitは自動化のための道具、RSpecは構造化のための道具✦ Output of subtest() suck. Define your own subtest alternatives. subtest()は出力が残念すぎる。自前関数お勧め。 copyright(c) 2012 kuwata-lab.com all rights reserved.
  38. 38. Section 3:Needs test plan事前にテストプラン (=テスト数) を必要とする copyright(c) 2012 kuwata-lab.com all rights reserved.
  39. 39. Sample: Test::Moreuse Test::More tests=>2;is(1+1, 2);is(1-1, 0);$ perl homhom.t1..2ok 1ok 2 copyright(c) 2012 kuwata-lab.com all rights reserved.
  40. 40. Sample: subtest()use Test::More tests=>1;subtest package Foo, sub { plan tests=>1; subtest sub meth(), sub { plan test2=>1; is(1+1, 2); is(1-1, 0); };}; copyright(c) 2012 kuwata-lab.com all rights reserved.
  41. 41. Pros of test plan✦ Detect unexpected test finishing テストの異常終了が検知できる • Compare number of (ok + not ok) with test plan 出力されたokやnot okの数と、 宣言されたテスト個数とを比較✦ Necessary to report test progress テスト実行時に進行状況を知るのに必要 • Especially for prove command 特にproveコマンドで copyright(c) 2012 kuwata-lab.com all rights reserved.
  42. 42. Cons of test plan✦ Too messy to keep currect value 正しい値に更新し続けるのが面倒すぎる • Update test plan when you add assertions assertion関数を追加したら忘れずに更新 • back to top when you add at the end of file ファイルの末尾にテストを追加したら、先頭に戻ってテスト数 を更新 copyright(c) 2012 kuwata-lab.com all rights reserved.
  43. 43. Sample: done_testing()use Test::More tests=>2;is(1+1, 2);is(1-1, 0);done_testing();$ perl homhom.tok 1ok 21..2 copyright(c) 2012 kuwata-lab.com all rights reserved.
  44. 44. done_testing() and subtest()use Test::More tests=>1;subtest "package Foo", sub { subtest "sub meth1()", sub { ok (1+1 == 2); ok (1-1 == 0); done_testing(); }; done_testing();};done_testing(); (No need to call done_testing() in subtest() since Test::More 0.95_01) copyright(c) 2012 kuwata-lab.com all rights reserved.
  45. 45. Solution: subtest() alternatives$ perl homhom.pl ok 1 ok 2 1..2 ok 1 - sub meth1() 1..1ok 1 - package Foo1..1 copyright(c) 2012 kuwata-lab.com all rights reserved.
  46. 46. Pros of done_testing()✦ No need to speicfy test plan! テスト数を指定しなくていい! copyright(c) 2012 kuwata-lab.com all rights reserved.
  47. 47. Cons of done_testing()✦ Need to expand TAP spec TAP仕様に拡張が必要 • Simplicity of TAP has gone もはやTAPの簡易性は損なわれた✦ Prove prints ? as number of tests proveコマンドで全体のテスト数が「?」に • Degeneration of interface インターフェースとしては退化 copyright(c) 2012 kuwata-lab.com all rights reserved.
  48. 48. Off Topic: Doubt about TAP✦ If TAP accepts test plan after running tests, テスト数がわかるのがテスト終了後でいいなら • End of test indicatior is necessary for TAP, but test plan is not, is it? テストの終わりが分かる何かがあればテスト数いらなくね? • Index number of test is not necessary, is it? そもそも ok や not ok に通し番号いらなくね? ok 1 # ..spec.. ok 2 # ..spec.. <<TEST END>> copyright(c) 2012 kuwata-lab.com all rights reserved.
  49. 49. The root cause of problem✦ Impossibleto count number of tests before running tests テスト数を事前に数えられない✦ To be あるべき姿 • Step1. Count and print number of tests テスト数を数えて出力 • Step2. Run tests テストを実行 copyright(c) 2012 kuwata-lab.com all rights reserved.
  50. 50. Solution: Intermediate data structureAs is:is 1+1, 2; ok 1is 1-1, 0; ok 2 1..2To be: topicis 1+1, 2; topic 1..2is 1-1, 0; ok 1 spec ok 2 spec Easy to count tests copyright(c) 2012 kuwata-lab.com all rights reserved.
  51. 51. Cons of intermediate data structure✦ Easyto count and filter tests before running tests テスト実行前にテストを数えたりフィルタするのが簡単にできる✦ No need to extend TAP specification TAPの仕様を拡張しなくてよい • No more done_testing() done_testing()なんていらない • No more nesting like subtest() subtest()のような入れ子対応はいらない copyright(c) 2012 kuwata-lab.com all rights reserved.
  52. 52. Sample: xUnitclass FooTest(TestCase): def test1(self): ... def test2(self): ...## build intermediate data structuresuite = TestSuite()suite.add(FooTest(test1))suite.add(FooTest(test2))## run testsTestRunner().run(suite) copyright(c) 2012 kuwata-lab.com all rights reserved.
  53. 53. Solution: topic() and spec() https://gist.github.com/3798000topic class Foo, sub { topic sub meth1(), sub { case_when "arg is given", sub { spec "1+1 should be 2", sub { OK(1+1 == 2); }; }; Change to build tree };}; Traverse treerun_all(); copyright(c) 2012 kuwata-lab.com all rights reserved.
  54. 54. Solution: topic() https://gist.github.com/3798000my $NODES = [];sub topic { my ($name, $block) = @_; my $node = {name=>$name, prefix=>*, children=>[]}; push @$NODES, $node; my $bkup = $NODES; $NODES = $node->{children}; $block->(); $NODES = $bkup;} copyright(c) 2012 kuwata-lab.com all rights reserved.
  55. 55. Solution: spec() https://gist.github.com/3798000my $NODES = [];sub spec { my ($text, $block) = @_; push @$NODES, [$text, $block];} copyright(c) 2012 kuwata-lab.com all rights reserved.
  56. 56. Solution: _count_specs() https://gist.github.com/3798000sub _count_specs { my ($nodes) = @_; my $n = 0; for (@$nodes) { if (ref($_) eq HASH) { # topic $n += _count_specs($_->{children}); } elsif (ref($_) eq ARRAY) { # spec $n += 1; } } return $n;} copyright(c) 2012 kuwata-lab.com all rights reserved.
  57. 57. Solution: run_all() https://gist.github.com/3798000sub run_all { print "1..", _count_specs($NODES), "n"; _run_all($NODES, 0, 0);}sub _run_all { my ($nodes, $depth, $num) = @_; my $indent = x $depth; for my $x (@$nodes) { if (ref($x) eq HASH) { # topic print "# $indent$x->{prefix} $x->{name}n"; $num = _run_all($x->{children}, $depth + 1, $num); } elsif (ref($x) eq ARRAY) { # spec my ($text, $block) = @$x; $num++; _run_spec($text, $num, $block); } } return $num;} copyright(c) 2012 kuwata-lab.com all rights reserved.
  58. 58. Conclustion in this seciton✦ Dont count tests manually. Use computer. テスト数を手動で数えるのはやめてコンピュータにさせよう✦ Noneed to expand TAP specification. Both done_testing() and subtest() are wrong. TAPの仕様拡張は不必要。done_testing()もsubtest()も間違い✦ Intermediate data structure solves the problem. テストを表す中間データ構造を作れば万事解決 copyright(c) 2012 kuwata-lab.com all rights reserved.
  59. 59. Section 4:No fixture featureフィクスチャ機能がない copyright(c) 2012 kuwata-lab.com all rights reserved.
  60. 60. What is fixture?"A test fixture (also known as a test context) is theset of preconditions or state needed to run a test.The developer should set up a known good statebefore the tests, and return to the original stateafter the tests." http://en.wikipedia.org/wiki/XUnit"テストを実行、成功させるために必要な状態や前提条件の集合を、フィクスチャと呼ぶ。これらはテストコンテキストとも呼ばれる。開発者はテストの実行前にテストに適した状態を整え、テスト実行後に元の状態を復元することが望ましい。" http://ja.wikipedia.org/wiki/XUnit copyright(c) 2012 kuwata-lab.com all rights reserved.
  61. 61. Fixture method✦ xUnit • setUp() / tearDown()✦ RSpec • before() / after()✦ Test::More • (nothing!) copyright(c) 2012 kuwata-lab.com all rights reserved.
  62. 62. Fault of setUp/tearDownAll tests in a class must share a setUp/tearDown. sub setUp { my ($self) = @_; $self->man = User->new(gender=>M); $self->woman = User->new(gender=>W); } sub test1 { my ($self) = @_; my $user = $self->man; ... } sub test2 { my ($self) = @_; my $user = $self->woman; ... } copyright(c) 2012 kuwata-lab.com all rights reserved.
  63. 63. Fault of setUp/tearDownSeparate TestCase class?package FooTestCase;sub setUp { ... }sub testFoo { ... }package BarTestCase;sub setUp { ... }sub testBar { ... } No. Test structure should follow specification reason, not fixture reason. copyright(c) 2012 kuwata-lab.com all rights reserved.
  64. 64. Another approach on fixtureDefine fixture method for each test data.sub fx_man { return User->new(gender=>M); }sub fx_woman { return User->new(gender=>W); }spec "test for man", sub { OK(fx_man()->{gender} eq M); }spec "test for woman", sub { OK(fx_woman()->{gender} eq W); } More flexible than setUp/tearDown copyright(c) 2012 kuwata-lab.com all rights reserved.
  65. 65. Cons of the approachspec "returns length of file", sub { my $file = fx_file("homhom"); OK(file_length($file) == 6); unlink($file);}; Troublesome task to do teardown for each fixture data fixtureごとに忘れずに解放処理を行うのは面倒 copyright(c) 2012 kuwata-lab.com all rights reserved.
  66. 66. Solution: at_end()sub fx_file { my ($content) = (@_); $file = "test-".rand().".txt"; write_file($file, $content); at_end { unlink($file); }; return $file;} Register task called at end of test テストの終わりに実行される処理を登録する copyright(c) 2012 kuwata-lab.com all rights reserved.
  67. 67. Solution: at_end()spec "returns length of file", sub { my $file = fx_file("homhom"); OK(file_length($file) == 6); unlink $file;}; No need to teardown for each test テストごとの終了処理を書く必要がなくなる copyright(c) 2012 kuwata-lab.com all rights reserved.
  68. 68. Solution: at_end() https://gist.github.com/3798046our @_CLOSURES = ();sub at_end(&) { my ($closure) = @_; push @_CLOSURES, $closure;} copyright(c) 2012 kuwata-lab.com all rights reserved.
  69. 69. Solution: _run_spec() https://gist.github.com/3798046sub _run_spec { my ($text, $num, $closure) = @_; eval { $closure->(); }; my $s = $@ ? "ok" : "not ok"; print "$s $num - $textn"; my $@ = undef; for my $clos (reverse(@_CLOSURES)) { $clos->(); } @_CLOSURES = ();} copyright(c) 2012 kuwata-lab.com all rights reserved.
  70. 70. Conclusion in this section✦ No fixture feature in Test::More Test::Moreにはfixture機能がない✦ SetUp()/tearDown() are not so good setUp()/tearDown()も良くはない✦ Use end_at() which registers teardown task in setup 作成時に解放処理を登録できるat_end()が便利 copyright(c) 2012 kuwata-lab.com all rights reserved.
  71. 71. Section 5:Hard to distinguish assertionsどれがアサーションなのか分かりにくい copyright(c) 2012 kuwata-lab.com all rights reserved.
  72. 72. Assertions in xUnitConsistent naming rule for assertion methods.アサーションメソッドに一貫した命名規則がある Easy to distinguish assertions in test code テストコード中でアサーションを見分けるのが簡単assertTrue()assertEqual()assertMatch()assertSet()... copyright(c) 2012 kuwata-lab.com all rights reserved.
  73. 73. Assertions in Test::MoreNo naming rule for assertion functions.アサーションメソッドに一貫した命名規則がない Hard to distinguish assertions in test code テストコード中でアサーションを見分けるのが困難ok()is()like()eq_set()cmp_ok()... copyright(c) 2012 kuwata-lab.com all rights reserved.
  74. 74. Solution: AssertionObject class https://gist.github.com/3798075Collect assertion functions in a class.アサーション関数を1つのクラスに集約package AssertionObject;## assertion methodssub num_eq { ... }sub str_eq { ... }sub match { ... }... copyright(c) 2012 kuwata-lab.com all rights reserved.
  75. 75. Solution: OK() https://gist.github.com/3798075Change OK() which returns AssertionObject.AssertionObjectインスタンスを返すようにOK()を変更sub OK { my ($actual) = @_; return AssertionObject->new($actual);} Easy to distinguish assertions## usage どれがアサーションかすぐ分かるOK (1+1)->num_eq(2);OK ("a")->str_eq("a");OK ("9")->match(qr/^d+/); copyright(c) 2012 kuwata-lab.com all rights reserved.
  76. 76. Solution: Operator overload https://gist.github.com/3798075package AssertionObject;use overload == => &num_eq, eq => &str_eq;package main;OK (1+1) == 2; // same as num_eq()OK ("a") eq "a"; // same as str_eq() Shortcut to assertion methods アサーションメソッドを簡単呼び出し copyright(c) 2012 kuwata-lab.com all rights reserved.
  77. 77. ok() vs. OK()ok(2 == 1);not ok 1# Failed test at homhom.t line 4.OK(2) == 1;not ok 1# AssertionFailed at homhom.t line 4.# $actual == $expected: failed# actual: 2# expected: 1 copyright(c) 2012 kuwata-lab.com all rights reserved.
  78. 78. is() vs. OK()is() :is("1.0", 1.0); #=> not okOK() : OK ("1.0") eq 1.0; #=> not ok OK ("1.0") == 1.0; #=> ok You can choose eq or ==. 文字列演算子と数値演算子を選べる copyright(c) 2012 kuwata-lab.com all rights reserved.
  79. 79. Test::More vs. OK()Test::More OK()ok(1+1 == 2); OK (1+1) == 2;isnt(1+1, 0); OK (1+1) != 0;cmp_ok(1+1, >=, 1); OK (1+1) >= 1; Consistent usage 一貫した使い方 copyright(c) 2012 kuwata-lab.com all rights reserved.
  80. 80. Conclusion in this section✦ Noconsistent naming rule of assertion functions in Test::Unit Test::Moreにはアサーション関数に一貫した命名規則がない✦ Hard to distinguish assertions in test テストコード中でどれがアサーションか見分けるのが困難✦ Solution: AssertionObject class and operator overload 解決策:AssertionObjectクラスと演算子オーバーロード copyright(c) 2012 kuwata-lab.com all rights reserved.
  81. 81. One More Thing...
  82. 82. Oktest.pm - a new style testing library http://search.cpan.org/~kwatch/Oktest/lib/Oktest.pmuse strict;use warnings;no warnings void; # RECOMMENDED!use Oktest;topic "ClassName", sub { topic "method_name()", sub { spec "...detail...", sub { OK (1+1) == 2; OK (a x 3) eq aaa; }; };};Oktest::main() if $0 eq __FILE__;1; copyright(c) 2012 kuwata-lab.com all rights reserved.
  83. 83. おしまい

×