Upcoming SlideShare
×

# What is wrong on Test::More? / Test::Moreが抱える問題点とその解決策

4,541 views
4,393 views

Published on

What is wrong on and how to improve Test::More.
Test::Moreの何が問題でどう解決すればいいか。

Published in: Technology
6 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

Views
Total views
4,541
On SlideShare
0
From Embeds
0
Number of Embeds
212
Actions
Shares
0
0
0
Likes
6
Embeds 0
No embeds

No notes for slide

### What is wrong on Test::More? / Test::Moreが抱える問題点とその解決策

2. 2. Agenda✦ Testing without spec 仕様を書かずにテストしてる✦ Not structured tests テストが構造化されてない✦ Needs test plan 事前にテストプラン (＝テスト数) を必要とする✦ No ﬁxture feature フィクスチャ機能がない✦ Hard to distinguish assertions どれがアサーションなのか分かりにくい copyright(c) 2012 kuwata-lab.com all rights reserved.
5. 5. Sample: Test::More (Perl)use Test::More tests => 4;is f(0), 0;is f(1), 1;is f(2), 1;is f(3), 2; copyright(c) 2012 kuwata-lab.com all rights reserved.
6. 6. Sample: RSpec (Ruby)describe f() it "calculates fibonacchi sequence" do f(0).should == 0 f(1).should == 1 f(2).should == 1 f(3).should == 2 endend copyright(c) 2012 kuwata-lab.com all rights reserved.
7. 7. Sample: unittest (Python)import unittestclass FooTest(unittest.TestCase): def test_calculates_fiboacchi_seq(self): """Calculates Fibonacchi sequence""" self.assertEqual(0, f(0)) self.assertEqual(1, f(1)) self.assertEqual(1, f(2)) self.assertEqual(2, f(3)) copyright(c) 2012 kuwata-lab.com all rights reserved.
8. 8. Goal of test✦ Test::More: Does that code run correctly? そのコードは意図した通りに動くか？✦ RSpec: Does that code satisfy the spec? そのコードは仕様を満たしているか？ copyright(c) 2012 kuwata-lab.com all rights reserved.
9. 9. Difference between Test::More and RSpec## Test::Morelike \$html, qr<h3>Hello</h3>; Higher-level information より高水準の情報## RSpecit "contains section title" do html.should =~ %r<h3>Hello</h3>end copyright(c) 2012 kuwata-lab.com all rights reserved.
10. 10. Solution: spec() https://gist.github.com/3797929sub spec { my (\$text, \$block) = @_; \$block->();}## usagespec "page contains section title", sub { ok(render() =~ qr`<h1>Hello</h1>`);}; copyright(c) 2012 kuwata-lab.com all rights reserved.
11. 11. Spec FirstStep 1. Write speciﬁcationsspec "...speciﬁcation1...";spec "...speciﬁcation2..."; #=> not ok 1 - ...spec... # TODO #=> not ok 2 - ...spec... # TODOStep 2. Add assertions according to specspec "...speciﬁcation...", sub { assertion1; assertion2;}; copyright(c) 2012 kuwata-lab.com all rights reserved.
12. 12. Solution: spec() https://gist.github.com/3797939sub spec { my (\$text, \$block) = @_; return \$block->() if \$block; TODO: { local \$TODO = ": not implemented yet"; ok(undef); }} copyright(c) 2012 kuwata-lab.com all rights reserved.
13. 13. Meaning of output linesAs is - ok when assertion passed, not ok when failed アサーションが成功したらok、失敗したらnot okok 1 - assertion1not ok 2 - assertion2To be - ok when spec satisﬁed, not ok when not コードが仕様を満たしたらok、満たさなければnot okok 1 - specification1not ok 2 - specification2 copyright(c) 2012 kuwata-lab.com all rights reserved.
14. 14. Spec : Assertion = 1 : N✦A speciﬁcation can contain some assertions 1つの仕様に複数のアサーションを書いてよいspec "returns pair of integer", sub { is scalar(@\$ret), 2; like \$ret->[0], qr/^d+\$/; like \$ret->[1], qr/^d+\$/;}; copyright(c) 2012 kuwata-lab.com all rights reserved.
15. 15. Spec : Assertion = 1 : NAs is Output lines per assertion アサーションごとに出力行ok 1ok 2ok 3To beok 1 - returns pair of integer Output lines per spec 仕様ごとに出力行 copyright(c) 2012 kuwata-lab.com all rights reserved.
16. 16. Solution: OK() https://gist.github.com/3797954sub OK { my (\$expr) = @_; unless (\$expr) { my (\$pkg, \$file, \$lineno) = caller(); die "AssertionFailed" ." at \$file line \$lineno.n"; }} copyright(c) 2012 kuwata-lab.com all rights reserved.
17. 17. Solution: spec() https://gist.github.com/3797954my \$spec = undef;my \$num = 0;sub spec { my (\$text, \$block) = @_; \$spec = \$text; \$num++; eval { \$block->(); }; my \$err = \$@; if (! \$err) { print "ok \$num - \$textn"; } else { print "not ok \$num - \$textn"; \$err =~ s/^/# /mg; \$err .= "n" if \$err !~ /nz/; print STDERR \$err; }} copyright(c) 2012 kuwata-lab.com all rights reserved.
18. 18. Conclusion of this section✦ Write test based on spec, not on code テストは、コードに対してではなく、仕様に対して書く✦ Spec specified? instead of Run correctly? 「正しく動作するか？」ではなく「仕様を満たしているか？」✦ Spec ﬁrst, assertion second 仕様を先に書いて、そのあとにアサーションを書く✦ Spec : Assertion = 1 : N 1つの仕様が複数のアサーションを含んでよい✦ Output line per spec, not assertion 出力行のok / not okはアサーション単位ではなく仕様単位に出す copyright(c) 2012 kuwata-lab.com all rights reserved.
24. 24. Sample: RSpec Test target (class, method, ...)require rspec テスト対象 (クラス、メソッド、…)describe Foo do describe #bar() do context when arg is provided do it "returns length" do Foo.new.methd1([0,1,2]).should == 3 Foo.new.methd1([]).should == 0 end end endend copyright(c) 2012 kuwata-lab.com all rights reserved.
25. 25. Sample: RSpec Test condition or situationrequire rspec 条件や状況describe Foo do describe #bar() do context when arg is provided do it "returns length" do Foo.new.methd1([0,1,2]).should == 3 Foo.new.methd1([]).should == 0 end end endend copyright(c) 2012 kuwata-lab.com all rights reserved.
26. 26. Sample: RSpecrequire rspecdescribe Foo do describe #bar() do context when arg is provided do it "returns length" do Foo.new.methd1([0,1,2]).should == 3 Foo.new.methd1([]).should == 0 end end end Speciﬁcation 仕様end copyright(c) 2012 kuwata-lab.com all rights reserved.
27. 27. Sample: unittest (Python)import unittestclass FooTest(unitteset.TestCase): def test_bar_1(self): """returns length of arg passed""" self.assertequal(3, Foo().bar([1,2,3])) def test_bar_2(self): """returns 0 when arg is not passed""" self.assertEqual(0, Foo().bar()) copyright(c) 2012 kuwata-lab.com all rights reserved.
28. 28. Sample: Test::Unit2 (Ruby)require test/unitrequire fooclass FooTest < Test::Unit::TestCase class MethTest < self def test_returns_length_of_arg n = Foo.new.bar([1,2,3]) assert_equal 3, n end endend ref: http://www.clear-code.com/blog/2012/4/25.html copyright(c) 2012 kuwata-lab.com all rights reserved.
29. 29. Sample: subtest() (Test::More)use Test::More tests=>1;subtest "package Foo", sub { plan tests=>1; subtest "sub bar()", sub { plan tests=>2; ok (1+1 == 2); ok (1-1 == 0); };}; copyright(c) 2012 kuwata-lab.com all rights reserved.
30. 30. Sample: subtest() (Test::More)\$ perl homhom.t1..1 1..1 1..2 ok 1 ok 2 ok 1 - sub bar()ok 1 - package Foo copyright(c) 2012 kuwata-lab.com all rights reserved.
31. 31. Sample: subtest() (Test::More)\$ perl homhom.t1..1 1..1 1..2 ok 1 ok 2 ok 1 - sub bar()ok 1 - package Foo copyright(c) 2012 kuwata-lab.com all rights reserved.
32. 32. Sample: subtest() (Test::More)\$ perl homhom.t1..1 1..1 1..2 ok 1 ok 2 ok 1 - sub bar()ok 1 - package Foo copyright(c) 2012 kuwata-lab.com all rights reserved.
33. 33. Solution: subtest() alternatives Test target テスト対象print "1..2n";topic package Foo, sub { topic sub meth1(), sub { case_when arg is passed, sub { spec "1+1 should be 2", sub { OK(1+1 == 2); }; spec "1-1 should be 0", sub { OK(1-1 == 0); }; }; Condition or situation }; 条件や状況}; copyright(c) 2012 kuwata-lab.com all rights reserved.
34. 34. Solution: subtest() alternatives\$ perl homhom.t1..2# * package Foo# * sub meth1()# - when arg is passedok 1 - 1+1 should be 2ok 2 - 1-1 should be 0 copyright(c) 2012 kuwata-lab.com all rights reserved.
35. 35. Solution: subtest() alternatives https://gist.github.com/3797976my \$depth = 0;sub topic { my (\$name, \$block) = @_; my \$indent = x \$depth; print "# \$indent* \$namen"; \$depth++; \$block->(); \$depth--;} copyright(c) 2012 kuwata-lab.com all rights reserved.
36. 36. Solution: subtest() alternatives https://gist.github.com/3797976my \$depth = 0;sub case_when { my (\$condition, \$block) = @_; my \$indent = x \$depth; print "# \$indent- \$conditionn"; \$depth++; \$block->(); \$depth--;} copyright(c) 2012 kuwata-lab.com all rights reserved.
37. 37. Conclution of this section✦ Testhas structure, because spec has structure. テストには構造がある。なぜなら仕様に構造があるから。✦ xUnit focuses on test automation, RSpec focuses on test structure. xUnitは自動化のための道具、RSpecは構造化のための道具✦ Output of subtest() suck. Deﬁne your own subtest alternatives. subtest()は出力が残念すぎる。自前関数お勧め。 copyright(c) 2012 kuwata-lab.com all rights reserved.
40. 40. Sample: subtest()use Test::More tests=>1;subtest package Foo, sub { plan tests=>1; subtest sub meth(), sub { plan test2=>1; is(1+1, 2); is(1-1, 0); };}; copyright(c) 2012 kuwata-lab.com all rights reserved.
41. 41. Pros of test plan✦ Detect unexpected test ﬁnishing テストの異常終了が検知できる • Compare number of (ok + not ok) with test plan 出力されたokやnot okの数と、 宣言されたテスト個数とを比較✦ Necessary to report test progress テスト実行時に進行状況を知るのに必要 • Especially for prove command 特にproveコマンドで copyright(c) 2012 kuwata-lab.com all rights reserved.
44. 44. done_testing() and subtest()use Test::More tests=>1;subtest "package Foo", sub { subtest "sub meth1()", sub { ok (1+1 == 2); ok (1-1 == 0); done_testing(); }; done_testing();};done_testing(); (No need to call done_testing() in subtest() since Test::More 0.95_01) copyright(c) 2012 kuwata-lab.com all rights reserved.
45. 45. Solution: subtest() alternatives\$ perl homhom.pl ok 1 ok 2 1..2 ok 1 - sub meth1() 1..1ok 1 - package Foo1..1 copyright(c) 2012 kuwata-lab.com all rights reserved.
47. 47. Cons of done_testing()✦ Need to expand TAP spec TAP仕様に拡張が必要 • Simplicity of TAP has gone もはやTAPの簡易性は損なわれた✦ Prove prints ? as number of tests proveコマンドで全体のテスト数が「?」に • Degeneration of interface インターフェースとしては退化 copyright(c) 2012 kuwata-lab.com all rights reserved.
48. 48. Off Topic: Doubt about TAP✦ If TAP accepts test plan after running tests, テスト数がわかるのがテスト終了後でいいなら • End of test indicatior is necessary for TAP, but test plan is not, is it? テストの終わりが分かる何かがあればテスト数いらなくね？ • Index number of test is not necessary, is it? そもそも ok や not ok に通し番号いらなくね？ ok 1 # ..spec.. ok 2 # ..spec.. <<TEST END>> copyright(c) 2012 kuwata-lab.com all rights reserved.
49. 49. The root cause of problem✦ Impossibleto count number of tests before running tests テスト数を事前に数えられない✦ To be あるべき姿 • Step1. Count and print number of tests テスト数を数えて出力 • Step2. Run tests テストを実行 copyright(c) 2012 kuwata-lab.com all rights reserved.
50. 50. Solution: Intermediate data structureAs is:is 1+1, 2; ok 1is 1-1, 0; ok 2 1..2To be: topicis 1+1, 2; topic 1..2is 1-1, 0; ok 1 spec ok 2 spec Easy to count tests copyright(c) 2012 kuwata-lab.com all rights reserved.
51. 51. Cons of intermediate data structure✦ Easyto count and ﬁlter tests before running tests テスト実行前にテストを数えたりフィルタするのが簡単にできる✦ No need to extend TAP speciﬁcation TAPの仕様を拡張しなくてよい • No more done_testing() done_testing()なんていらない • No more nesting like subtest() subtest()のような入れ子対応はいらない copyright(c) 2012 kuwata-lab.com all rights reserved.
53. 53. Solution: topic() and spec() https://gist.github.com/3798000topic class Foo, sub { topic sub meth1(), sub { case_when "arg is given", sub { spec "1+1 should be 2", sub { OK(1+1 == 2); }; }; Change to build tree };}; Traverse treerun_all(); copyright(c) 2012 kuwata-lab.com all rights reserved.
54. 54. Solution: topic() https://gist.github.com/3798000my \$NODES = [];sub topic { my (\$name, \$block) = @_; my \$node = {name=>\$name, prefix=>*, children=>[]}; push @\$NODES, \$node; my \$bkup = \$NODES; \$NODES = \$node->{children}; \$block->(); \$NODES = \$bkup;} copyright(c) 2012 kuwata-lab.com all rights reserved.
55. 55. Solution: spec() https://gist.github.com/3798000my \$NODES = [];sub spec { my (\$text, \$block) = @_; push @\$NODES, [\$text, \$block];} copyright(c) 2012 kuwata-lab.com all rights reserved.
56. 56. Solution: _count_specs() https://gist.github.com/3798000sub _count_specs { my (\$nodes) = @_; my \$n = 0; for (@\$nodes) { if (ref(\$_) eq HASH) { # topic \$n += _count_specs(\$_->{children}); } elsif (ref(\$_) eq ARRAY) { # spec \$n += 1; } } return \$n;} copyright(c) 2012 kuwata-lab.com all rights reserved.
57. 57. Solution: run_all() https://gist.github.com/3798000sub run_all { print "1..", _count_specs(\$NODES), "n"; _run_all(\$NODES, 0, 0);}sub _run_all { my (\$nodes, \$depth, \$num) = @_; my \$indent = x \$depth; for my \$x (@\$nodes) { if (ref(\$x) eq HASH) { # topic print "# \$indent\$x->{prefix} \$x->{name}n"; \$num = _run_all(\$x->{children}, \$depth + 1, \$num); } elsif (ref(\$x) eq ARRAY) { # spec my (\$text, \$block) = @\$x; \$num++; _run_spec(\$text, \$num, \$block); } } return \$num;} copyright(c) 2012 kuwata-lab.com all rights reserved.
58. 58. Conclustion in this seciton✦ Dont count tests manually. Use computer. テスト数を手動で数えるのはやめてコンピュータにさせよう✦ Noneed to expand TAP speciﬁcation. Both done_testing() and subtest() are wrong. TAPの仕様拡張は不必要。done_testing()もsubtest()も間違い✦ Intermediate data structure solves the problem. テストを表す中間データ構造を作れば万事解決 copyright(c) 2012 kuwata-lab.com all rights reserved.
60. 60. What is ﬁxture?"A test ﬁxture (also known as a test context) is theset of preconditions or state needed to run a test.The developer should set up a known good statebefore the tests, and return to the original stateafter the tests." http://en.wikipedia.org/wiki/XUnit"テストを実行、成功させるために必要な状態や前提条件の集合を、フィクスチャと呼ぶ。これらはテストコンテキストとも呼ばれる。開発者はテストの実行前にテストに適した状態を整え、テスト実行後に元の状態を復元することが望ましい。" http://ja.wikipedia.org/wiki/XUnit copyright(c) 2012 kuwata-lab.com all rights reserved.
61. 61. Fixture method✦ xUnit • setUp() / tearDown()✦ RSpec • before() / after()✦ Test::More • (nothing!) copyright(c) 2012 kuwata-lab.com all rights reserved.
62. 62. Fault of setUp/tearDownAll tests in a class must share a setUp/tearDown. sub setUp { my (\$self) = @_; \$self->man = User->new(gender=>M); \$self->woman = User->new(gender=>W); } sub test1 { my (\$self) = @_; my \$user = \$self->man; ... } sub test2 { my (\$self) = @_; my \$user = \$self->woman; ... } copyright(c) 2012 kuwata-lab.com all rights reserved.
63. 63. Fault of setUp/tearDownSeparate TestCase class?package FooTestCase;sub setUp { ... }sub testFoo { ... }package BarTestCase;sub setUp { ... }sub testBar { ... } No. Test structure should follow speciﬁcation reason, not ﬁxture reason. copyright(c) 2012 kuwata-lab.com all rights reserved.
64. 64. Another approach on ﬁxtureDeﬁne ﬁxture method for each test data.sub fx_man { return User->new(gender=>M); }sub fx_woman { return User->new(gender=>W); }spec "test for man", sub { OK(fx_man()->{gender} eq M); }spec "test for woman", sub { OK(fx_woman()->{gender} eq W); } More ﬂexible than setUp/tearDown copyright(c) 2012 kuwata-lab.com all rights reserved.
65. 65. Cons of the approachspec "returns length of file", sub { my \$file = fx_file("homhom"); OK(file_length(\$file) == 6); unlink(\$file);}; Troublesome task to do teardown for each ﬁxture data ﬁxtureごとに忘れずに解放処理を行うのは面倒 copyright(c) 2012 kuwata-lab.com all rights reserved.
66. 66. Solution: at_end()sub fx_file { my (\$content) = (@_); \$file = "test-".rand().".txt"; write_file(\$file, \$content); at_end { unlink(\$file); }; return \$file;} Register task called at end of test テストの終わりに実行される処理を登録する copyright(c) 2012 kuwata-lab.com all rights reserved.
67. 67. Solution: at_end()spec "returns length of file", sub { my \$file = fx_file("homhom"); OK(file_length(\$file) == 6); unlink \$file;}; No need to teardown for each test テストごとの終了処理を書く必要がなくなる copyright(c) 2012 kuwata-lab.com all rights reserved.
68. 68. Solution: at_end() https://gist.github.com/3798046our @_CLOSURES = ();sub at_end(&) { my (\$closure) = @_; push @_CLOSURES, \$closure;} copyright(c) 2012 kuwata-lab.com all rights reserved.
69. 69. Solution: _run_spec() https://gist.github.com/3798046sub _run_spec { my (\$text, \$num, \$closure) = @_; eval { \$closure->(); }; my \$s = \$@ ? "ok" : "not ok"; print "\$s \$num - \$textn"; my \$@ = undef; for my \$clos (reverse(@_CLOSURES)) { \$clos->(); } @_CLOSURES = ();} copyright(c) 2012 kuwata-lab.com all rights reserved.
70. 70. Conclusion in this section✦ No ﬁxture feature in Test::More Test::Moreにはﬁxture機能がない✦ SetUp()/tearDown() are not so good setUp()/tearDown()も良くはない✦ Use end_at() which registers teardown task in setup 作成時に解放処理を登録できるat_end()が便利 copyright(c) 2012 kuwata-lab.com all rights reserved.