Successfully reported this slideshow.
Your SlideShare is downloading. ×

BDD style Unit Testing


Check these out next

1 of 93 Ad

More Related Content

Slideshows for you (20)

Similar to BDD style Unit Testing (20)


More from Wen-Tien Chang (20)

Recently uploaded (20)


BDD style Unit Testing

  1. 1. How BDD style Unit Testing helps you write better test code (and some caveat) @ihower at RubyKaigi 2011 Ruby Taiwan & OptimisDev
  2. 2. About Me • Chang, Wen-Tien • a.k.a. ihower • • I’m ihower and here is my blog and t witter I have program ruby from 2006 for ruby on rails And I’m Yet another rubyist since rails.
  3. 3. From Taiwan 3.5hrs I come from Taiwan which is far away about 4hours flight I’m sorry that I can’t speak Japanese, and English is not my native language If I speak Chinese, I guess you will escape now. So please forgive my english. I hope it’s simple to understand.
  4. 4. I work at Optimis International, which is an america company in Taipei
  5. 5. Ruby Taiwan Beside daily job, I’m the founder and organizer of Ruby Taiwan community, I found the community since 2008.
  6. 6. RubyConf Taiwan 2011/8/26-27 We are organizing rubyconf taiwan now. The date is augest 26 and 27 and it open registration now. Welcome to Taiwan.
  7. 7. Last year photo. Nice small meeting room. pretty sweet.
  8. 8. Agenda • Why Unit Testing? • How to do unit test in BDD style? • Some BDD caveat Today’s agenda. There will be three part: First, I will talk about why, then how to do, finally talk about some caveat.
  9. 9. How many people do unit test? Before get into BDD, how many people here do unit test all the way? Could you raise your hand?
  10. 10. How many people use BDD framework (RSpec)? And how many people use rspec?
  11. 11. Two kinds of test: Verification Unit Test Are we writing the code right? Customer Test Validation (Acceptance Test) Are we writing the right software? There are roughly t wo kinds of test: Unit test and Customer test(or called Acceptance test) Unit test is about verification, the question is are we writing the code right? Customer test is about validation, the question is are we writing the right soft ware?
  12. 12. Two kinds of test: Test individual class and Unit Test method Customer Test Test functionality of the (Acceptance Test) over-all system Unit test test individual class and method And Customer Test test functionality of the over-all system
  13. 13. Today is all about Unit Testing Today is all about unit testing
  14. 14. 1. Why Unit Testing? Let get started part one
  15. 15. 1.Verification Do we write the code right? First, it does verification
  16. 16. How to verify this?There are three execution path def correct_year_month(year, month) if month > 12 if month % 12 == 0 year += (month - 12) / 12 month = 12 else year += month / 12 month = month % 12 end end return [year, month] end For example, how to verify this simple if else code. There are three execution path which means if you manually test you need play it 3 times in different way
  17. 17. via UI?
  18. 18. via irb console?
  19. 19. Manually test is inefficient
  20. 20. Unit testing give us instant coding feedback it "should return correct year and month" do correct_year_month(2011, 3).should == [2011, 3] correct_year_month(2011, 14).should == [2012,2] correct_year_month(2011, 24).should == [2012,12] end automatic unit testing can give us instant feedback you can test many times you want.
  21. 21. 2.Check regression Safety Net: when you add new feature and refactoring test also check the regression it’s a safety net when you add new feature or refactoring It can give you confidence that you won’t break up any existed behavior
  22. 22. 3.API design guide you write better API, especially test first. third, Since test code is the client for your production code, so if the test code is hard to write, it may means your API is not good. Writing unit testing can guide you write better API, especially when you write test first.
  23. 23. 4.Documentation What the result should be? Communication is the key for software development. finally, uniting test can be your code documentation. It can answer you what’s the class and method result should be? Communication the key for successful soft ware development.
  24. 24. Brief Summary • Your code is not trivial: Automatic test can save your verify/debug time • Your code is not throwing way: Regression test • Test first: Guide you write better API • Tests as Documentation
  25. 25. 2. How to do unit test in BDD style?
  26. 26. xUnit Framework • Each test has four-phase: • Setup • Exercise • Verify (assertion) • Teardown First, We use xUnit framework to test code. Each test should be isolation and has four phase: 1. Setup Data 2. Exercise production code 3. Verify the result 4. Cleanup the data
  27. 27. Ruby Test::Unit class OrderTest < Test::Unit::TestCase def setup 1 @order = end def test_order_amount assert_equal 0, @order.amount end 3 2 end The is the classical ruby test::unit code Red 1 is setup Red 2 is exercise Red 3 is verify. We assert the result equal to zero
  28. 28. Minitest::Spec describe Order do before do 1 @order = end it "should have default amount is 0" do @order.amount.must_equal 0 end 2 3 end Here is the BDD style using minitest::spec red 1 is setup red 2 is exercise red 3 is verify. the result must equal to zero
  29. 29. First impression about BDD • Syntax is different • Like specification • More readable So, what’s the first impression about bdd style? Ok, the syntax is different, it looks like spec and may more readable Other wise the structure is not much different
  30. 30. What’s BDD? • An improved xUnit Framework • Focus on clearly describe the expected behavior • The emphasis is Tests as Documentation rather than merely using tests for verification. Let’s back to see what’s BDD means. It’s not totally new test framework, it just an improved xUnit framework It focus on describe the expected behavior It emphasize tests as documentation, not just for verification.
  31. 31. Terminology changed New paradigm: Executable Specification • “Test” becomes “Spec” • “Assertion” becomes “Expectation” • “test method” becomes “example” (RSpec) • “test case” becomes “example group” (RSpec) For correspond to the new paradigm: the test is the executable specification The “test” becomes “spec” “Assertion” becomes “Expectation”
  32. 32. Ruby BDD tools • RSpec • Minitest/Spec In Ruby, there are some choices: RSpec is the most powerful, has many features and many awesome syntax. Minitest/spec is the Ruby 1.9 build-in xunit library. It’s simple, fast, and just works!
  33. 33. Learn BDD style • It’s about syntax • syntax • syntax • syntax Let’s learn how to write BDD style test code. Basically, it’s just learn about syntax.
  34. 34. BDD style element (1) Context
  35. 35. one test case describe Order do # ... end a class or string # or describe "A Order" do # ... end Instead of test case class, BDD style use “describe” block for one test case It accept parameter which means the stuff you want to describe, usually a class name or just description text.
  36. 36. Can be nested describe Order do describe "#amount" do # ... #method means end instance method describe ".size" do # ... end .method means class method end “describe” block can be nested. And inner “describe” usually describe method. Pound means instance method dot means class method
  37. 37. In RSpec, alias_method :context, :describe 1 describe Order do 2 context "when initialized" do 3 describe "#amount" do # ... end 3 describe ".size" do # ... end end Is RSpec, it alsoend describe to context. alias it used to organize test code into different context.
  38. 38. BDD style element (2) Declarative test
  39. 39. one test method: it you can write test description in plain text describe Order do describe "#amount" do describe "when user is vip" do it "should discount five percent if total > 1000" do # ... end it "should discount ten percent if total > 10000" do # ... end end end Each test method is “it” block And it accept one parameter which is description text
  40. 40. Setup method: before describe Order do before do @user = :is_vip => true ) @order = :user => @user ) end end the setup method is “before” block
  41. 41. nested before btw, the outer data will be shared, make the test faster describe Order do 1 before do @user = :is_vip => true ) @order = :user => @user ) end describe "#amount" do 2 before do @order2 = :user => @user ) end end “before” block also support nested end More important, the outer setup data will be shard for all inside test methods. Make the test faster since we need not create again every time.
  42. 42. class SomeTest < Test::Unit::TestCase setup do @value = 1 end test "sample test" do assert_equal 1, @value end context "a context" do setup do @value += 1 end test "more tests" do assert_equal 2, @value end end There are library which make end test/unit support context and declarative test , like contest
  43. 43. class UserTest < Test::Unit::TestCase context "A new User" do before do @user = User.first end test "should have the right full_name" do assert_equal "Dude Man", @user.full_name end end Or context
  44. 44. Rails support declarative test (but no nested context) class PostsControllerTest < ActionController::TestCase setup @post = posts(:one) end test "should show post" do get :show, :id => assert_response :success end end Even rails, It supports declarative test syntax. It use “test” block But no nested context.
  45. 45. BDD style element (3) Expectation
  46. 46. Expectation(Assertion) Minitest/Spec @order.amount.must_equal 1900 @order.amount.wont_equal 2000 lambda{ order.ship! }.must_raise NotPaidError BDD style assertion is called expectation. In menitest/spec, the methods are must_equal, wont_equal, must_raise In unit/test, some people may confuse assert method that first parameter and second parameter, which one is actual and which one is expect. But BDD style, it’s impossible to confuse it.
  47. 47. Expectation(Assertion) RSpec @order.amount.should == 1900 @order.amount.should_not == 2000 lambda{ order.ship! }.should_raise NotPaidError # or expect { order.ship! }.to raise_error(NotPaidError) Here is rspec version. should equal and should_not equal
  48. 48. Matchy x.should == 42 y.length.should_not be(4) lambda { raise "FAIL" }.should raise_error There is also library can make test/unit support bdd style expectation only.
  49. 49. BDD cares about the output
  50. 50. RSpec output :) Output (rspec) Really nice code documentation RSpec does the output very well, you can see really nice code documentation
  51. 51. HTML format
  52. 52. Minitest Output :( But I can’t not find how to generate code documentation in minitest/spec
  53. 53. require 'minitest/pride' # Show your testing pride! I found there is a minitest/pride you can required! It may help format the output, so I try it!
  54. 54. Very funny @_@ And here is the output....... very funny. Good job!
  55. 55. Use turn gem gem 'minitest' # use gem version instead of stdlib require 'minitest/spec' require 'turn' MiniTest::Unit.use_natural_language_case_names = true MiniTest::Unit.autorun Anyway, There is a gem called trun which used in Rails 3.1 It can generate nice test output for unit/test So here I require it. Bt w, I suggest you use the latest minitest gem version, not standard library. Since the gem version is much new. I totally agree tenderlove suggestion yesterday that we should move some standard library into gem.
  56. 56. Minitest::Spec output :) Here is the output, very nice.
  57. 57. Brief Summary • Three BDD style elements: • context • declarative test • expectation • test output can be code documentation
  58. 58. So... Why Syntax Matters?
  59. 59. Your language shape how you think! Language will not limit you, but it does influence how you think.
  60. 60. For example:
  61. 61. test_datetime_format # ruby/test/logger/test_logger.rb class TestLogger < Test::Unit::TestCase def test_datetime_format dummy = STDERR logger = log = log_add(logger, INFO, "foo") assert_match(/^dddd-dd-ddTdd:dd:dd.s*d+ $/, log.datetime) logger.datetime_format = "%d%b%Y@%H:%M:%S" log = log_add(logger, INFO, "foo") assert_match(/^ddwwwdddd@dd:dd:dd$/, log.datetime) logger.datetime_format = "" log = log_add(logger, INFO, "foo") assert_match(/^$/, log.datetime) end end This is a test code from ruby core for Logger it test datetime_format method
  62. 62. the same test in rubyspec # rubyspec/logger/logger/datetime_format_spec.rb describe "Logger#datetime_format" do # ... it "returns the date format used for the logs" it "returns nil logger is using the default date format" end describe "Logger#datetime_format=" do # ... it "sets the date format for the logs" do format = "%Y" @logger.datetime_format = "%Y" @logger.datetime_format.should == "%Y" @logger.add(Logger::WARN, "Test message") @log_file.rewind regex = /2[0-9]{3}.*Test message/ @log_file.readlines.first.should =~ regex end it "follows the Time#strftime format" end This is the same test in rubyspec you can see it divide into many describe and it block.
  63. 63. In this case: • Test::Unit version just do verify • Spec version not only verify but also describe its behavior: • It help us understand what’s the method should do • It increases the test coverage
  64. 64. The BDD syntax guides you focus on the expected behavior
  65. 65. Emphasis on Tests as Documentation
  66. 66. Rather than merely using tests for verification
  67. 67. 3. BDD style caveat
  68. 68. 1. Too magic?
  69. 69. Metaprogramming? • The BDD framework using metaprograming should not be the problem. • The problem is some (RSpec) advanced awesome syntax may make the test code hard to understand or not intuitive. Metaprogramming implementation should be the problem. If it is, then many ruby library is bad including rails. The problem is some advanced awesome syntax may hard to understand.
  70. 70. implicit subject (RSpec) describe Order do its(:status) { should == "New" } end For example. RSpec has a feature callsd implict subject. You can omit the receiver of expectation method. It also has a syntax call its, its...... its.... well, hard to explain... let see another equal version
  71. 71. Equal to this version: describe Order do before do @order = end it "should have default status is New" do @order.status.should == "New" end end oh.... I see... its will covert Order instance and call its method well, seems magic!
  72. 72. would like DRY? describe PostsController do describe "PUT #update" do before do 1 @post = Factory(:post) end it "allows an author to edit a post" do sign_in 2 put :update, :id =>, :body => "new content" 3 response.should be_successful end it "does not allow a non-author to edit a post" do sign_in Factory(:user) 2 put :update, :id =>, :body => "new content" 3 response.should be_forbidden end end Another example end to test rails controller: red 1 is setup part red 2 is exercise red 3 is verify The red 2 exercise are almost the same, so some people may want to DRY.
  73. 73. But readable? describe PostsController do describe "PUT #update" do 1 let(:post) { Factory(:post) } before do sign_in user 2 put :update, :id =>, :body => "new content" end context "when logged in as post's author" do let(:user) { } it "allows the post to be updated" do 3 response.should be_success end end context "when logged in as another user" do let(:user) { Factory(:user) } it "does not allow the post to be updated" do 3 response.should be_forbidden end end So they move the code into before block end and rspec has a feature called “let”, it’s a lazy and memorized method for setup end data. Finally in the test method, Only left assertion: response.should be_success
  74. 74. Four-phase Test form is key to understand the test code • Setup Why? I think for test code: • Exercise four-phase test form is the key to understand. • Verify (assertion) When I see one test method, I must need to figure out what’s the setup, exercise and verify. • Teardown So, it should be easy to trace.
  75. 75. 2.English-like DSL? the-language-in.html Second caveat is english-like DSL. Dave Thomas has a nice article explain this problem.
  76. 76. “The language in a DSL should be the language of the domain, not the natural language of the developer. “ by DaveThomas He said... so... To be english-like is not DSL’s purpose. The easy to use and readable DSL API is the purpose
  77. 77. # Bacon it 'should have an object identity' do end the be is just “self” Let’s see one obvious example. This is a bacon test code. There is a be method inside method chain, but it does nothing and just return self. It existed only because we want to make it like english. Is it right?
  78. 78. RSpec matcher (1) target.should be_true # targer.should == true target.should be_false # targer.should == false target.should be_nil # targer.should == nil target.should be_a_kind_of(Array) # target.class.should == Array target.should be_an_instance_of(Array) # target.class.should == Array On the other hand, RSpec has core feature called matcher for expectation. It also make the test code readable. In some way, It’s also very easy to use. like * be_true, * be_false * be_a_kind_of
  79. 79. RSpec matcher (2) target.should respond_to(:foo) # target.repond_to?(:foo).should == true target.should have_key(:foo) # target[:foo].should_not == nil target.should include(4) # target.include?(4).should == true target.should have(3).items # target.items.length == 3 There are more example like * have_key * include
  80. 80. RSpec matcher (3) target.should be_empty # target.empty?.should == true target.should be_blank # target.blank?.should == true target.should be_admin # target.admin?.should == true any prefix be_ matcher will covert it to that method with question mark.
  81. 81. RSpec matcher is very powerful # Custom Matcher describe 9 do it "should be a multiple of 3" do 9.should be_a_multiple_of(3) # (9 % 3).should == 0 end end # RSpec supports custom matchers DSL RSpec::Matchers.define :be_a_multiple_of do | expected| match do |actual| actual % expected == 0 end end RSpec matcher is very powerful since it even provide DSL to define your custom matcher which make the your test readable.
  82. 82. Some matchers I like, but some I dislike. It becomes personal taste :|
  83. 83. 3. Deep nested context? describe Order do before do 1 @order = end describe "#amount" do before do 1 @order2 = :total => 2000) end describe "when user is vip" do before do 1 @user = => true) end it "should discount five percent if total >= 1000" do # ... end end end There are three end nested context here. For the most inner test method, it has three before block outside.
  84. 84. Deep nested is hateful It’s hard to understand what’s setup going on in sub-sub-sub-sub context Deep nested context is hateful, It hard to trace the setup part when there is sub- sub-sub-sub context
  85. 85. Four-phase Test form is key to understand the test code • Setup Again. Four-phase Test form: • Exercise Deep context make setup part hard to trace. • Verify (assertion) That’s why Rails does not support nested context. • Teardown Rails team like flat test structure.
  86. 86. 4. Too mockist? • rspec-rails gem divides tests into model,controller,view,helper and encourages you use mocks. • should_receive easily make people test implementation rather than result by default. Make your test fragile. RSpec ship with very nice mock feature, And RSpec-rails divides tests into four layer which encourage you use mocks. But mocks has many caveats too, It make the test fragile easily.
  87. 87. 5.Performance penalty? • minitest/spec performance is almost equal to test/unit. It’s really fast. • RSpec performance is slower comparing to test/unit. minitest/spec performance is pretty well, you need not to worry about it. RSpec is slower, But when running test, I think the bottleneck is your production code and the testing framework factor is not big problem.
  88. 88. Conclusion
  89. 89. yes, it’s not required... • BDD testing framework is unlike ActiveRecord for SQL, Rails for web development. It does not “simplify” test. It’s not an abstract tool for testing. • So, yes, It’s not required and some people think it’s unnecessary.
  90. 90. But it’s very valuable for your team!
  91. 91. • The BDD syntax guides you focus on the expected behavior, and increasing the test coverage. • Emphasis on Tests as Documentation rather than merely using tests for verification. It focus on describe the expected behavior and increase the test coverage It emphasize tests as documentation, not just for verification.
  92. 92. Thanks for listening! Please tweet me if you have questions: @ihower
  93. 93. Reference • RSpec slides, ihower OSDC.TW 2011 • xUnit Test Patterns • the-literate/ • thoughts-on-testing-part-1.html