Here are some potential caveats with BDD style testing: 1. The framework can feel too "magical" if it uses too much metaprogramming under the hood. The implementation details should not be obscured. 2. Descriptions can become too vague if they only state intentions at a very high level without concrete examples. Tests as documentation only works if the descriptions are clear and specific. 3. There is a risk of duplication if contexts or examples are not properly organized and nested. Descriptions should aim to be DRY. 4. Performance overhead of the framework if it does a lot of runtime reflection or string manipulation. The framework should have a small performance footprint. 5. Tests become brittle

1. How BDD style Unit Testing
helps you write better
test code
(and some caveat)
@ihower at RubyKaigi 2011
Ruby Taiwan & OptimisDev

2. About Me
• Chang, Wen-Tien
• a.k.a. ihower
• http://ihower.tw
• http://twitter.com/ihower
I’m ihower and here is my blog and t witter
I have program ruby from 2006 for ruby on rails
And I’m Yet another rubyist since rails.

3. From Taiwan
3.5hrs
I come from Taiwan which is far away about 4hours flight
I’m sorry that I can’t speak Japanese, and English is not my native language
If I speak Chinese, I guess you will escape now.
So please forgive my english. I hope it’s simple to understand.

4. I work at Optimis International, which is an america company in Taipei

5. Ruby Taiwan
http://ruby.tw
Beside daily job, I’m the founder and organizer of Ruby Taiwan community, I found
the community since 2008.

6. RubyConf Taiwan 2011/8/26-27
http://rubyconf.tw
We are organizing rubyconf taiwan now.
The date is augest 26 and 27 and it open registration now.
Welcome to Taiwan.

7. Last year photo. Nice small meeting room. pretty sweet.

8. Agenda
• Why Unit Testing?
• How to do unit test in BDD style?
• Some BDD caveat
Today’s agenda. There will be three part:
First, I will talk about why, then how to do, finally talk about some caveat.

9. How many people do
unit test?
Before get into BDD, how many people here do unit test all the way?
Could you raise your hand?

10. How many people use
BDD framework
(RSpec)?
And how many people use rspec?

11. Two kinds of test:
Verification
Unit Test Are we writing the code right?
Customer Test Validation
(Acceptance Test) Are we writing the right software?
There are roughly t wo kinds of test:
Unit test and Customer test(or called Acceptance test)
Unit test is about verification, the question is are we writing the code right?
Customer test is about validation, the question is are we writing the right soft ware?

12. Two kinds of test:
Test individual class and
Unit Test method
Customer Test Test functionality of the
(Acceptance Test) over-all system
Unit test test individual class and method
And Customer Test test functionality of the over-all system

13. Today is all about
Unit Testing
Today is all about unit testing

14. 1. Why Unit Testing?
Let get started part one

15. 1.Verification
Do we write the code right?
First, it does verification

16. How to verify this?There are three execution path
def correct_year_month(year, month)
if month > 12
if month % 12 == 0
year += (month - 12) / 12
month = 12
else
year += month / 12
month = month % 12
end
end
return [year, month]
end
For example, how to verify this simple if else code.
There are three execution path which means if you manually test
you need play it 3 times in different way

17. via UI?

18. via irb console?

19. Manually test is
inefficient

20. Unit testing give us
instant coding feedback
it "should return correct year and month" do
correct_year_month(2011, 3).should == [2011, 3]
correct_year_month(2011, 14).should == [2012,2]
correct_year_month(2011, 24).should == [2012,12]
end
automatic unit testing can give us instant feedback
you can test many times you want.

21. 2.Check regression
Safety Net: when you add new feature and refactoring
test also check the regression
it’s a safety net when you add new feature or refactoring
It can give you confidence that you won’t break up any existed behavior

22. 3.API design
guide you write better API, especially test first.
third, Since test code is the client for your production code,
so if the test code is hard to write, it may means your API is not good.
Writing unit testing can guide you write better API, especially when you write
test first.

23. 4.Documentation
What the result should be?
Communication is the key for software development.
finally, uniting test can be your code documentation.
It can answer you what’s the class and method result should be?
Communication the key for successful soft ware development.

24. Brief Summary
• Your code is not trivial:
Automatic test can save your verify/debug time
• Your code is not throwing way:
Regression test
• Test first: Guide you write better API
• Tests as Documentation

25. 2. How to do unit test
in BDD style?

26. xUnit Framework
• Each test has four-phase:
• Setup
• Exercise
• Verify (assertion)
• Teardown
First, We use xUnit framework to test code.
Each test should be isolation and has four phase:
1. Setup Data
2. Exercise production code
3. Verify the result 4. Cleanup the data

27. Ruby Test::Unit
class OrderTest < Test::Unit::TestCase
def setup
1 @order = Order.new
end
def test_order_amount
assert_equal 0, @order.amount
end 3 2
end
The is the classical ruby test::unit code
Red 1 is setup
Red 2 is exercise
Red 3 is verify. We assert the result equal to zero

28. Minitest::Spec
describe Order do
before do
1 @order = Order.new
end
it "should have default amount is 0" do
@order.amount.must_equal 0
end 2 3
end
Here is the BDD style using minitest::spec
red 1 is setup
red 2 is exercise
red 3 is verify. the result must equal to zero

29. First impression
about BDD
• Syntax is different
• Like specification
• More readable
So, what’s the first impression about bdd style?
Ok, the syntax is different, it looks like spec and may more readable
Other wise the structure is not much different

30. What’s BDD?
• An improved xUnit Framework
• Focus on clearly describe the expected
behavior
• The emphasis is Tests as Documentation
rather than merely using tests for
verification.
Let’s back to see what’s BDD means.
It’s not totally new test framework, it just an improved xUnit framework
It focus on describe the expected behavior
It emphasize tests as documentation, not just for verification.

31. Terminology changed
New paradigm: Executable Specification
• “Test” becomes “Spec”
• “Assertion” becomes “Expectation”
• “test method” becomes “example” (RSpec)
• “test case” becomes “example group” (RSpec)
For correspond to the new paradigm: the test is the executable specification
The “test” becomes “spec”
“Assertion” becomes “Expectation”

32. Ruby BDD tools
• RSpec
• Minitest/Spec
In Ruby, there are some choices:
RSpec is the most powerful, has many features and many awesome syntax.
Minitest/spec is the Ruby 1.9 build-in xunit library. It’s simple, fast, and just
works!

33. Learn BDD style
• It’s about syntax
• syntax
• syntax
• syntax
Let’s learn how to write BDD style test code.
Basically, it’s just learn about syntax.

34. BDD style element (1)
Context

35. one test case
describe Order do
# ...
end a class or
string
# or
describe "A Order" do
# ...
end
Instead of test case class, BDD style use “describe” block for one test case
It accept parameter which means the stuff you want to describe, usually a class
name or just description text.

36. Can be nested
describe Order do
describe "#amount" do
# ... #method means
end instance method
describe ".size" do
# ...
end
.method means
class method
end
“describe” block can be nested.
And inner “describe” usually describe method.
Pound means instance method
dot means class method

37. In RSpec,
alias_method :context, :describe
1 describe Order do
2 context "when initialized" do
3 describe "#amount" do
# ...
end
3 describe ".size" do
# ...
end
end
Is RSpec, it alsoend describe to context.
alias
it used to organize test code into different context.

38. BDD style element (2)
Declarative test

39. one test method: it
you can write test description in plain text
describe Order do
describe "#amount" do
describe "when user is vip" do
it "should discount five percent if total > 1000" do
# ...
end
it "should discount ten percent if total > 10000" do
# ...
end
end
end
Each test method is “it” block
And it accept one parameter which is description text

40. Setup method: before
describe Order do
before do
@user = User.new( :is_vip => true )
@order = Order.new( :user => @user )
end
end
the setup method is “before” block

41. nested before
btw, the outer data will be shared, make the test faster
describe Order do
1 before do
@user = User.new( :is_vip => true )
@order = Order.new( :user => @user )
end
describe "#amount" do
2 before do
@order2 = Order.new( :user => @user )
end
end
“before” block also support nested
end
More important, the outer setup data will be shard for all inside test methods.
Make the test faster since we need not create again every time.

42. https://github.com/citrusbyte/contest
class SomeTest < Test::Unit::TestCase
setup do
@value = 1
end
test "sample test" do
assert_equal 1, @value
end
context "a context" do
setup do
@value += 1
end
test "more tests" do
assert_equal 2, @value
end
end
There are library which make
end test/unit support context and declarative test ,
like contest

43. https://github.com/jm/context
class UserTest < Test::Unit::TestCase
context "A new User" do
before do
@user = User.first
end
test "should have the right full_name" do
assert_equal "Dude Man", @user.full_name
end
end
Or context

44. Rails support declarative test
(but no nested context)
class PostsControllerTest < ActionController::TestCase
setup
@post = posts(:one)
end
test "should show post" do
get :show, :id => @post.id
assert_response :success
end
end
Even rails, It supports declarative test syntax. It use “test” block
But no nested context.

45. BDD style element (3)
Expectation

46. Expectation(Assertion)
Minitest/Spec
@order.amount.must_equal 1900
@order.amount.wont_equal 2000
lambda{ order.ship! }.must_raise NotPaidError
BDD style assertion is called expectation.
In menitest/spec, the methods are must_equal, wont_equal, must_raise
In unit/test, some people may confuse assert method that first parameter and
second parameter, which one is actual and which one is expect. But BDD style, it’s
impossible to confuse it.

47. Expectation(Assertion)
RSpec
@order.amount.should == 1900
@order.amount.should_not == 2000
lambda{ order.ship! }.should_raise NotPaidError
# or
expect { order.ship! }.to raise_error(NotPaidError)
Here is rspec version. should equal and should_not equal

48. Matchy
https://github.com/jm/matchy
x.should == 42
y.length.should_not be(4)
lambda { raise "FAIL" }.should raise_error
There is also library can make test/unit support bdd style expectation only.

49. BDD cares about
the output

50. RSpec output :)
Output (rspec)
Really nice code
documentation
RSpec does the output very well, you can see really nice code documentation

51. HTML format

52. Minitest Output :(
But I can’t not find how to generate code documentation in minitest/spec

53. require 'minitest/pride'
# Show your testing pride!
I found there is a minitest/pride you can required! It may help format the output,
so I try it!

54. Very funny @_@
And here is the output....... very funny. Good job!

55. Use turn gem
gem 'minitest' # use gem version instead of stdlib
require 'minitest/spec'
require 'turn'
MiniTest::Unit.use_natural_language_case_names = true
MiniTest::Unit.autorun
Anyway, There is a gem called trun which used in Rails 3.1
It can generate nice test output for unit/test
So here I require it.
Bt w, I suggest you use the latest minitest gem version, not standard library. Since
the gem version is much new.
I totally agree tenderlove suggestion yesterday that we should move some
standard library into gem.

56. Minitest::Spec output :)
Here is the output, very nice.

57. Brief Summary
• Three BDD style elements:
• context
• declarative test
• expectation
• test output can be code documentation

58. So...
Why Syntax Matters?

59. Your language shape how
you think!
Language will not limit you, but it does influence how you think.
http://www.nytimes.com/2010/08/29/magazine/29language-t.html

60. For example:

61. test_datetime_format
# ruby/test/logger/test_logger.rb
class TestLogger < Test::Unit::TestCase
def test_datetime_format
dummy = STDERR
logger = Logger.new(dummy)
log = log_add(logger, INFO, "foo")
assert_match(/^dddd-dd-ddTdd:dd:dd.s*d+ $/, log.datetime)
logger.datetime_format = "%d%b%Y@%H:%M:%S"
log = log_add(logger, INFO, "foo")
assert_match(/^ddwwwdddd@dd:dd:dd$/, log.datetime)
logger.datetime_format = ""
log = log_add(logger, INFO, "foo")
assert_match(/^$/, log.datetime)
end
end
This is a test code from ruby core for Logger
it test datetime_format method

62. the same test in rubyspec
# rubyspec/logger/logger/datetime_format_spec.rb
describe "Logger#datetime_format" do
# ...
it "returns the date format used for the logs"
it "returns nil logger is using the default date format"
end
describe "Logger#datetime_format=" do
# ...
it "sets the date format for the logs" do
format = "%Y"
@logger.datetime_format = "%Y"
@logger.datetime_format.should == "%Y"
@logger.add(Logger::WARN, "Test message")
@log_file.rewind
regex = /2[0-9]{3}.*Test message/
@log_file.readlines.first.should =~ regex
end
it "follows the Time#strftime format"
end
This is the same test in rubyspec
you can see it divide into many describe and it block.

63. In this case:
• Test::Unit version just do verify
• Spec version not only verify but also describe its
behavior:
• It help us understand what’s the method should do
• It increases the test coverage

64. The BDD syntax guides
you focus on the
expected behavior

65. Emphasis on
Tests as Documentation

66. Rather than merely
using tests for
verification

67. 3. BDD style caveat

68. 1. Too magic?

69. Metaprogramming?
• The BDD framework using metaprograming
should not be the problem.
• The problem is some (RSpec) advanced
awesome syntax may make the test code hard
to understand or not intuitive.
Metaprogramming implementation should be the problem.
If it is, then many ruby library is bad including rails.
The problem is some advanced awesome syntax may hard to understand.

70. implicit subject
(RSpec)
describe Order do
its(:status) { should == "New" }
end
For example. RSpec has a feature callsd implict subject.
You can omit the receiver of expectation method.
It also has a syntax call its, its...... its....
well, hard to explain... let see another equal version

71. Equal to this version:
describe Order do
before do
@order = Order.new
end
it "should have default status is New" do
@order.status.should == "New"
end
end
oh.... I see...
its will covert Order instance and call its method
well, seems magic!

72. would like DRY?
describe PostsController do
describe "PUT #update" do
before do
1 @post = Factory(:post)
end
it "allows an author to edit a post" do
sign_in @post.author
2 put :update, :id => post.id, :body => "new content"
3 response.should be_successful
end
it "does not allow a non-author to edit a post" do
sign_in Factory(:user)
2 put :update, :id => post.id, :body => "new content"
3 response.should be_forbidden
end
end
Another example
end to test rails controller:
red 1 is setup part
red 2 is exercise
red 3 is verify
The red 2 exercise are almost the same, so some people may want to DRY.

73. But readable?
describe PostsController do
describe "PUT #update" do
1 let(:post) { Factory(:post) }
before do
sign_in user
2 put :update, :id => post.id, :body => "new content"
end
context "when logged in as post's author" do
let(:user) { post.author }
it "allows the post to be updated" do
3 response.should be_success
end
end
context "when logged in as another user" do
let(:user) { Factory(:user) }
it "does not allow the post to be updated" do
3 response.should be_forbidden
end
end
So they move the code into before block
end
and rspec has a feature called “let”, it’s a lazy and memorized method for setup
end
data.
Finally in the test method, Only left assertion: response.should be_success

74. Four-phase Test form is
key to understand the
test code
• Setup Why?
I think for test code:
• Exercise four-phase test form is the
key to understand.
• Verify (assertion)
When I see one test method, I
must need to figure out what’s
the setup, exercise and verify.
• Teardown So, it should be easy to trace.

75. 2.English-like DSL?
http://pragdave.blogs.pragprog.com/pragdave/2008/03/
the-language-in.html
Second caveat is english-like DSL.
Dave Thomas has a nice article explain this problem.

76. “The language in a DSL should be the
language of the domain, not the natural
language of the developer. “ by DaveThomas
He said...
so... To be english-like is not DSL’s purpose.
The easy to use and readable DSL API is the purpose

77. # Bacon
it 'should have an object identity' do
@ary.should.not.be.same_as Array.new
end
the be is just “self”
Let’s see one obvious example. This is a bacon test code.
There is a be method inside method chain, but it does nothing and just return self.
It existed only because we want to make it like english. Is it right?

78. RSpec matcher (1)
target.should be_true # targer.should == true
target.should be_false # targer.should == false
target.should be_nil # targer.should == nil
target.should be_a_kind_of(Array) # target.class.should == Array
target.should be_an_instance_of(Array) # target.class.should == Array
On the other hand, RSpec has core feature called matcher for expectation.
It also make the test code readable. In some way, It’s also very easy to use.
like
* be_true,
* be_false
* be_a_kind_of

79. RSpec matcher (2)
target.should respond_to(:foo)
# target.repond_to?(:foo).should == true
target.should have_key(:foo)
# target[:foo].should_not == nil
target.should include(4)
# target.include?(4).should == true
target.should have(3).items
# target.items.length == 3
There are more example like
* have_key
* include

80. RSpec matcher (3)
target.should be_empty
# target.empty?.should == true
target.should be_blank
# target.blank?.should == true
target.should be_admin
# target.admin?.should == true
any prefix be_ matcher will covert it to that method with question mark.

81. RSpec matcher is very
powerful
# Custom Matcher
describe 9 do
it "should be a multiple of 3" do
9.should be_a_multiple_of(3)
# (9 % 3).should == 0
end
end
# RSpec supports custom matchers DSL
RSpec::Matchers.define :be_a_multiple_of do |
expected|
match do |actual|
actual % expected == 0
end
end
RSpec matcher is very powerful since it even provide DSL to define your custom
matcher which make the your test readable.

82. Some matchers I like,
but some I dislike.
It becomes personal taste :|

83. 3. Deep nested context?
describe Order do
before do
1 @order = Order.new
end
describe "#amount" do
before do
1 @order2 = Order.new( :total => 2000)
end
describe "when user is vip" do
before do
1 @user = User.new(:is_vip => true)
end
it "should discount five percent if total >= 1000" do
# ...
end
end
end
There are three
end nested context here. For the most inner test method, it has
three before block outside.

84. Deep nested is hateful
It’s hard to understand what’s setup going on in sub-sub-sub-sub context
Deep nested context is hateful, It hard to trace the setup part when there is sub-
sub-sub-sub context

85. Four-phase Test form is
key to understand the
test code
• Setup Again. Four-phase Test form:
• Exercise Deep context make setup part
hard to trace.
• Verify (assertion)
That’s why Rails does not
support nested context.
• Teardown Rails team like flat test
structure.

86. 4. Too mockist?
• rspec-rails gem divides tests into
model,controller,view,helper and encourages
you use mocks.
• should_receive easily make people test
implementation rather than result by default.
Make your test fragile.
RSpec ship with very nice mock feature,
And RSpec-rails divides tests into four layer which encourage you use mocks.
But mocks has many caveats too, It make the test fragile easily.

87. 5.Performance penalty?
• minitest/spec performance is almost equal to
test/unit. It’s really fast.
• RSpec performance is slower comparing to
test/unit.
minitest/spec performance is pretty well, you need not to worry about it.
RSpec is slower, But when running test, I think the bottleneck is your
production code and the testing framework factor is not big problem.

88. Conclusion

89. yes, it’s not required...
• BDD testing framework is unlike
ActiveRecord for SQL, Rails for web
development. It does not “simplify” test. It’s
not an abstract tool for testing.
• So, yes, It’s not required and some people
think it’s unnecessary.

90. But it’s very valuable
for your team!

91. • The BDD syntax guides you focus on
the expected behavior, and increasing
the test coverage.
• Emphasis on Tests as Documentation
rather than merely using tests for
verification.
It focus on describe the expected behavior and increase the test coverage
It emphasize tests as documentation, not just for verification.

92. Thanks for listening!
Please tweet me if you have questions:
@ihower

93. Reference
• RSpec slides, ihower
OSDC.TW 2011
• xUnit Test Patterns
• http://avdi.org/devblog/2011/04/07/rspec-is-for-
the-literate/
• http://objectreload.com/articles/2010/09/
thoughts-on-testing-part-1.html