TESTING SURVIVAL GUIDE
                                                        Thilo Utke




A high level view on how to ...
WHY DO WE TEST?




why we take this extra step
TESTING CONS


                                   Write Extra Code

                                   Interrupt the Flow
...
TESTING PROS

                                  Think Double

                                  Avoid Errors

            ...
Failing Scenarios:

  cucumber features/currency_conversion.feature:129 # Scenario: Using the web-
  service supplied exch...
BUG IN CODE?
BUG IN TEST?
EXTERNAL DEPENDENCY?
A NEW CUCUMBER VERSION?
A BUG IN RUBY BIG DECIMAL!




forget to use rvm ruby
TESTS GIVE US CONFIDENCE
ALLOW US TO MOVE FORWARD




IMHO Lack of confidence reason why software in Enterprise tend to become outdated
Not to philo...
DOING TDD WITH RUBY IS
        EASY
class Test{

     //JMock & JUnit

     public void testWithCategoryNameGetNameOfPostCategory(){
       final Category cat...
# mocha

 def test_show_gets_name_of_post_category
   category = mock('category', :name => 'test') # mock, name must be ca...
DOING TDD RIGHT IS HARD




How often do you swear at your breaking test?
How often does you feel that tests break your flo...
Simple

                             Fast

                        Maintainable

                          Durable

      ...
GUIDELINES NOT RULES




1. Part
I assume the basics like using setups, cleaning up
mostly unittests
1. SIMPLICITY




Break down the domain problem in no brainers
THINK DOUBLE


      What I want     Presentation

     Required Info    Controller

How get these Info    DomainModel
STORY



    For customer information I want a listing of payment providers
    and their logos. Their logo should be link...
REDUCE TO NO-BRAINER
                            LOGO                        URL

                               X        ...
BE EXPLICIT

 it "should show the description for a payment provider" do
   payme = PaymentProvider.new(description: 'Pay ...
BE EXPLICIT


  it "should show the description for a payment provider" do
    payme = PaymentProvider.new(description: 'P...
1 THING AT A TIME


  payment_provider_listing(payme).should ==
  '<a href="pay.me">PayMe</a>, Get Paid'

  VS

  payment_...
2. DURABILITY
1 THING AT A TIME


payment_provider_listing(payme).should ==
'<a href="pay.me">PayMe</a>, Get Paid'

VS

payment_provider...
RELAX SPECIFICATION

  payment_provider_listing(payme).should
  match(/<a.*href="pay.me".*>PayMe</a>/)




Improves durabi...
RELAX SPECIFICATION
 it "should initialize the correct gateway with the order" do
   order = Order.make(:ecurrency => 'Gol...
MAINTAINABILITY
it "should return the last created owner as the current owner" do
      gateway = Gateway.create!(api_key: 'XYZ', url: 'ap...
it "should return the last created owner as the current owner" do
      @provider_with_two_owners.current_owner.should == ...
2. CONTEXT




Context is important
What are you dealing with
TOO NOISY


 it "should return the last created owner as the current owner" do
   gateway = Gateway.create!(api_key: 'XYZ'...
NO CONTEXT



     it "should return the last created owner as the current owner" do
       @provider_with_two_owners.curr...
MAINTAIN CONTEXT


    it "should return the last created owner as the current owner" do
      provider = Provider.make
  ...
SPLIT SETUP TO DRY
                     CONTEXT
  describe 'transaction' do
    before(:each) do
      @payer = User.make
...
BETTER UNDERSTANDABLE
             THAN DRY



others have to work with your code
3. SPEED




Most important for unit tests as you run them over and over again
TEST IN ISOLATION


  it "should be false when order has a single product from a single partner" do
    partner = Partner....
TEST IN ISOLATION


      it "should be false when order has a single product from a single partner" do
    product = Prod...
ISOLATION THROUGH
                  MOCKING



Instead of real dependencies inject mocks
different techniques
FAKES



 • Mimic  the behavior of the real object but don’t share all
    characteristics




good example are in memory ...
FAKE USAGE
   class FakeActivityLogger
  def log(object)
    @changes[object.id] ||= []
    @changes[object.id] << object....
STUBS



• Pretend   to be some object but without any logic
STUB USAGE

it "should change the given attribute" do
  logger = stub('stub_logger', log: true)
  @logger_config.register(...
MOCKS



• Pretend to be some object, also no logic but monitor if
 interaction with them is specified
MOCK USAGE


it "should call loggers on changes" do
  logger = mock('mock_logger')
  @logger_config.register(logger, User)...
MOCKS AND FAKES CAN
        HIDE INTEGRATION BUGS



Integration or Acceptancetests to the rescue
excessive use of mocking...
WRONG USAGE OF MOCKS
   HURT DURABILITY
MOCK BEHAVIOR UNDERT
      TEST AND STUB THE REST



Rule of thump
LISTEN TO YOUR TESTS
            “If something hurts you probably doing it wrong”




examples taken from real code I was ...
TO MANY DEPENDENCIES

 it "should be false when order has a single product from a single partner" do
   partner = Partner....
TO MANY DEPENDENCIES


• split   up

• add     layer

• decouple        logic
MANY MOCKS / SETUP FOR
           INTERNALS
  before(:each) do
    @converter = mock_model CurrencyConverter, convert: 4, ...
INTERNAL DEPENDENCIES



• Inject   Dependencies

• Iffrom callbacks, use a observer or think about Presenter/
  Service
STUB CHAINS/METHOD
                 CHAINS

 it "should be false when order has a single product from a single partner" do...
EXPLAINING COMMENTS ON
       EXPECTATIONS

it "should sum all credits for the partner" do
  credit1 = @partner.credits.ma...
TOOLS




help to write faster/better tests
beside your test/mock framework of choice
MORE INFRASTRUCTURE




more to maintain
BENEFIT > COST ?
SPORK


  + Reduce  startup time for testing frameworks (RSpec,
      Cucumber, Test-Unit)

  -   Reloading breaks for cod...
BUNDLER


  + full   dependency resolution at once

  + version    lockdown

  -   beta




fixed version are great, no sur...
FACTORIES
    (MACHINIST, FACTORY GIRL)

+ Greatly   remove noise in tests

+ Dry   Setups

+ Keep   Context

-   DB Overh...
HYDRA/PARALLEL SPEC


  + distribute    tests on multiple cores or even machines

  - extra setup

  - concurrency/load or...
CAPYBARA


 + Allow     to run cucumber features agains different backends

 + full   stack testing with culerity or selen...
WEBMOCK/SHAM_RACK



+ Allow   to fake or mock http apis

-   Don’t tell you when the real api changes ;)
ENVIRONMENTS



  + Allow    to isolate tools completely

  - Extra startup time




cucumber is doing it, you can do this...
INFO


• Name: Thilo   Utke

• Company: Upstream-Agile   GmbH

• Web: http://upstre.am

• Twitter: @freaklikeme
Upcoming SlideShare
Loading in...5
×

Testing survival Guide

2,015

Published on

In this talk that I gave at RailsWayCon I talk about practices that help to maintain readable, fast and simple test. I also show some examples where hard tests point to design issues. In the last part i introduced some tool that may help to maintain a good test suite.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,015
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
16
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Testing survival Guide

  1. 1. TESTING SURVIVAL GUIDE Thilo Utke A high level view on how to keep sane while doing tdd
  2. 2. WHY DO WE TEST? why we take this extra step
  3. 3. TESTING CONS Write Extra Code Interrupt the Flow Add Complexity to our Codebase More Maintenance the negative effects are either blends or can be minimized
  4. 4. TESTING PROS Think Double Avoid Errors Narrow Down Bugs Prevent Regression Improve Design This is what I get out of TDD
  5. 5. Failing Scenarios: cucumber features/currency_conversion.feature:129 # Scenario: Using the web- service supplied exchange rate for XAU cucumber features/statistics.feature:161 # Scenario: view statistics for completed withdrawals cucumber features/statistics.feature:189 # Scenario: view statistics for completed deposits cucumber features/transaction_confirmation.feature:6 # Scenario: Fund transactions must be confirmed before the funding is added to the balance In situations like this I ask myself the following questions
  6. 6. BUG IN CODE?
  7. 7. BUG IN TEST?
  8. 8. EXTERNAL DEPENDENCY?
  9. 9. A NEW CUCUMBER VERSION?
  10. 10. A BUG IN RUBY BIG DECIMAL! forget to use rvm ruby
  11. 11. TESTS GIVE US CONFIDENCE
  12. 12. ALLOW US TO MOVE FORWARD IMHO Lack of confidence reason why software in Enterprise tend to become outdated Not to philosophical
  13. 13. DOING TDD WITH RUBY IS EASY
  14. 14. class Test{ //JMock & JUnit public void testWithCategoryNameGetNameOfPostCategory(){ final Category category = context.mock(Category.class) //MockObjects final Post post = context.mock(Post.class) oneOf (post).name; will(returnValue('TestPost')); //Stub Methods oneOf (post).category; will(returnValue(category)) context.checking(new Expectations() {{ //Mock oneOf (category).name; will(returnValue('TestCat')) }} PostNamesWithCategoryLoader loader = new PostNamesWithCategoryLoader context.assertIsSatisfied(); } } might be improved a little (anotations?)
  15. 15. # mocha def test_show_gets_name_of_post_category category = mock('category', :name => 'test') # mock, name must be called post = stub('post', :category => category) #stub Post.stubs(:find).returns(post) #partial stub for find get :show end Ruby the better Language for Testing DSL
  16. 16. DOING TDD RIGHT IS HARD How often do you swear at your breaking test? How often does you feel that tests break your flow?
  17. 17. Simple Fast Maintainable Durable Side-effect Free Repeatable Thats what you want.
  18. 18. GUIDELINES NOT RULES 1. Part I assume the basics like using setups, cleaning up mostly unittests
  19. 19. 1. SIMPLICITY Break down the domain problem in no brainers
  20. 20. THINK DOUBLE What I want Presentation Required Info Controller How get these Info DomainModel
  21. 21. STORY For customer information I want a listing of payment providers and their logos. Their logo should be linked when I provide a link. If their is no logo, show the link instead. How many started to think of it as one problem to solve? Split it up in separate problems.
  22. 22. REDUCE TO NO-BRAINER LOGO URL X X X X Do this on complex problems and you will implement them easier, sometimes this will leave you wondering.
  23. 23. BE EXPLICIT it "should show the description for a payment provider" do payme = PaymentProvider.new(description: 'Pay me') payment_provider_listing(payme).should include(payme.description) end Not a good Idea.
  24. 24. BE EXPLICIT it "should show the description for a payment provider" do payme = PaymentProvider.new(description: 'Pay me') payment_provider_listing(payme).should include('Pay me') end Be explicit.
  25. 25. 1 THING AT A TIME payment_provider_listing(payme).should == '<a href="pay.me">PayMe</a>, Get Paid' VS payment_provider_listing(payme).should include('<a href="pay.me">PayMe</a>') That also relates to another problem
  26. 26. 2. DURABILITY
  27. 27. 1 THING AT A TIME payment_provider_listing(payme).should == '<a href="pay.me">PayMe</a>, Get Paid' VS payment_provider_listing(payme).should include('<a href="pay.me">PayMe</a>')
  28. 28. RELAX SPECIFICATION payment_provider_listing(payme).should match(/<a.*href="pay.me".*>PayMe</a>/) Improves durability but not so easy to read anymore. Some things contradict.
  29. 29. RELAX SPECIFICATION it "should initialize the correct gateway with the order" do order = Order.make(:ecurrency => 'GoldPay') GoldPayGateway.should_receive(:new).with(order, anything) GatewayFactory.build(order) end Next Topic
  30. 30. MAINTAINABILITY
  31. 31. it "should return the last created owner as the current owner" do gateway = Gateway.create!(api_key: 'XYZ', url: 'api.pay.me') provider = PaymentProvider.create!(url: 'pay.me', name: 'Payme', gateway: gateway) owner_1 = Owner.create!(first_name: 'Phil', name: 'Adams', provider: provider) owner_2 = Owner.create!(first_name: 'Maria', name: 'Williams', provider: provider) provider.current_owner.should == owner_2 end This is not good!
  32. 32. it "should return the last created owner as the current owner" do @provider_with_two_owners.current_owner.should == @owner_2 end This neither You ask why?
  33. 33. 2. CONTEXT Context is important What are you dealing with
  34. 34. TOO NOISY it "should return the last created owner as the current owner" do gateway = Gateway.create!(api_key: 'XYZ', url: 'api.pay.me') provider = PaymentProvider.create!(url: 'pay.me', name: 'Payme', gateway: gateway) owner_1 = Owner.create!(first_name: 'Phil', last_name: 'Adams', provider: provider) owner_2 = Owner.create!(first_name: 'Maria', last_name: 'Williams', provider: provider) provider.current_owner.should == owner_2 end To much
  35. 35. NO CONTEXT it "should return the last created owner as the current owner" do @provider_with_two_owners.current_owner.should == @owner_2 end To little
  36. 36. MAINTAIN CONTEXT it "should return the last created owner as the current owner" do provider = Provider.make owner_1 = Owner.make provider: provider owner_2 = Owner.make provider: provider provider.current_owner.should == owner_2 end right amout
  37. 37. SPLIT SETUP TO DRY CONTEXT describe 'transaction' do before(:each) do @payer = User.make end describe "percentage payout bonus set" do before(:each) do PayoutBonus.create!(amount: 20, unit: :percentage) end end describe "fixed payout bonus set" do before(:each) do PayoutBonus.create!(amount: 10, unit: :usd) end end end another contradiction because this also increases creates complexity by adding new places
  38. 38. BETTER UNDERSTANDABLE THAN DRY others have to work with your code
  39. 39. 3. SPEED Most important for unit tests as you run them over and over again
  40. 40. TEST IN ISOLATION it "should be false when order has a single product from a single partner" do partner = Partner.make product = Product.make new_partner_name: partner.name order = Order.make_unsaved partner: partner, contact: Contact.make, address: Address.make order.items.build product: product, price: 100, scale_basis: 1, quantity: 1 order.save! order.reload order.should_not have_multiple_product_partners end Most speed is gained if only that code executes that is necessary for that test
  41. 41. TEST IN ISOLATION it "should be false when order has a single product from a single partner" do product = Product.make new_partner_name: "Pear" order = Order.make_unsaved order.items << OrderItem.make_unsaved :product = product order.should_not have_multiple_product_partners end Most speed is gained if only that code executes that is necessary for that test
  42. 42. ISOLATION THROUGH MOCKING Instead of real dependencies inject mocks different techniques
  43. 43. FAKES • Mimic the behavior of the real object but don’t share all characteristics good example are in memory data storage vs. persistence.
  44. 44. FAKE USAGE class FakeActivityLogger def log(object) @changes[object.id] ||= [] @changes[object.id] << object.changes end def changes_for(object) @changes[object.id] end end it "should call loggers on changes" do logger = FakeActivityLogger.new @logger_config.register(logger, User) user = User.make(name: 'Paul') user.update_attribute(:name, 'Paula') logger.changes_for(user).should = [:name, 'Paul', 'Paula'] end
  45. 45. STUBS • Pretend to be some object but without any logic
  46. 46. STUB USAGE it "should change the given attribute" do logger = stub('stub_logger', log: true) @logger_config.register(logger, User) user = User.make(name: 'Paul') user.update_attribute(:name, 'Paula') user.name.should == 'Paula' end
  47. 47. MOCKS • Pretend to be some object, also no logic but monitor if interaction with them is specified
  48. 48. MOCK USAGE it "should call loggers on changes" do logger = mock('mock_logger') @logger_config.register(logger, User) user = User.make(name: 'Paul') logger.expects(:log).with(user).once user.update_attribute(:name, 'Paula') end
  49. 49. MOCKS AND FAKES CAN HIDE INTEGRATION BUGS Integration or Acceptancetests to the rescue excessive use of mocking my counteract fast testing if more integration test is required
  50. 50. WRONG USAGE OF MOCKS HURT DURABILITY
  51. 51. MOCK BEHAVIOR UNDERT TEST AND STUB THE REST Rule of thump
  52. 52. LISTEN TO YOUR TESTS “If something hurts you probably doing it wrong” examples taken from real code I was involved.
  53. 53. TO MANY DEPENDENCIES it "should be false when order has a single product from a single partner" do partner = Partner.make product = Product.make new_partner_name: partner.name order = Order.make_unsaved partner: partner, contact: Contact.make, address: Address.make order.items.build product: product, price: 100, scale_basis: 1, quantity: 1 order.save! order.reload order.should_not have_multiple_product_partners end Bad Design
  54. 54. TO MANY DEPENDENCIES • split up • add layer • decouple logic
  55. 55. MANY MOCKS / SETUP FOR INTERNALS before(:each) do @converter = mock_model CurrencyConverter, convert: 4, exchange_fee: 4, convertible?: true CurrencyConverter.stub!(:new).and_return(@converter) @modified_converter = mock_model ModifiedCurrencyConverter convert: 200 ModifiedCurrencyConverter.stub!(:new).and_return(@modified_converter) @user = mock_model(User, cleared_balance: 1000, add_balance_transaction: true, request_balance_transaction: true) @currency_conversion = CurrencyConversion.new source_amount: 100, source_currency: 'USD', destination_currency: 'EUR', user: @user end it "should request source amount plus fee from users source currency balance" do @user.should_receive(:request_balance_transaction).with(@currency_conversion, 104, 'USD') @currency_conversion.save! end 5 mocks/partial mocks
  56. 56. INTERNAL DEPENDENCIES • Inject Dependencies • Iffrom callbacks, use a observer or think about Presenter/ Service
  57. 57. STUB CHAINS/METHOD CHAINS it "should be false when order has a single product from a single partner" do order = Order.make_unsaved item = OrderItem.make_unsaved :name => 'iPet' order.stub_chain(:items, :delivered, :from_partner, :last => item) last_delivered_item_for_partner_label(partner, order).should include('iPed') end artificial exposes to many internals
  58. 58. EXPLAINING COMMENTS ON EXPECTATIONS it "should sum all credits for the partner" do credit1 = @partner.credits.make(:order => @order, :payment => 100) credit1.items.make :price => 100, :quantity => 1 credit2 = @partner.credits.make(:order => @order, :payment => 100) credit2.items.make :price => 150, :quantity => 1 @partner.credits.sum_for_month(Date.today.month, Date.today.year).should == 297.5 # including 19% tax end
  59. 59. TOOLS help to write faster/better tests beside your test/mock framework of choice
  60. 60. MORE INFRASTRUCTURE more to maintain
  61. 61. BENEFIT > COST ?
  62. 62. SPORK + Reduce startup time for testing frameworks (RSpec, Cucumber, Test-Unit) - Reloading breaks for code loaded in environment/ initializers Great timesaver in unittest for bigger projects with lot of gems and plugins
  63. 63. BUNDLER + full dependency resolution at once + version lockdown - beta fixed version are great, no surprises with unexpected updates.
  64. 64. FACTORIES (MACHINIST, FACTORY GIRL) + Greatly remove noise in tests + Dry Setups + Keep Context - DB Overhead
  65. 65. HYDRA/PARALLEL SPEC + distribute tests on multiple cores or even machines - extra setup - concurrency/load order issues So far no serious project running with them
  66. 66. CAPYBARA + Allow to run cucumber features agains different backends + full stack testing with culerity or selenium where required - not one feature on many backends setup is a super easy with cucumber
  67. 67. WEBMOCK/SHAM_RACK + Allow to fake or mock http apis - Don’t tell you when the real api changes ;)
  68. 68. ENVIRONMENTS + Allow to isolate tools completely - Extra startup time cucumber is doing it, you can do this too.
  69. 69. INFO • Name: Thilo Utke • Company: Upstream-Agile GmbH • Web: http://upstre.am • Twitter: @freaklikeme
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×