Testing
Finding the Middle Path




                          source: http://flic.kr/p/6v82qN
The
Flickr
Way




         source: http://flic.kr/p/6MQYyj
The QA Way




             source: http://flic.kr/p/tvUHq
Test Everything Way




                      source: http://flic.kr/p/7kKDuo
TATFT




        source: http://flic.kr/p/7kKDuo
Test All The Fucking Time




                       source: http://flic.kr/p/7kKDuo
Overloaded With Tests




                    source: http://flic.kr/p/GNhm3
Balance




          source: http://flic.kr/p/3enph8
Finding The Way




                  source: http://flic.kr/p/5tZaUe
On Rails




           source: http://flic.kr/p/4WR1F9
Test All At Once




                   source: http://flic.kr/p/7ZSbRQ
Don’t Test To Narrowly




                     source: http://flic.kr/p/6Zeo1V
Don’t Test To Broadly




                        source: http://flic.kr/p/7azreS
Testing What
Is Already Tested




                    source: http://flic.kr/p/3c89jn
The Database
Cucumber




           source: http://flic.kr/p/5aZYkP
What You Should Do




                     source: http://flic.kr/p/7qQZni
Integration




              source: http://flic.kr/p/kZtKo
Test The Bugs




                source: http://flic.kr/p/JAE3v
Refactoring




              source: http://flic.kr/p/88X9DY
Test Then Change




                   source: http://flic.kr/p/aCDk7
Mocky Stubbly Things




                       source: http://flic.kr/p/6Q6en
Make Tests Fast




                  source: http://flic.kr/p/5tZaUe
Make Tests Fast

          TESTING




    TESTING!




                    source: http://flic.kr/p/5tZaUe
Fixtures


light fixture?




                source: http://flic.kr/p/5tZaUe
Data For Tests




                 source: http://flic.kr/p/5tZaUe
true is not false




                    source: http://flic.kr/p/5tZaUe
Tests Aren’t DRY




                   source: http://flic.kr/p/5tZaUe
They’re Damp




               source: http://flic.kr/p/5tZaUe
Atomic




         source: http://flic.kr/p/5tZaUe
Atomic Failure




                 source: http://flic.kr/p/5tZaUe
Outside In




             source: http://flic.kr/p/4KgEv
Self Contained




                 source: http://flic.kr/p/5ZFbHE
Red Green




            source: http://flic.kr/p/5tZaUe
That’s My Middle Path
@rabble - cuboxsa.com




                        source: http://flic.kr/p/6v82qN
Upcoming SlideShare
Loading in...5
×

Finding the Middle Way of Testing

2,374

Published on

Talk about what should and shouldn't be tested in web application development. How to find the sweet spot of testing.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,374
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
38
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide
  • I’ve wanted to do this talk for a while. I’ve given a bunch of talks on the mechanics of testing. But i feel like there is something missing. We need to talk more about what to test and what not to test. There is a kind of macho, who can test the We need a balance
  • One way of thinking of testing is to not do them, but to release often, quickly, write code with feature flags. I like to call this the flickr way, it works. It’s very similar to how facebook works really. Lots of small iterations. No tests, in code, you’ve got users for that! If the space between releases is really short, this model works.
  • http://flic.kr/p/tvUHq
  • The other direction is testing everything. You want it to be like this. Rushing along, everything is a blur as you race through development. This is called the TATFT method of software development.
    http://flic.kr/p/7kKDuo

  • This is the ‘test all the fucking time’ method of software development. It’s appealing because it’s hard core. Alpha male, Test First, Test Always style.
  • This is the ‘test all the fucking time’ method of software development. It’s appealing because it’s hard core. Alpha male, Test First, Test Always style.
  • But if done to extreme, without regard to utility, you get this. Burdened down by your tests.
    http://flic.kr/p/GNhm3
  • Sometimes we need to find balance between these ways. Testing is a technique we use to make better software.
  • So the question is then what’s the middle way. What rules should we use to decide what should and shouldn’t be tested. I’m going to go through a bunch of best and worst practices. My rules of thumb for the way of testing.
  • n ways to find the middle way in testing. What to test, what not to test, and when to know the difference. http://flic.kr/p/8kAmwf
  • I do rails development, the examples come from rails. But the point is try and extract ideas which are useful in many technology stacks.
  • The worse of the bad testing techniques is to go from no tests to spending weeks just writing tests. To catch up!

    http://flic.kr/p/7ZSbRQ
  • It’s important not to test to little, to be to focused on the details. You end up testing the implementation. Tests fail, but not because the software as a whole is broken. The simple, Assert 2, 1+1. It’s not interesting. http://flic.kr/p/6Zeo1V
  • Just as testing the details of an implementation, is bad, testing to broadly also can fail for two reasons. First is time. Testing is a tool for debugging. Debugging is what we are really doing most of the time when we say we’re programming. http://flic.kr/p/7azreS
  • One classic problem is when people start writing tests is that they are testing their use of somebody else’s library. Presumably that library should have tests itself, or if it doesn’t has at least a stable API. Treat it as a black box. http://flic.kr/p/3c89jn
  • One example in Rails is to do tests that confirm Active Record is working correctly. That a has_many association can be created and deleted.
  • I know there was just a talk about Gherkin, the extension of cucumber to make it work in multiple human languages. But i think you should just say no to Cucumber. Clients don’t want to write them. More to the point they can’t do them well. It’s a fascinating exercise, but cucumber stories do not translate in to good code. It’s bulky, nobody reads the stories, code when it fails is easier to understand.

    http://flic.kr/p/5aZYkP


  • http://flic.kr/p/7qQZni
  • Even though natural language tests like cucumber are, in my humble opinion a disaster, integration tests are a great idea. They’re code, they are ways of dealing with our web apps in a way similar to real use, but simulated. We’re mocking the real browser, and replacing it with something more useful for testing.

    http://flic.kr/p/kZtKo
  • This is what you should do. Test the bugs. Write tests when things break. What you should do is manually try and reproduce a bug. Then once you can ‘see’ it, then you write the test. Then with the test, you can fix it. This is the way to do it. Don’t do that sprint thing.

    http://flic.kr/p/JAE3v

  • When you go in an refactor, then is vital. Really this is the only time it makes sense to spend any time writing just tests.

    http://flic.kr/p/88X9DY
  • So when you’re refactoring, as opposed to updating, then you need tests. Because by definition, refactoring, is to change the implementation without changing the functionality. So you need a test to confirm the functionality is the same.

    http://flic.kr/p/aCDk7
  • The mocky stubbly things. Taking functionality out and replacing it with fakes. There be dragons here. On the one hand, if you have the full system, it’s slow to test. Things like networks, payment processing gateways, they aren’t things you want to hit when

    http://flic.kr/p/6Q6en
  • Show XKCD comic of compiling....

    Testing is the new compiling. Perhaps we needed the cognitive space. A break in our work, and so in interpreted languages we started writing tests, to fill the gap which used to be created by compiling.


  • Show XKCD comic of compiling....

    Testing is the new compiling. Perhaps we needed the cognitive space. A break in our work, and so in interpreted languages we started writing tests, to fill the gap which used to be created by compiling.


  • http://flic.kr/p/4heE23


  • http://flic.kr/p/55d56F
  • One lesson from the BDD experience is that you assertions must make sense. Think of writing the tests so when they fail, the test name makes sense, and the failure is directed.

    http://flic.kr/p/4Jrqs1
  • Tests are different from application code. It’s ok to repeat yourself. Some.

    http://flic.kr/p/gLUeh
  • The tests should be damp. As in, repeat yourself some. Because each test is of a variation of functionality, it requires some repetition. Not DRY, but damp as it were.

    http://flic.kr/p/4qBKRw
  • Your tests should be atomic. Each test has the same environment. This means your fixtures need to work well. If one test has an effect on another, then you’re in for a world for hurt. You know what happens when you start smashing volatile atoms together, right?

    http://flic.kr/p/7Gssw1
  • If you fail to keep your tests separate from each other, things fall apart

    http://en.wikipedia.org/wiki/Nuclear_explosion
  • I strongly agree with what Trotter said in the previous talk. Outside In. Finding the sweet spot, outside of your application, outside of the implementation, which you can test.
  • Similar to the atomic thing. You want your tests to be self contained. A single activity or thing. The tricky part here is to keep each test self contained, a bit of functionality, a unit as it were, but not so focused as to be a sub-unit.


    http://flic.kr/p/5ZFbHE
  • Easily be able to see what works and what doesn’t, clear red green results. If you’ve got one of those grahpical IDE things, then use that. The idea is you have to easily, automatically, know what works.

    So, it’s code, look to see if worked, code, see if it worked, code, see if it worked. That is the old school model of software development.

    http://flic.kr/p/7vVAJW
  • That’s my middle path, there is no one path. But i think we need to start calling out what we want, what we do, what we shouldn’t do in testing. While there may be more than one way to do it, all the ways aren’t the same, some work, some don’t.

  • Finding the Middle Way of Testing

    1. 1. Testing Finding the Middle Path source: http://flic.kr/p/6v82qN
    2. 2. The Flickr Way source: http://flic.kr/p/6MQYyj
    3. 3. The QA Way source: http://flic.kr/p/tvUHq
    4. 4. Test Everything Way source: http://flic.kr/p/7kKDuo
    5. 5. TATFT source: http://flic.kr/p/7kKDuo
    6. 6. Test All The Fucking Time source: http://flic.kr/p/7kKDuo
    7. 7. Overloaded With Tests source: http://flic.kr/p/GNhm3
    8. 8. Balance source: http://flic.kr/p/3enph8
    9. 9. Finding The Way source: http://flic.kr/p/5tZaUe
    10. 10. On Rails source: http://flic.kr/p/4WR1F9
    11. 11. Test All At Once source: http://flic.kr/p/7ZSbRQ
    12. 12. Don’t Test To Narrowly source: http://flic.kr/p/6Zeo1V
    13. 13. Don’t Test To Broadly source: http://flic.kr/p/7azreS
    14. 14. Testing What Is Already Tested source: http://flic.kr/p/3c89jn
    15. 15. The Database
    16. 16. Cucumber source: http://flic.kr/p/5aZYkP
    17. 17. What You Should Do source: http://flic.kr/p/7qQZni
    18. 18. Integration source: http://flic.kr/p/kZtKo
    19. 19. Test The Bugs source: http://flic.kr/p/JAE3v
    20. 20. Refactoring source: http://flic.kr/p/88X9DY
    21. 21. Test Then Change source: http://flic.kr/p/aCDk7
    22. 22. Mocky Stubbly Things source: http://flic.kr/p/6Q6en
    23. 23. Make Tests Fast source: http://flic.kr/p/5tZaUe
    24. 24. Make Tests Fast TESTING TESTING! source: http://flic.kr/p/5tZaUe
    25. 25. Fixtures light fixture? source: http://flic.kr/p/5tZaUe
    26. 26. Data For Tests source: http://flic.kr/p/5tZaUe
    27. 27. true is not false source: http://flic.kr/p/5tZaUe
    28. 28. Tests Aren’t DRY source: http://flic.kr/p/5tZaUe
    29. 29. They’re Damp source: http://flic.kr/p/5tZaUe
    30. 30. Atomic source: http://flic.kr/p/5tZaUe
    31. 31. Atomic Failure source: http://flic.kr/p/5tZaUe
    32. 32. Outside In source: http://flic.kr/p/4KgEv
    33. 33. Self Contained source: http://flic.kr/p/5ZFbHE
    34. 34. Red Green source: http://flic.kr/p/5tZaUe
    35. 35. That’s My Middle Path @rabble - cuboxsa.com source: http://flic.kr/p/6v82qN
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×