Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Beyond Agile Testing to Lean Development — Rakuten Technology Conference

4,467 views

Published on

Talk at Rakuten Technology Conference, Shinagawa Seaside, Tokyo, Japan, 25 October, 2014

Published in: Software
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Beyond Agile Testing to Lean Development — Rakuten Technology Conference

  1. 1. Beyond Agile Testing to Lean Development James O. Coplien Gertrud & Cope
  2. 2. Processes and Tools over Individuals and Interactions
  3. 3. Test Testing System Testing Exploratory Testing Usability Testing
  4. 4. Why doesn’t unit testing work? We can’t test things of business value Not knowing what to test, we test many things that don’t need testing Automated crap is still crap
  5. 5. Unit tests can’t test things of business value Can you demonstrate that Method X of Object Y will ever be invoked in practice? If a unit test fails, how does it affect ROI? What’s amazing is that we continue to test… Paths that will never be exercised Methods that are never used Naive method invocation sequences
  6. 6. Most unit tests are unnecessary We test scenarios that will never be exercised in the real world We test methods that will never be invoked We supply data that a real application would never have to process
  7. 7. Automated crap is still crap Design requires intelligence: intelligence cannot be automated Autonomation instead of automation Testing is about exploratory inference Get manual procedures working first and then automate the mundane parts
  8. 8. Automated testing is not testing
  9. 9. Nokia Test Score 2: Testing No dedicated QA - 0 Unit tested - 1 Feature tested - 5 Feature tested as soon as completed - 7 Software passes acceptance testing - 8 Software is deployed - 10
  10. 10. Weinberg’s Law of Decomposition
  11. 11. Weinberg’s Law of Decomposition
  12. 12. Weinberg’s Law of Decomposition
  13. 13. Weinberg’s Law of Decomposition Used Interface Used Interface
  14. 14. Weinberg’s Law of Decomposition Used Interface Used Interface Mu s t unit-te s t! Must unit-test! Must unit-test!
  15. 15. Weinberg’s Law of Decomposition
  16. 16. Weinberg’s Law of Decomposition Wasted testing effort
  17. 17. Weinberg’s Law of Decomposition Wasted testing effort Test Size
  18. 18. Weinberg’s Law of Decomposition Wasted testing effort Use Size Test Size
  19. 19. Part III: You Will Have Bugs! Scenarios we test in the lab Scenarios exhibiting faults Scenarios invoked in the field
  20. 20. Testing waste Scenarios we test in the lab Scenarios exhibiting faults Scenarios invoked in the field
  21. 21. Testing waste Scenarios we test in the lab Scenarios exhibiting faults Scenarios invoked in the field Type 1 Muda Type 1: Defects
  22. 22. Testing waste Scenarios we test in the lab Scenarios exhibiting faults Scenarios invoked in the field Type 2 Muda Type 1 Muda Type 1: Defects Type 2: Stuff customers don’t ask for
  23. 23. Testing waste Scenarios we test in the lab Scenarios exhibiting faults Scenarios invoked in the field Type 2 Muda Type 1 Muda Type 4 Muda Type 1: Defects Type 2: Stuff customers don’t ask for Type 4: Overprocessing
  24. 24. Testing waste Scenarios exhibiting faults Scenarios invoked in the field Type 1 Muda Type 4 Muda Type 1: Defects Type 2: Stuff customers don’t ask for Type 4: Type Overprocessing 1 Muda (field)
  25. 25. Testing waste Removing tests leaves field faults Scenarios exhibiting faults Scenarios invoked in the field undetected! Type 1 Muda Type 4 Muda Type 1: Defects Type 2: Stuff customers don’t ask for Type 4: Type Overprocessing 1 Muda (field)
  26. 26. Why do we unit test? Because we can It doesn’t require teamwork We don’t have to talk with the business We get to interact with tools instead of people Our decisions are emotional, not rational
  27. 27. Reduction of Waste Type 1: Better testing (e.g., going beyond contextual testing to exploratory & experience-based testing) (and better yet, better analysis and design, where the real payoffs are) Type 2: Don’t test dead code Type 4: Remove redundant and tautalogical tests, and tests that give no information Type 1 in the field: Ship your tests!
  28. 28. Don’t take the tests out when you ship! Remember, you will have bugs in the field! Deployment is your largest test driver Ship your tests as part of the code IN case of a failure, it saves you from Type 9 muda (unsafe execution)
  29. 29. Be Customer- Focused Assertions and Design by Contract!
  30. 30. How Adding ASSERTS reduces work 2500000 2000000 1500000 1000000 500000 0 15 bugs p=0.01 15 bugs p=0.02 10 bugs p=0.01 10 bugs p=0.02 5 bugs p=0.01 5 bugs p=0.02 0 1 2 5 10 20 50 100 number of ASSERTS Work, Tsinglechecks Overload Magazine, October 2014
  31. 31. Defensive Programming … is largely the incorporation of tests in the code Long known for high-reliability systems The value of asserting the obvious Design by Contract
  32. 32. Assertions make code readable! - (void) prepareForSegue: (UIStoryboardSegue*) segue sender: (id) sender { assert(segue != nil); if ([[segue identifier] isEqualToString: @"SegueAvatarPickerToPickOneLoginPlayer"]) { assert((UI_USER_INTERFACE_IDIOM() == UIUserInterfaceIdiomPhone)); // The field parentViewController_ (instance variable) is // maybe null if we are in iPhone mode and doing a real // segue, instead of a popover as we do on the iPad. So // get the right object from the segue record. KnowsyPlayerPickerCommonViewController *parentViewController = [segue destinationViewController]; [parentViewController setKnowsyData: knowsyData_]; } else { // iPad - no special processing - should be no popover assert(parentViewController_ == nil); } }
  33. 33. Part IV: Beyond agile testing, Lean testing When a test finds a bug, it’s about fixing the process — not the bug The long-term payoff is higher! No bug-tracking systems: we fix problems NOW. The tester should be the team’s conscience
  34. 34. Other Lean-isms Deming: Quality is inversely proportional to the number of testers GM kept around its cars for 1 - 6 weeks for test and repair: Toyota drove them off the end of the assembly line onto the car carriers.
  35. 35. Conclusion Be humble about what tests can accomplish It’s not about testing, but value Design tests to assess properties of business value Go beyond unit testing to defensive programming and design-by-contract As agile testers, focus on process first and product second

×