[Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development
•
2 likes•885 views
Rakuten Technology Conference 2014 "Beyond Agile Testing to Lean Development" James O. Coplien (Gertrud & Cope)
Recommended
More Related Content
What's hot
What's hot (20)
Similar to [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development
Similar to [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development (20)
More from Rakuten Group, Inc.
More from Rakuten Group, Inc. (20)
Recently uploaded
Recently uploaded (20)
[Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development
- 1. Beyond Agile Testing to Lean Development James O. Coplien Gertrud & Cope
- 3. Processes and Tools over Individuals and Interactions
- 4. Test Testing System Testing Exploratory Testing Usability Testing
- 6. Why doesn’t unit testing work? We can’t test things of business value Not knowing what to test, we test many things that don’t need testing Automated crap is still crap
- 7. Unit tests can’t test things of business value Can you demonstrate that Method X of Object Y will ever be invoked in practice? If a unit test fails, how does it affect ROI? What’s amazing is that we continue to test… Paths that will never be exercised Methods that are never used Naive method invocation sequences
- 8. Most unit tests are unnecessary We test scenarios that will never be exercised in the real world We test methods that will never be invoked We supply data that a real application would never have to process
- 9. Automated crap is still crap Design requires intelligence: intelligence cannot be automated Autonomation instead of automation Testing is about exploratory inference Get manual procedures working first and then automate the mundane parts
- 10. Automated testing is not testing
- 11. Nokia Test Score 2: Testing No dedicated QA - 0 Unit tested - 1 Feature tested - 5 Feature tested as soon as completed - 7 Software passes acceptance testing - 8 Software is deployed - 10
- 12. Weinberg’s Law of Decomposition
- 13. Weinberg’s Law of Decomposition Used Interface Used Interface Mu s t unit-te s t! Must unit-test! Must unit-test!
- 14. Weinberg’s Law of Decomposition Wasted testing effort Use Size Test Size
- 15. Part III: You Will Have Bugs! Scenarios we test in the lab Scenarios exhibiting faults Scenarios invoked in the field
- 16. Testing waste Removing tests leaves field faults Scenarios we test in the lab Scenarios exhibiting faults Scenarios invoked in the field undetected! Type 2 Muda Type 1 Muda Type 4 Muda Type 1: Defects Type 2: Stuff customers don’t ask for Type 4: Type Overprocessing 1 Muda (field)
- 17. Why do we unit test? Because we can It doesn’t require teamwork We don’t have to talk with the business We get to interact with tools instead of people Our decisions are emotional, not rational
- 18. Reduction of Waste Type 1: Better testing (e.g., going beyond contextual testing to exploratory & experience-based testing) (and better yet, better analysis and design, where the real payoffs are) Type 2: Don’t test dead code Type 4: Remove redundant and tautalogical tests, and tests that give no information Type 1 in the field: Ship your tests!
- 19. Don’t take the tests out when you ship! Remember, you will have bugs in the field! Deployment is your largest test driver Ship your tests as part of the code IN case of a failure, it saves you from Type 9 muda (unsafe execution)
- 20. Be Customer- Focused Assertions and Design by Contract!
- 21. How Adding ASSERTS reduces work 2500000 2000000 1500000 1000000 500000 0 15 bugs p=0.01 15 bugs p=0.02 10 bugs p=0.01 10 bugs p=0.02 5 bugs p=0.01 5 bugs p=0.02 0 1 2 5 10 20 50 100 number of ASSERTS Work, Tsinglechecks Overload Magazine, October 2014
- 22. Defensive Programming … is largely the incorporation of tests in the code Long known for high-reliability systems The value of asserting the obvious Design by Contract
- 23. Assertions make code readable! - (void) prepareForSegue: (UIStoryboardSegue*) segue sender: (id) sender { assert(segue != nil); if ([[segue identifier] isEqualToString: @"SegueAvatarPickerToPickOneLoginPlayer"]) { assert((UI_USER_INTERFACE_IDIOM() == UIUserInterfaceIdiomPhone)); // The field parentViewController_ (instance variable) is // maybe null if we are in iPhone mode and doing a real // segue, instead of a popover as we do on the iPad. So // get the right object from the segue record. KnowsyPlayerPickerCommonViewController *parentViewController = [segue destinationViewController]; [parentViewController setKnowsyData: knowsyData_]; } else { // iPad - no special processing - should be no popover assert(parentViewController_ == nil); } }
- 24. Part IV: Beyond agile testing, Lean testing When a test finds a bug, it’s about fixing the process — not the bug The long-term payoff is higher! No bug-tracking systems: we fix problems NOW. The tester should be the team’s conscience
- 25. Other Lean-isms Deming: Quality is inversely proportional to the number of testers GM kept around its cars for 1 - 6 weeks for test and repair: Toyota drove them off the end of the assembly line onto the car carriers.
- 26. Conclusion Be humble about what tests can accomplish It’s not about testing, but value Design tests to assess properties of business value Go beyond unit testing to defensive programming and design-by-contract As agile testers, focus on process first and product second
Editor's Notes
- BEYOND AGILE TESTING TO LEAN DEVELOPMENT - Jim Coplien • Testing is a business function that should be focused on risk • Lean defines testing as waste — why do it? • Automated tests should reflect requirements, but testers shouldn't test requirements • Testers are the team's conscience 50 Minutes
- Mike Cohn’s testing triangle
- http://qz.com/196200/toyota-is-becoming-more-efficient-by-replacing-robots-with-humans/ Max Niesen, April 7, 2014 James Bach: “Testing Requires Inference.” “Exploratory inference may sound like a strange idea. It means one idea leading to another in ways you can’t predict in advance” [and, therefore, can’t script].
- What if the methods we break them down with are private and used just to make the code more readable? I'm sure you could pronounce it faster... And I'm sure that it takes less time to read individual sentences than to read all of Moby Dick. But readability isn't the entire story, and comprehension is a big part of that. Appreciation comes in the whole. You don't appreciate the beauty of the Mona Lisa by studying only her nose: you take in the whole. It is more than the sum of its parts, just as understanding must be more than the sum of its parts. There are some strange loops going in the other direction, and this is where I've picked up a new tool in the past weeks: Weinberg's law of decomposition. Each part is much more than a decomposition of the whole. Because it is so much more, it is harder to understand as an entity than it is to understand when contextualized by its surroundings. So the phrase, "Frankly, dear, I don't give a damn" has some meaning in its own right but 100 times as much meaning when read in the context of "Gone With The Wind." Poetry is about about compression: about using cultural context, and the context of the rest of the work, to amplify meaning. Great software is group poetry writing. If you're not doing that, you're just a machine automating the generation of some object code in an assembly language called Java.
- What if the methods we break them down with are private and used just to make the code more readable? I'm sure you could pronounce it faster... And I'm sure that it takes less time to read individual sentences than to read all of Moby Dick. But readability isn't the entire story, and comprehension is a big part of that. Appreciation comes in the whole. You don't appreciate the beauty of the Mona Lisa by studying only her nose: you take in the whole. It is more than the sum of its parts, just as understanding must be more than the sum of its parts. There are some strange loops going in the other direction, and this is where I've picked up a new tool in the past weeks: Weinberg's law of decomposition. Each part is much more than a decomposition of the whole. Because it is so much more, it is harder to understand as an entity than it is to understand when contextualized by its surroundings. So the phrase, "Frankly, dear, I don't give a damn" has some meaning in its own right but 100 times as much meaning when read in the context of "Gone With The Wind." Poetry is about about compression: about using cultural context, and the context of the rest of the work, to amplify meaning. Great software is group poetry writing. If you're not doing that, you're just a machine automating the generation of some object code in an assembly language called Java.
- What if the methods we break them down with are private and used just to make the code more readable? I'm sure you could pronounce it faster... And I'm sure that it takes less time to read individual sentences than to read all of Moby Dick. But readability isn't the entire story, and comprehension is a big part of that. Appreciation comes in the whole. You don't appreciate the beauty of the Mona Lisa by studying only her nose: you take in the whole. It is more than the sum of its parts, just as understanding must be more than the sum of its parts. There are some strange loops going in the other direction, and this is where I've picked up a new tool in the past weeks: Weinberg's law of decomposition. Each part is much more than a decomposition of the whole. Because it is so much more, it is harder to understand as an entity than it is to understand when contextualized by its surroundings. So the phrase, "Frankly, dear, I don't give a damn" has some meaning in its own right but 100 times as much meaning when read in the context of "Gone With The Wind." Poetry is about about compression: about using cultural context, and the context of the rest of the work, to amplify meaning. Great software is group poetry writing. If you're not doing that, you're just a machine automating the generation of some object code in an assembly language called Java.
- Type 2: Weinberg’s Law of Decomposition Type 4: Keeping tests that never fail
- Last point: Most tests are irrational: i.e., you can never presume software correctness from the tests. But we do feel better by having checked, so we test anyhow.
- Be customer-focused
- Debug Complexity: How Assertions Affect Debugging Time. Sergey Ignatchenko and Dmytro Ivanchykhin, Overload, October 2014
- Conscience… Curmudgeonly and like Emperor’s New Clothes — not in the sense of standing in the way. That’s business’s job.
- No unit - subsystem - system - integration - acceptance test leveling.