Bug deBug Chennai 2012 Talk - Driving innovation using pattern based thinking by Vipul Kocher


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Root-cause analysis is one step in pattern recognition but not the only one RC – Communication/Training whereas pattern is things not getting done properly
  • Bug deBug Chennai 2012 Talk - Driving innovation using pattern based thinking by Vipul Kocher

    1. 1. 1
    2. 2. 2
    3. 3.  Patterns in real world Design Patterns and Test Patterns Usefulness of patterns in testing Patterns in testing  Data Patterns  Test Patterns  Defect Patterns  Test management Patterns  People management Patterns  Control Patterns What we do not know today What we might need to create – pattern language 3
    4. 4. 4
    5. 5. 5
    6. 6.  The sum of the numbers in any row is equal to 2n Source - http://ptri1.tripod.com/ 6
    7. 7.  How do you learn your native language? How do you learn your second language? 7
    8. 8.  The brain is a pattern-mad supposing machine. Given just a little stimuli, it divines the probable. …it recognizes familiar patterns and acts with conviction.  As much of a confection as our mental maps are, they (Patterns) allow us to speculate, rehearse and make plans.*http://www.nytimes.com/2004/06/15/science/essay-i-sing-the-body-s-pattern-recognition-machine.html 8
    9. 9.  *A form or model proposed for imitation *Something designed or used as a model for making things Design pattern - general reusable solution to a commonly occurring design problemFrom MW Dictionary* 9
    10. 10.  I predict that one of the future breakthrough in testing will come through pattern recognition Example  Patterns of defects arising out of patterns of errors in code/design/requirements  Patterns of project failures arising out of data collected during project execution 10
    11. 11.  Patterns exists everywhere - in defects, code, design, processes Pattern identification helps predict the next set of events in the chain enabling you to do risk mitigation Patterns enable you to distinguish the woods from the trees 11
    12. 12.  Data Patterns Test Patterns Defect Patterns Test management Patterns People management Patterns Control Patterns 12
    13. 13.  Ross Collard listed a number of data patterns for functional and performance testing -  Routine Live Data  Baseline  Batch Volume or Parallel  Benchmark  Pristine  Truncated  Minimally Redundant etc. 13
    14. 14.  People often work on one side of boundary while testing Make them work on both sides/all sides of boundary Example –  Triangle Problem negative test cases - no value, 1 value, 2 values, all 3 values  what about the other side of the boundary - more number of values? 14
    15. 15.  Patterns can help uncover new test design techniques OR new ways to look at existing test design techniques  Apply test design techniques to create tests  Monitor the bugs arising OUT(SIDE) of tests  Establish the bugs that could have been found using existing techniques  Defects not “findable” using techniques should be analyzed for patterns and a technique using which these could be found 15
    16. 16. Some patterns in type of bugs not found usingtests created using techniques Interaction of application with the operating system  Reason – Design specifications hardly go to that level of detail and system specifications never Defects related to interdependency of features and code structures causing one portion of code to be bound to another piece of code  Reason – Code dependencies are either not called out explicitly, born in either assumptions or ignorance (most of the time) 16
    17. 17. Associations Contains User Inputs A model System Inputs Output Data Entity under test based Input Data Output Actions technique Input Actors On other Resembles elements/ entities Environment (Ext. State) state State (Internal State) diagrams On OS - Disk - File - Process - Screen IO - System Messages Transient /Stable State Parameters & their attributes Parameters State of Parameters 17
    18. 18.  How many bugs do you find every month? How many of them are type – UI, Feature, cross- feature, database, security… Can you sub-classify them? Do you know what are other testers finding? Do you see any patterns in  What type of bugs you find  What types of bugs you don’t find? 18
    19. 19.  If you find predictable bugs (based on your past experience and pattern recognition) can there be pattern recognition  for code?  Design?  Requirements? 19
    20. 20.  Are there testers who do good bug isolation? Are there those who do not? How do they fail? What things they do not do? What patterns exist? If I know those patterns can I improve their skills? 20
    21. 21.  Example  Project Failure pattern – Communication/delivery/no-interested-client- stakeholder  Automation Outsourcing Failure – Lack of ~effort appreciation/well-written tests/domain knowledge 21
    22. 22. Date Action Issue 24-Jan-01Clear work division in overlapping areas Performance test cases being written by Joe and Jane. and ensuring communication between Test were duplicated. Some missing test cases the overlapping/concerned people. because every body was not aware hence did not review/contribute to the tests.28-Jan-01Knowledge of the system (WES, WAS Some members of the team did not know what they performance, tuning, deployment) had to do, others did not have required technical should be available to the team. knowledge14-Feb-01Communication of the documents, The release meeting with developers happened Agenda, Minutes to the testing team without a copy of test plan (esp. dates) being should be formal, via mail. available. 9-Mar-01User Guides esp. “how to get started” Swami sent a mail asking how to use the ABC app. We should always accompany a build had to write some steps and send him the document.  What pattern do you see here?  This pattern emerges from long term view - The four entries here are from a list of 133 entries between Jan and April 2001.  How is pattern recognition different from Root cause analysis? 22
    23. 23.  Example – Patterns while Interviewing  assessing people, forming impressions, validating, creating and refining patterns  Remember - you have not discovered the last pattern hence you may be wrong  Keep updating your impression of patterns and be ready for the worst 23
    24. 24.  Example – “Only a Guest” pattern  when people (are about to) leave the organization without information to the management there are some behavioral patterns that manifest themselves.  Indifference, extra-hard work, sudden forgiveness and not picking up of battles, unplanned absenteeism, putting all paper-work in place etc. 24
    25. 25.  Patterns for  Predicting  Inventing  Improving  Preventing  … 25
    26. 26.  Unified Modeling Language – Unified Testing Language No guarantee of success and no predictions of failure 26
    27. 27.  Applying patterns requires  Ignoring the small and looking at the large  Large amount of data (usually)  Reduction in the number of variables  Elimination of outliers  Pattern of outliers  Variations and taking cognizance of those  Variations and eliminating them 27
    28. 28. http://www.rense.com/general86/stun.htm 28
    29. 29. http://www.rense.com/general86/stun.htm 29
    30. 30.  “Homo sapiens is about pattern recognition, he says. Both a gift and a trap.” - William Gibson – Pattern Recognition 30
    31. 31.  http://www.exampler.com/testing-com/test-patterns/ http://c2.com/cgi-bin/wiki?TestingPatterns http://www.testingreflections.com/node/view/9 31
    32. 32. Thank you everybody <<meta-pattern – help>> for images, text and ideas 32