What is good evidence - Let's Test 2013

2,333 views

Published on

By marshaling credible and persuasive evidence, influential testers answer three basic questions: How good is the product? What testing did you do? Why is the testing any good? Feeble evidence and the behaviors associated with it are common testing maladies and threats to your ability to tell a compelling story. What are the qualities of strong evidence, and the anti-patterns and risks of weak evidence? Griffin Jones presents the different qualities and types of evidence. We review threats to the credibility and persuasiveness of your work, such as: The danger of false-negative results reported as “pass – as expected”, “Lullaby Language” as an anti-pattern of Lean’s Genchi Genbutsu (go-and-see), the danger of obsessing on efficiency and how it biases the observer. Finally, we show how over-scripted procedures can superficially conceal fatal evidentiary flaws. Be intentional and critical about the type and quality of your evidence. Leave with the skills to recognize and evaluate the dangers and risks, strengths and weaknesses of the evidence you use for your testing story.

Published in: Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,333
On SlideShare
0
From Embeds
0
Number of Embeds
1,337
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

What is good evidence - Let's Test 2013

  1. 1. What is GoodEvidence?May 2013 © 2013 Congruent Compliance LLC 1
  2. 2. Griffin JonesConsultantAgile / Testing / RegulatoryMay 2013 © 2013 Congruent Compliance LLC 2This Presentation Covers …• Types of Evidence• Qualities of Evidence• Traps and Anti-PatternsWhy I Care, and Why You Should• Civil and Criminal Courts• Credible and Persuasive Story-of-Testing
  3. 3. May 2013 © 2013 Congruent Compliance LLC 3TYPES OF EVIDENCESources• FDA’s 21 CFR Part 11 – Electronic Records• FDA’s Good Manufacturing Practices• (Adjust for your context)Records versus Testimony• Speaks for Itself• Cross-Examination
  4. 4. May 2013 © 2013 Congruent Compliance LLC 4QUALITIES …Clear• To Whom?Objective• To What Degree?
  5. 5. May 2013 © 2013 Congruent Compliance LLC 5… QUALITIES …Authentic• Originals or Copies?• Resistant to Falsification?Integrity• Protected from Damage / Deterioration?• Can not be Readily Repudiated?
  6. 6. May 2013 © 2013 Congruent Compliance LLC 6Readable• Human and Machine?Changes• Do Not Obscure Previous Information?• Single line-out, initial, date, reason?… QUALITIES …
  7. 7. May 2013 © 2013 Congruent Compliance LLC 7… QUALITIES …Attributable• Do we know the Specific Person?Contemporary• Created At-The-Moment?• Are Not Recollections?
  8. 8. May 2013 © 2013 Congruent Compliance LLC 8… QUALITIES …Audit Trail• Is there a Chain-of-Custody?• Is there a Revision History?Retrievable• Is this stored where I can Access it?
  9. 9. May 2013 © 2013 Congruent Compliance LLC 9… QUALITIES3rd Party Review / Reconstruction• Minimal Outside Assistance?• Reach the Same Results & Conclusions?
  10. 10. May 2013 © 2013 Congruent Compliance LLC 10TRAPS, THREATS ANDANTI-PATTERNSDeficient Qualities• Leads to FeeblenessFeebleness• Leads to Unpersuasive Stories• Maybe a Systemic Risk smell?
  11. 11. May 2013 © 2013 Congruent Compliance LLC 11“PASS-AS EXPECTED”CHECKBOXESFear potential False-Positives• The danger of the Green-Bar• Falsification versus ConfirmationWhat you Intended to do• Not what you Actually Did
  12. 12. May 2013 © 2013 Congruent Compliance LLC 12Shuts-down Conversation• Discourages feedback• Promotes Shallow Agreement• Lulls your mind into a False Sense ofSecurityStyled Ceremony and Ritual• To provoke a behavior, not inform adecisionLULLABY LANGUAGE
  13. 13. May 2013 © 2013 Congruent Compliance LLC 13… VERSUS LEAN’SGENCHI GENBUTU“… It’s Lies! All Lies!”• Metrics and reports are just maps …• Get out of your office• Go observe where the work happens• Take your own measurements
  14. 14. May 2013 © 2013 Congruent Compliance LLC 14EFFICIENCY AND TIME PRESSUREBiasing and Priming the Subject• Over-Scripting over-focuses attention• The “Moon-Walking Bear” problem• Target Lock-OnUnringing the Bell• The button story and pixel-drop out
  15. 15. May 2013 © 2013 Congruent Compliance LLC 15Recognize the Different Qualities• Clear, Objective, Authentic, Integrity,Readable, Changes, Attributable,Contemporary, Audit Trail, Retrievable,ReconstructableHow is your evidence deficient?SUMMARY … A
  16. 16. May 2013 © 2013 Congruent Compliance LLC 16Recognize the Traps• Pass-As-Expected• Lullaby Language• Moon-Walking BearGo Observe the WorkSUMMARY… B
  17. 17. May 2013 © 2013 Congruent Compliance LLC 17QUESTIONS AND STORIES
  18. 18. Thank you for attending this session.Please fill out the evaluation form.Griffin JonesGriffin.Jones@CongruentCompliance.comMay 2013 © 2013 Congruent Compliance LLC 18

×