Michael mahlberg exploratory-testing-the_missing_half_of_bdd
Report
Share
•4 likes•2,018 views
![So what’s the problem?
■ Test
[...]
Etymology
From Old French test (“an earthen vessel,
especially a pot in which metals were tried”)
Source: http://en.wiktionary.org/wiki/test
Tuesday, June 5, 2012](https://image.slidesharecdn.com/michaelmahlberg-exploratorytesting-themissinghalfofbdd-120605030441-phpapp01/75/michael-mahlberg-exploratorytestingthemissinghalfofbdd-29-2048.jpg?cb=1669270025)
![Acceptance Criteria
• To accomodate pre-conditions
Acceptance Criteria can be expressed as
• Given [Precondition]
When [Actor and Action]
Then [Observable Result]
• Naresh Jain Slide 29
Tuesday, June 5, 2012](https://image.slidesharecdn.com/michaelmahlberg-exploratorytesting-themissinghalfofbdd-120605030441-phpapp01/75/michael-mahlberg-exploratorytestingthemissinghalfofbdd-49-2048.jpg?cb=1669270025)
Michael mahlberg exploratory-testing-the_missing_half_of_bdd
•4 likes•2,018 views
Report
Share
Download to read offline
"We should just call it testing - when it's not exploratory testing it's not real testing anyway" -Twitter, Summer 2011 Lately many professional testers have started to make a clear distinctions between thing that we call testing (like TDD and BDD) and what they consider testing - referring to TDD and BDD mostly as checking. And actually I – and I would think many of you as well – have seen projects with a test coverage of 80% and more that still fail to meet the clients' needs. Even though they meet the specifications perfectly. This points to some value that could be added to techniques like BDD and TDD by embracing the ideas from people like James Marcus Bach, Paul Carvalho and Michael Bolton. After giving an overview of current trends in the testing community like ET (exploratory testing) and ATDD (Acceptance Test Driven Development) this session will try to do exactly that: discuss the - often missing - intersection between BDD and exploratory testing and suggest ways to fill it.
More Related Content
Viewers also liked
Viewers also liked(6)
Similar to Michael mahlberg exploratory-testing-the_missing_half_of_bdd
Similar to Michael mahlberg exploratory-testing-the_missing_half_of_bdd(20)
More from Michael Mahlberg
More from Michael Mahlberg(20)
![The Product Owner's Survival Kit - ein Überblick [DE]](https://cdn.slidesharecdn.com/ss_thumbnails/oop2017ndo4productownersurvivalkitmahlbergslideshare-170209124144-thumbnail.jpg?width=768&fit=bounds)
Recently uploaded
Recently uploaded(20)
Michael mahlberg exploratory-testing-the_missing_half_of_bdd
- 1. Michael Mahlberg, 05.06.2012 Exploratory Testing – The missing half of BDD? Tuesday, June 5, 2012
- 2. Testing? Tuesday, June 5, 2012
- 3. AGAIN? Tuesday, June 5, 2012
- 4. It’s not about Rails http://www.flickr.com/photos/chris_gin/2490980153/ Tuesday, June 5, 2012
- 5. Several Stories Tuesday, June 5, 2012
- 6. BDD Specification Design by TDD by Example Contract Defensive Programming Software- Development Tuesday, June 5, 2012
- 7. Exploratory Testing Tuesday, June 5, 2012
- 8. Tuesday, June 5, 2012
- 9. Tuesday, June 5, 2012
- 10. Tuesday, June 5, 2012
- 11. Tuesday, June 5, 2012
- 12. Paul Carvalho @can_test Tuesday, June 5, 2012
- 13. Michael Bolton No, Not this one Tuesday, June 5, 2012
- 14. Michael Bolton @michaelbolton Tuesday, June 5, 2012
- 15. Markus Gärtner @mgaertne Tuesday, June 5, 2012
- 16. Players in the Game • James Bach • Michael Bolten (not the singer) • Cem Kaner • Paul Carvalho Tuesday, June 5, 2012
- 17. Ressources Human Resource Paul Carvalho http://swtester.blogspot.de/ Michael Bolton http://www.developsense.com/blog/ James Bach http://www.satisfice.com/blog/ Elisabeth Hendrickson http://testobsessed.com/ Markus Gärtner http://www.shino.de/blog/ Tuesday, June 5, 2012
- 18. Ressources by James Bach Topic Ressource SBTM Session Based http://satisfice.com/articles/sbtm.pdf Test Management http://people.eecs.ku.edu/~saiedian/ Exploratory Testing Teaching/Fa07/814/Resources/ exploratory-testing.pdf Hands-on Exploratory http://www.satisfice.com/tools/ procedure.pdf Testing Rapid Software Testing http://www.satisfice.com/rst.pdf Tuesday, June 5, 2012
- 19. The basic misunderstanding • <X> is necessary but not sufficient Tuesday, June 5, 2012
- 20. An example from agile software development Tuesday, June 5, 2012
- 21. An example from agile software development Working Software over Comprehensive Documentation Tuesday, June 5, 2012
- 22. Derived Assumption: Working software is sufficient Tuesday, June 5, 2012
- 23. Reality: Working software is necessary, but not sufficient! Tuesday, June 5, 2012
- 24. Tuesday, June 5, 2012
- 25. Tuesday, June 5, 2012
- 26. Tuesday, June 5, 2012
- 27. So what’s the problem? ■ Software testing, the process of verifying that a software program works as expected Source: http://en.wikipedia.org/wiki/Test Tuesday, June 5, 2012
- 28. So what’s the problem? ... but ... Tuesday, June 5, 2012
- 29. So what’s the problem? ■ Test [...] Etymology From Old French test (“an earthen vessel, especially a pot in which metals were tried”) Source: http://en.wiktionary.org/wiki/test Tuesday, June 5, 2012
- 30. What do test pilots do? Tuesday, June 5, 2012
- 31. What do test pilots do? Tuesday, June 5, 2012
- 32. What do test pilots do? Tuesday, June 5, 2012
- 33. What do test pilots do? Tuesday, June 5, 2012
- 34. What do test pilots do? Tuesday, June 5, 2012
- 35. Pushing the envelope Tuesday, June 5, 2012
- 36. Checking things Tuesday, June 5, 2012
- 37. Scripted testing With a script, you miss the same things every time. Exploratory Testing @ QUEST 2008" Copyright © 2008 Cem Kaner Tuesday, June 5, 2012
- 38. Defensive programming - Check all input parameters - Ascertain availability of resources etc. Defensive programming defends against the currently impossible (c2 Wiki) Tuesday, June 5, 2012
- 39. Design by Contract Betrand Meyer Since 1986 Programming language: Eiffel Tuesday, June 5, 2012
- 40. Design by Contract Betrand Meyer Since 1986 Programming language: Eiffel Preconditions Postconditions Invariants Tuesday, June 5, 2012
- 41. TDD • Unit Tests • Black-Box Tests • White-Box Tests • RSpec etc. Tuesday, June 5, 2012
- 42. BDD • Acceptance Tests • (Mostly) Black-Box Tests • Based on Scenarios Tuesday, June 5, 2012
- 43. ATDD • Bigger gap between test creation and execution • Exclusively Black Box Testing Tuesday, June 5, 2012
- 44. An adage from agile software development Tuesday, June 5, 2012
- 45. An adage from agile software development INVEST in stories and come up with SMART tasks Tuesday, June 5, 2012
- 46. INVEST Independent Negotiable Valuable Estimatable Small Testable Tuesday, June 5, 2012
- 47. SMART Specific Mesuarable Achievable Realistic Tangible Tuesday, June 5, 2012
- 48. Independent Specific Negotiable Mesuarable Valuable Achievable Estimatable Realistic Small Tangible Testable Tuesday, June 5, 2012
- 49. Acceptance Criteria • To accomodate pre-conditions Acceptance Criteria can be expressed as • Given [Precondition] When [Actor and Action] Then [Observable Result] • Naresh Jain Slide 29 Tuesday, June 5, 2012
- 50. Specification by Example ... It is the result of a research on how teams all over the world specify, develop, test and deliver the right software, without defects, in very short iterative delivery cycles. ... Tuesday, June 5, 2012
- 51. What’s left for the Testers? Tuesday, June 5, 2012
- 52. DbC vs. TDD Design by Contract implies automated defensive programming pre- and postconditions and Test Driven Design is invariants are checked at mostly applied during design- runtime time. Expensive checks are Tests are executed whenever inhibitive appropriate - but usually not in the production environment Expensive checks are possible but unwarranted Tuesday, June 5, 2012
- 53. Checking vs. Testing Checking is something that we do with the motivation of confirming existing beliefs Testing is something that we do with the motivation of finding new information. http://www.developsense.com/blog/2009/08/testing-vs-checking/ Tuesday, June 5, 2012
- 54. Test Driven Development is an oxymoron Check Driven Development? Tuesday, June 5, 2012
- 55. The Test-Cycles Tuesday, June 5, 2012
- 56. Red-Green-Refactor Failing Test Simplify Coding Remove Redundancy Tuesday, June 5, 2012
- 57. Exploratory Testing From: Paul Carvalho http://swtester.blogspot.de/2012/05/what-is-exploratory-testing.html Tuesday, June 5, 2012
- 58. So what is Exploratory Testing? "Exploratory Testing is simultaneously learning about the system while designing and executing tests, using feedback from the last test to inform the next." Elisabeth Hendrickson, @testobsessed Quoted by Paul Carvalho Tuesday, June 5, 2012
- 59. Cone of uncertainty Agile and Iterative Development: A Manager's Guide, Craig Larman 2003 Tuesday, June 5, 2012
- 60. Cone of uncertainty Tuesday, June 5, 2012
- 61. Cone of test-cases http://swtester.blogspot.de/2012/05/what-is-exploratory-testing.html Tuesday, June 5, 2012
- 62. Agile testing quadrants http://lisacrispin.com/wordpress/2011/11/08/using-the-agile-testing-quadrants/ Tuesday, June 5, 2012
- 63. The Prime Directive Be Thoughtful and Methodical Throughout the test procedure, as you complete the tasks, you have lots of freedom about how you do the work. But you must work methodically, and follow the procedure. In the course of creating the result for each task, you’ll find that you have to make a lot of guesses, and some of them will be wrong. But you must think. If you find yourself making wild and uneducated guesses about how the product works, areas of instability, or anything else, stop and talk to the Test Manager. James Bach, 08/26/99, http://www.satisfice.com/tools/procedure.pdf Tuesday, June 5, 2012
- 64. Scientific Method Formulate a question Hypothesis Prediction Test Analysis Tuesday, June 5, 2012
- 65. Scientific Method Formulate a question Hypothesis Prediction Test Analysis Tuesday, June 5, 2012
- 66. Exploratory Testing Complete these five tasks: ✓ Identify the purpose of the product. ✓ Identify functions. ✓ Identify areas of potential instability. ✓ Test each function and record problems. ✓ Design and record a consistency verification test. Source: http://www.satisfice.com/tools/procedure.pdf pg. 11 to 22 Tuesday, June 5, 2012
- 67. Identify the purpose of the product • With a tangible result (i.e. Document) • Value for others - verifiable etc. • Value for the tester - builds understanding Tuesday, June 5, 2012
- 68. Identify functions • By actually using (skimming) the product • Explorative (not based on documentation) • Tangible output (Function outline) Tuesday, June 5, 2012
- 69. From James Bach: http://www.satisfice.com/tools/procedure.pdf Tuesday, June 5, 2012
- 70. Identify areas of potential instability • Results: • Documented areas (before testing) • Documented data for testing • Documented Strategies Tuesday, June 5, 2012
- 71. Test each function and record problems • Actually this means: Try to do things the development teams did not think of. • Remember: A Problem is just the difference between the expected behavior and the observed behavior - whether it is cause for action has to be decided by the customer Tuesday, June 5, 2012
- 72. Design and record a consistency verification test • Try to extrapolate if the Software would run in (slightly) modified environments. (The original 5 steps are from 1999 (gasp) and relate to specific windows versions - the basic requirements still hold through) Tuesday, June 5, 2012
- 73. Tool Support Area Tool URL http:// Recording spector www.spectorsoft.com/ http:// Recording BBTestAssistant www.bbsoftware.co.uk/ products.aspx http:// Recording Wink www.debugmode.com/ wink/ http://www.sirius- All in One TestExplorer sqa.com/ http://www.satisfice.com/blog/archives/77 Tuesday, June 5, 2012
- 74. Session Based Test Management • Original description of ET : 1999 • Today : 2012 • SBTM: A way to organize exploratory testing • Working in 45 to 90 min Session based on Charters • See: http://satisfice.com/articles/sbtm.pdf Tuesday, June 5, 2012
- 75. What are Charters Counterpart to the use case / user story Some of the same attributes - Roles, activities, actors, systems Examples: - “Analyze <Product>’s View menu functionality and report on areas of potential risk.” Tuesday, June 5, 2012
- 76. Hierarchy Tuesday, June 5, 2012
- 77. Hierarchy Ad-Hoc Testing Tuesday, June 5, 2012
- 78. Hierarchy Exploratory Testing Ad-Hoc Testing Tuesday, June 5, 2012
- 79. Hierarchy Session Based Testing Exploratory Testing Ad-Hoc Testing Tuesday, June 5, 2012
- 80. Hierarchy Session Based Test Management Session Based Testing Exploratory Testing Ad-Hoc Testing Tuesday, June 5, 2012
- 81. ET-Spectrum Tuesday, June 5, 2012
- 82. ET-Spectrum Exploratory Testing according to: http://www.satisfice.com/rst.pdf Tuesday, June 5, 2012
- 83. Can we prove it? Testing shows the presence, not the absence of bugs J.N. Buxton and B. Randell, eds, Software Engineering Techniques, April 1970, p. 16. Report on a conference sponsored by the NATO Science Committee, Rome, Italy, 27–31 October 1969. via http://en.wikiquote.org/wiki/Edsger_W._Dijkstra Program testing can be used to show the presence of bugs, but never to show their absence! Source: Notes On Structured Programming, 1972, at the end of section 3, On The Edsger Wybe Dijkstra Reliability of Mechanisms. EWD249 (1970). Tuesday, June 5, 2012
- 84. TDD • Technique to drive the design of your code y, t! r n sa cie s fi ce uf e s N ot t n Bu Tuesday, June 5, 2012
- 85. BDD • Technique to capture requirements and drive desired functionality y, t! r n sa cie s fi ce uf e s N ot t n Bu Tuesday, June 5, 2012
- 86. ET • Technique to drive the discovery of unwanted, unexpected and unpredicted behaviour y, t! r n sa cie s fi ce uf e s N ot t n Bu Tuesday, June 5, 2012
- 87. About me “Consulting on software development processes and software architecture since the last millennium” - @MMahlberg Google me: “Michael Mahlberg” Tuesday, June 5, 2012
- 88. Discussion? Tuesday, June 5, 2012
- 89. Flight-Envelope Photo Credits http://en.wikipedia.org/wiki/File:PerformanceEnvelope.gif Testpilots: http://www.flickr.com/photos/x-ray_delta_one/4106250812/in/photostream/ Spreading junction http://www.flickr.com/photos/elsie/4164899/in/photostream/ Tokio Tower http://www.flickr.com/photos/wilhelmja/330965980/in/photostream/ Eiffel Tower http://www.flickr.com/photos/gcattiaux/2565101112/in/photostream/ Spectrum http://www.flickr.com/photos/eiriknewth/132859486/in/photostream/ Armor http://www.flickr.com/photos/nicmcphee/1563082378/in/photostream/ Aerobatics http://www.flickr.com/photos/ssandars/6827302/in/photostream/ Airbus http://www.flickr.com/photos/mwanasimba/4097232435/ Tuesday, June 5, 2012