Exploratory test - Making test soup on a nail
Upcoming SlideShare
Loading in...5

Like this? Share it with your network

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 11

https://twitter.com 11

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Making Test-Soup on a Nail Getting From Nothing to Something Gitte Ottosen Gitte.ottosen@sogeti.dk 1
  • 2. Sogeti Profile •  Sogeti is a leading provider of structured testing solutions •  Part of the Sogeti Group, which brings together more than 20,000 professionals in 15 countries and is present in over 100 locations in Europe, USA and India •  Creators of the globally recognized methodologies TMap NEXT ® and TPI NEXT® •  Well established cooperation with HP, IBM and Microsoft © Sogeti
  • 3. And yes… the HORROR •  I AM CERTIFIED!!! –  ISEB Practitioner –  TMap test engineer –  Certified Scrum master –  Certified Agile Test (CAT) Trainer –  Certificate TPI NEXT Foundation –  …And I am probably not done yet •  But I have also –  Been in the Royal Danish Air force for 9 years –  Tested complex mission critical systems the last 18 years –  Read a lot of books… about a lot of “stuff” J –  Been at conferences – to learn and share –  Been part of networks – to learn and share –  Been a scout for 10 years – learning by doing –  Been doing a lot of crafts and creative stuff that stimulates my brain –  Been looking at the fish in my aquariums … © Sogeti 3
  • 4. Why This Presentation Anyway? •  Reactions from last Agile testing days –  It sounds simple but how do we do in practice? –  Like your presentation, great with a practical description of exploratory –  Have read and heard a lot about exploratory but how do I get started? •  Less theory more practice –  I want to show you a practical example of how I have used exploratory in a certain context in order to inspire you to get started. –  I haven’t got the ”best practice” – but I have a good practice from the context I was in at that point in time. © Sogeti 4
  • 5. Let’s get started 5
  • 6. Definition of Exploratory Test ”An interactive process of simultaneous learning, test design, and test execution.” Exploratory testing is not against the idea of scripting. In some contexts, you will achieve your testing mission better through a more scripted approach; in other contexts, your mission will benefit more from the ability to create and improve tests as you execute them. I find that most situations benefit from a mix of scripted and exploratory approaches. James Bach Exploratory Testing Explained © Sogeti
  • 7. The Case •  The Project –  Small agile project – (6 including me) –  Distributed team – developers in Kuala Lumpur –  Extension to system in production •  The System –  Web based application –  Complex capabilities for creating customized dialogues and overviews –  Complex definitions of calculations from the GUI –  Connection to Data Warehouse © Sogeti 7
  • 8. The Challenge •  Organization –  Requirements from TMO –  Product owner vs Project manager •  Test basis –  No system specifications for existing system –  Only limited user documentation (two slides) –  A lot of user stories but no overview and limited business perspective –  No testware for the part of the system already in production •  The ”human challenge” –  –  –  –  © Sogeti Product owner only part time available Could not see the value in a tester Don’t waste time on documenting – that’s overhead Only limited access to people in the project 8
  • 9. Challenge Accepted Then what? 9
  • 10. Identifying the Stakeholders Project manager Test management office (TMO) Users Super users © Sogeti Product owner Lead developer Developers 10
  • 11. Requirement from TMO •  Define and document test strategy •  Continuous test during development •  Creation of regression test •  Document test in Quality Center © Sogeti 11
  • 12. Getting the Initial Overview Product owner presenting the system •  Existing functionality – an overall presentation •  The new area – a more detailed presentation. •  Only verbal – no documentation © Sogeti 12
  • 13. From the Conversation - Initial Mind Map © Sogeti 13
  • 14. Getting an Overview Aggregating stories to features © Sogeti 14
  • 15. Test Strategy in a page •  Objective •  Scope –  Product risk –  Project risk •  Test ”levels” –  –  –  –  –  Unit test Unit integration test Test on story level Test in the sprint Regressions test •  Tools and techniques –  Test design techniques –  Test documentation – QC © Sogeti 15
  • 16. Getting the Overview •  Initial Exploration –  Started with the operational (stable) part of the application –  Getting acquainted with the concepts –  Identifying: •  General concepts •  General problems Taking lots of notes © Sogeti 16
  • 17. Result after First Exploration © Sogeti 17
  • 18. Getting Structured •  A lot more exploration •  Identifying and using relevant test design techniques •  Identifying and using relevant heuristics ation ssific Cla Trees EP Pair-wise © Sogeti Pro ces flow Tes t 18
  • 19. Where to Apply? © Sogeti 19
  • 20. EP/BVA and Classification Trees Field1 Field 2 Field 3 Field 4 Field5 © Sogeti 20
  • 21. Pair-wise © Sogeti 21
  • 22. Process Cycle Test •  •  •  •  •  Meeting the user Drawing and discussing Exploration in the system Amending the drawing Presenting to PO Yes Yes No No © Sogeti 22
  • 23. But Also Being Creative… •  I wonder whether they looked at field sizes in the database when they specified field lengths? •  Have they remembered the support for Danish letters ÆØÅ? •  You can have so many types of measurements… what if you combine the wrong ones in the calculations (one in KG the other in Liter) •  When you can specify reporting frequency day, week, month… what if you combine different frequencies in the summary? •  What if the user doesn’t add columns when creating the dialogue? •  What if the user gets interrupted and leaves for 15 minutes without saving…. •  What type of users? Do they understand English (error messages were in English)… do they understand technical English? •  ….. Etc etc © Sogeti 23
  • 24. © Sogeti 24
  • 25. Presenting the Findings •  •  •  •  Showing the classification tree Presenting the ”bugs” Discussed severity and priority Documented in defect management tool (backlog) © Sogeti 25
  • 26. Supporting the Team •  •  •  •  •  •  Input to test design Continuous testing of stories Clarification of stories Support creation of acceptance criteria Reproducing defects (sharing desktop) Verifying defect fixes © Sogeti 26
  • 27. How do we Make a Good Unit Integration Test? Automating what was originally manual test Create a new meter with the following values… Remember negative scenarios… © Sogeti 27
  • 28. Regression Test: From Mind map to Quality Center •  •  •  •  •  Attaching the test strategy From mind map to test tree Documenting the charter/mission Attaching the output of test design techniques NO STEPS (or only one – use the attached….) © Sogeti 28
  • 29. The Result •  Maximum time doing testing minimum doing documentation •  Thorough test of new functionality •  Regression test of existing functionality •  Creation of “regression test suite” •  Support for unit integration testing •  Clarification of user stories •  Feedback to/from product owner •  A lot of defects in both new and existing software •  And a product owner that now saw the value in someone doing structured testing •  Can you come back next year? © Sogeti 29
  • 30. What I Took With Me •  Re-discovered the joy in doing exploratory testing •  Focus on how to combine what I know – using my tool belt •  Once again recognizing; the world is not either black or white – there are many colors… it is all about context. © Sogeti 30
  • 31. Questions? Please evaluate my presentation and use for this the AgileTD Mobile App which you can find at www.touchmyconference.com/ATD2013. I would appreciate your feedbacks. Thank you very much! © Sogeti 31
  • 32. A Bit About Unicorns •  Medival and rennasiance belief: a symbol of purity and grace, which could only be captured by a virgin. In the encyclopedias its horn was said to have the power to render poisoned water potable and to heal sickness. •  As big as a donkey, fast, wild and fierce in battle, impossible to catch alive. •  According to Marco Polo: "scarcely smaller than elephants. They have the hair of a buffalo and feet like an elephant's. They have a single large black horn in the middle of the forehead... They have a head like a wild boar's… They spend their time by preference wallowing in mud and slime. They are very ugly brutes to look at. They are not at all such as we describe them when we relate that they let themselves be captured by virgins, but clean contrary to our notions." © Sogeti 32
  • 33. Unicorns – and the Danish Perspective According to legend, the Throne Chair is made of the horn of unicorns © Sogeti 33