© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Explorato...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
What is a...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Explorato...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Explorato...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Explorato...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
The „free...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Cognifide...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Why?
• Re...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
When?
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Improveme...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Auditable...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Wolf and ...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Improveme...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
AC state
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Checklist...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Heuristic...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Testing D...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Requireme...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Explorato...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Session B...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
SBTM – br...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
SBTM – Bo...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Metrics
•...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Summary
J...
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Q&A
Upcoming SlideShare
Loading in …5
×

Exploratory testing in practice, short story how approach influenced on strategy

996 views

Published on

Presentation for Polish TestingCup

Authors: Zbyszek Moćkun, Łukasz Morawski

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
996
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • ISTQB – technika, według osób, które praktykują testy aksploracyjne to podejście. Radek bardzo fajnie porównał różnice na Testerzy.pl - http://www.testerzy.pl/baza-wiedzy/testowanie-eksploracyjne-kontra-istqb
  • Whittaker – wile tour’ow jest opisanych na stronach msdn: http://msdn.microsoft.com/en-us/library/jj620911.aspx Touring - bardzo przydatny np jesli wybierzemy Session based testing, pomaga nam zaplanowac testowanie – podzielic na sesjie –po takim przejsciu mamy praktycznie test plan. Jest jeszcze kilka takich heurystyk/technik – te sa najbardziej popularne. SFDPO Mnemonic  - Mike Kelly, 2005 Structure, Function Data, Platform. Operation HICCUPPS heuristic -Bach History, Image, Compare products, ...
  • Whittaker – ze strony msdn o exploratory testing
  • http://agile2010.agilealliance.org/files/Telling%20Your%20Exploratory%20Story%20Agile2010.pdf Vague – niejasne Where you are, where you want to be? Where are your process?. How looks Cognifide’s road? Cognifide – pure scripted an freestyle exloparotry projects, than on ly freestyle – and now something between fragmentary test cases and charters
  • Dlaczego wprowadziliśmy testy eksploracyjne zmienność wymagań, możliwość adaptacji testów do aktualnych potrzeb, szybki feedback dla developera, współpraca tester-developer (gdzie developer na bieżąca naprawia błędy ,znalezione przez testera), domena - brak możliwości napisania wszystkich scenariuszy poźne zaangażowanie QA w projekt, późno QA w projekcie, niestety wymagania często doprecyzowane w czasie sprintu, co oznacza brak czasu na przygotowanie odpowiednich skyptów retesty - brak narzutu na zarządzanie skryptami testowymi, wybór odpowiednich pod suite do retestu, …. avoid miniwaterfall model
  • Testy Akceptacyjne (Story acceptance testing) - testy wewnętrzne wykonywane przez testera który ma potwierdzić że funkcjonalność działa jak powinna i nie działa jak nie powinna. Tutaj testy eksploracyjne się sprawdzają gdzie ważny jest “Time to Market”: szybki odpowiedź zwrotna o błędach - naprawa błędu - retest część powodów z góry jak odporność na zmiany wymagań Lean - czyli rób tylko to co da wartość biznesowa dla klienta Testy regresyjne - Tutaj jest gorzej. Lepiej sprawdzają się skrypty które mozna powtórzyć ale wymagamy też analizy sesji przez testera i pomysłów (test idea). poziom detalicznosci testów testy, ktore daja testerowi odrobine wolnosci charakter testera - dociekliwy, ... test scenario jak mini sesja np. 5 minutowa,
  • Szybko wprowadzilismy kilka usprawnien Zaczlismy robic notatki (na tamta chwile raczej bylo to zaczerpniecie z session). Dlugo szukalisy narzedzia, ktore pomoze nam tworzyc czytelne notatki, niestety nic takiego nie znalezlismy, wiec kolega napisal wlasne, Wlasne checklisty
  • Jej uwagi do naszego procesu. brak powiazan z Acceptance Criteria z notatek z sesji nie wynikalo, iz dany akceptance criteria jest spelnoniony jakie scenariusze testowe zostaly wykonane przeciwko temu AC/story zalaczniki nieczytelne graficzna wizualizacja wynikow np - ktore acceptance pass, ktore failed, jakie bledy byly zglaszane przeciwko ktoremu acceptance, … Jakie testy byly wykonane brak scenariuszy testowych, bo notatki z sesji sa nieczytelne dla osob spoza projektu / przyzwyczajonych do test scenariuszy Brak integracji z jira - np wyszukiwanie potrzebnej informacji z notatek czy tez brak traceability
  • Zawasze jest to trudna decyzja, ale trzeba ja podjac i poszukac kompromisow. Mimo iz chcemy byc jak najblizej lewej strony, musimy zadowolic klienta czy tez nasze wewnetrzen standardy
  • Szybko wprowadzilismy kilka usprawnien Zaczlismy robic notatki (na tamta chwile raczej bylo to zaczerpniecie z session). Dlugo szukalisy narzedzia, ktore pomoze nam tworzyc czytelne notatki, niestety nic takiego nie znalezlismy, wiec kolega napisal wlasne, Wlasne checklisty
  • Szybko wprowadzilismy kilka usprawnien Zaczlismy robic notatki (na tamta chwile raczej bylo to zaczerpniecie z session). Dlugo szukalisy narzedzia, ktore pomoze nam tworzyc czytelne notatki, niestety nic takiego nie znalezlismy, wiec kolega napisal wlasne, Wlasne checklisty
  • Szybko wprowadzilismy kilka usprawnien Zaczlismy robic notatki (na tamta chwile raczej bylo to zaczerpniecie z session). Dlugo szukalisy narzedzia, ktore pomoze nam tworzyc czytelne notatki, niestety nic takiego nie znalezlismy, wiec kolega napisal wlasne, Wlasne checklisty
  • Zasada oparta na praktyce, zbior dorbych praktyk
  • Szybko wprowadzilismy kilka usprawnien Zaczlismy robic notatki (na tamta chwile raczej bylo to zaczerpniecie z session). Dlugo szukalisy narzedzia, ktore pomoze nam tworzyc czytelne notatki, niestety nic takiego nie znalezlismy, wiec kolega napisal wlasne, Wlasne checklisty
  • Czyli czy testowac wymagania. Testy eksploracyjne sa reactive, czyli już na zaimplementowanym kodzie. Czy w takim razie nie musimy testowac wymagan? Nigdzie tak nie jest powiedziane – tutaj przychodzi nam Agile i Scrum z pomoca. Grooming session – powinno odbywac sie minimu tydzien przed sprintem, albo wczesniej. Wymagania sa przegladane przez zespol – czyli tez QA. Jest to czas, kiedy mozna zadac pytania, stwierdzic iz acceptance tests sa nie testowalne, zglaszac uwagi itp Sprint planning nie jest miejscem do tego. Tutaj planujemy sprint, pczest przechodzimy ze story points do czasu, QA planuje swoja prace w czasie sprintu, ... – to nie jest czas testowania/review – juz jest za pozno No i eksploracja –tutaj niestety dzialamy juz na zaimplementowanym kodzie, nie znaczy ze zawsze wszystko udala nam sie wczesniej wyjasnic, na pewno znajdziemy nie jasnosci, ... A moze bedziemy chcieli wplynac na ksztalt wymagan, gdyz to co zostalo stworzeone nie do konca ma sens, dziala intucicyjnie, ...
  • Past. What happened during the session? Results. What was achieved during the session? Obstacles. What got in the way of good testing? Outlook. What still needs to be done? Feelings. How does the tester feel about all this
  • Jakie plusy ma bonfire Sesje z poziomu jira Demo
  • http://agile2010.agilealliance.org/files/Telling%20Your%20Exploratory%20Story%20Agile2010.pdf Vague – niejasne Where you are, where you want to be? Where are your process?. How looks Cognifide’s road? Cognifide – pure scripted an freestyle exloparotry projects, than on ly freestyle – and now something between fragmentary test cases and charters
  • Exploratory testing in practice, short story how approach influenced on strategy

    1. 1. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Exploratory Tests in practice Short story about how test approach influenced on test strategy Zbyszek Moćkun, Łukasz Morawski
    2. 2. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. What is about? Exploratory Testing
    3. 3. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Exploratory tests definition • Approach or technique? • What about ISTQB? • Test techniques (boundary values, decision table, ...) • Scripting against exploratory „A technique is a recipe for solving a problem, whereas a style (or approach) is a way of thinking around a theme that stretches far beyond solving a particular problem” Henrik Emilsson (Blog Thoughts From The Test Eye)
    4. 4. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Exploratory approach techniques • „The Intelectual” tour • Landmark tour • Money tour • Supermodel tour • Feature • Complexity • Claims • Configuration • User • Testability • Scenario • Variability • Interoperability • Data • Structure FCC CUTS VIDS tour by Michael Kelly Touring by James Whittaker
    5. 5. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Exploratory as extension for scripting „Having formal scripts can provide a structure to frame exploration, and exploratory methods can add an element of variation to scripts that can amplify their effectiveness.” James Whittaker
    6. 6. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. The „freedom” scale Jon Bach, Telling your exploratory story Where are you on this scale?
    7. 7. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Cognifide journey Exploratory Tests
    8. 8. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Why? • Requirements − Changeability − Clarification of the requirements in the development of production or testing • Tester – Dev cooperation − Close Dev – Tester cooperation − Start as fast as possible − Avoid miniwaterfall model − Quick feedback − CI environment − QA services − Retesting − Raising improvements (usability as example)
    9. 9. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. When?
    10. 10. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Improvements - part I • Repeatable − Notes − Checklists • Visualization (Integration with dashboard) − Requirement status − Testing activities • Review process − No time for debrief − Cross testing (mostly for juniors)
    11. 11. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Auditable? Pharmaceutical company performed the audit − Text notes are unclear and contains mistakes/not all information − Lack of test scripts − No traceability  Test scenarios against AC  Issues against AC − No visual status of Acceptance Criteria − No JIRA integration − No regression execution history − No automation coverage Audit went well, but showed room to improve
    12. 12. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Wolf and sheep John Rieger – wolf and sheep
    13. 13. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Improvements - visualization
    14. 14. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. AC state
    15. 15. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Checklist: web domain
    16. 16. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Heuristics • Rule of thumb • Test Heuristics Cheat Sheet by Quality Tree Software
    17. 17. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Testing Dojo
    18. 18. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Requirement review
    19. 19. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Exploratory tests in steps • Task delivered to QA (or small part of the task) • Learning phase (including documention) • Test planning (tour session) • Execute planned sessions (note, bugs, questions, AC update) • Task rejected (back to developer) • Fixing • Retest planning (sessions, or no need) − Retest bugs − Regression testing if needed (sessions) • Writing test scenario based for regression purpose • Accept story
    20. 20. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Session Based Test Management • Time management • Test planning (FCC CID VIDS tour) • Charters • Test status • Debrief (PROOF concept) • Notes • Tools − Research existing tools − Session Tester (our own tool) − Bonfire
    21. 21. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. SBTM – break the rule • Sessions are not interrupted? − Tester - developer pairing − Continuous Integration environment − Raising bugs take several seconds  Bonfire plugin as example • Notes − Level of details • One session per task • Session debrief − Lonely knight − Time pressure − Cross testing − Project audit
    22. 22. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. SBTM – Bonfire (JIRA Capture) • Session Management • JIRA integration • Plugin for most common browsers
    23. 23. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Metrics • Exploratory metrics againist scripting approach • Internal against External Quality − Rejects − Bugs • Requirement coverage − Scripts against sessions • Sessions metrics
    24. 24. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Summary Jon Bach, Telling your exploratory story
    25. 25. © 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only. Q&A

    ×