W10
Agile Testing
5/1/2013 1:45:00 PM

Exploratory Testing on Agile Projects:
Combining SBTM and TBTM
Presented by:
Christin Wiedemann
Professional Quality Assurance, Ltd.

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
Christin Wiedemann
After eleven years as an astroparticle physicist, Christin Wiedemann brings her logical and analytical
problem-solving skills to the world of testing. Five years into her new career, Christin is still eager to learn,
looking for new ways to test more efficiently, constantly trying new approaches, and keen to share her
experiences. In her roles as tester, test lead, trainer, and speaker, she uses her scientific background and
pedagogic abilities to continually develop her own skills and those of others. Co-creator of the exploratory
testing approach xBTM, Christin lives in Vancouver, where she has joined Professional Quality Assurance
Ltd. Follow Christin on Twitter @c_wiedemann or visit her blog christintesting.wordpress.com.
4/16/2013

Exploratory Testing on Agile Projects:
Combining SBTM and TBTM
STAREAST, Orlando, FL, May 1st, 2013

Christin Wiedemann, PQA Ltd.

test smarter...

Outline
Why test?
Agile testing
Exploratory testing
SBTM, TBTM, xBTM
What’s in it for me?

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

2

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

1
4/16/2013

Why Do We Test?

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

3

The Objective of Testing

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

4

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

2
4/16/2013

Testing on Agile Projects

http://gardenofeaden.blogspot.ca

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

5

Exploratory Testing

Explorer Kazimierz Nowak

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

6

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

3
4/16/2013

Pair-Testing

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

7

Testing Spectrum
xBTM
Manual Scripted

SBTM

TBTM

Freestyle
Exploratory

Exploratory Testing

Figure adapted from original by Jon Bach
©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

8

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

4
4/16/2013

Session-Based Test Management

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

9

Session-Based Test Management
Test Charter
‣ Risk
‣ Coverage
‣ Time frame

Test Charter

Credit: Michael D. Kelly
©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

10

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

5
4/16/2013

Session-Based Test Management
Session Report
‣ When?
‣ Who?

Time break-down

‣ What?
‣ How?
‣ Bugs?

Design
Execu on
Repor ng
Other

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

11

Session-Based Test Management

Test Charters
16
14
12
10
8
6
4
2
0
User Account

Shopping Cart

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

Payment

12

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

6
4/16/2013

Session-Based Test Management
Debriefing
‣ P – Past
‣ R – Result
‣ O – Obstacles
‣ O – Outlook
‣ F – Feelings

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

13

Thread-Based Test Management

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

14

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

7
4/16/2013

Threads – Example

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

15

Threads – Example

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

16

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

8
4/16/2013

Thread-Based Test Management
The essence of TBTM:
‣ Activities change over time
‣ “For me ‘test’ is a verb. Testing is something that I do, not so much

something that I create.”*
‣ Focus on “doing” and not “getting done”

* James Bach, http://www.satisfice.com/blog/archives/503
©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

17

Thread-Based Test Management
How do you do it?
Function area
Test technique

Thread

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

18

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

9
4/16/2013

xBTM

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

19

xBTM – Demo

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

20

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

10
4/16/2013

xBTM – Demo

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

21

xBTM – Demo

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

22

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

11
4/16/2013

xBTM – Demo

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

23

xBTM – Demo

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

24

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

12
4/16/2013

Traditional Workflow

Planning

Design

Execution

Reporting

Waterfall
‣ Consecutive phases

Agile
‣ All phases repeated in every iteration

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

25

xBTM Workflow

Planning

Design

Execution

Reporting

Planning
‣ Threads in mind map
‣ Function areas and/or test techniques
‣ Group threads into sessions (optional)
‣ Estimate number of charters needed (optional)

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

26

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

13
4/16/2013

xBTM Workflow

Planning

Design

Execution

Reporting

Design, Execution, Reporting
‣ Update mind map
‣ Session reports (optional)
‣ Update test charters (optional)
‣ Add threads
‣ Create additional test charters (optional)

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

27

Summary and Conclusion
‣ Exploratory testing can be very structured
‣ Exploratory testing can be thoroughly documented
‣ Mind maps are great for visualization and collaboration
‣ Use SBTM when possible
‣ Use TBTM when environment is too hectic
‣ Adapt the method to your needs

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

28

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

14
4/16/2013

Links – Mind Mapping
XMind: Powerful tool with a lot of nice features, used in examples
‣ http://www.xmind.net

mindmeister: Collaborative tool.
‣ http://www.mindmeister.com

FreeMind: The simpler of the mind mapping tools, but still very useful.
‣ http://freemind.sourceforge.net/wiki/index.php/Main_Page

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

29

Links – SBTM Tools
Rapid Reporter: A note taking tool for exploratory testing sessions.
‣ http://testing.gershon.info/reporter/

Session Tester: A tool for recording and managing exploratory testing
session.
‣ http://sessiontester.openqa.org

SBTExecute: A tool that produces summary reports and calculates
metrics from an Excel session report template.
‣ http://www.addq.se/utforskande-testmetodik-xbtm/ (scroll down to

bottom of page for English)

©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com

30

©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca

15

Exploratory Testing on Agile Projects: Combining SBTM and TBTM

  • 1.
    W10 Agile Testing 5/1/2013 1:45:00PM Exploratory Testing on Agile Projects: Combining SBTM and TBTM Presented by: Christin Wiedemann Professional Quality Assurance, Ltd. Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  • 2.
    Christin Wiedemann After elevenyears as an astroparticle physicist, Christin Wiedemann brings her logical and analytical problem-solving skills to the world of testing. Five years into her new career, Christin is still eager to learn, looking for new ways to test more efficiently, constantly trying new approaches, and keen to share her experiences. In her roles as tester, test lead, trainer, and speaker, she uses her scientific background and pedagogic abilities to continually develop her own skills and those of others. Co-creator of the exploratory testing approach xBTM, Christin lives in Vancouver, where she has joined Professional Quality Assurance Ltd. Follow Christin on Twitter @c_wiedemann or visit her blog christintesting.wordpress.com.
  • 3.
    4/16/2013 Exploratory Testing onAgile Projects: Combining SBTM and TBTM STAREAST, Orlando, FL, May 1st, 2013 Christin Wiedemann, PQA Ltd. test smarter... Outline Why test? Agile testing Exploratory testing SBTM, TBTM, xBTM What’s in it for me? ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 2 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 1
  • 4.
    4/16/2013 Why Do WeTest? ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 3 The Objective of Testing ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 4 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 2
  • 5.
    4/16/2013 Testing on AgileProjects http://gardenofeaden.blogspot.ca ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 5 Exploratory Testing Explorer Kazimierz Nowak ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 6 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 3
  • 6.
    4/16/2013 Pair-Testing ©2012 Professional QualityAssurance Ltd. (PQA) -- http://www.pqatesting.com 7 Testing Spectrum xBTM Manual Scripted SBTM TBTM Freestyle Exploratory Exploratory Testing Figure adapted from original by Jon Bach ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 8 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 4
  • 7.
    4/16/2013 Session-Based Test Management ©2012Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 9 Session-Based Test Management Test Charter ‣ Risk ‣ Coverage ‣ Time frame Test Charter Credit: Michael D. Kelly ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 10 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 5
  • 8.
    4/16/2013 Session-Based Test Management SessionReport ‣ When? ‣ Who? Time break-down ‣ What? ‣ How? ‣ Bugs? Design Execu on Repor ng Other ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 11 Session-Based Test Management Test Charters 16 14 12 10 8 6 4 2 0 User Account Shopping Cart ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com Payment 12 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 6
  • 9.
    4/16/2013 Session-Based Test Management Debriefing ‣P – Past ‣ R – Result ‣ O – Obstacles ‣ O – Outlook ‣ F – Feelings ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 13 Thread-Based Test Management ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 14 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 7
  • 10.
    4/16/2013 Threads – Example ©2012Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 15 Threads – Example ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 16 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 8
  • 11.
    4/16/2013 Thread-Based Test Management Theessence of TBTM: ‣ Activities change over time ‣ “For me ‘test’ is a verb. Testing is something that I do, not so much something that I create.”* ‣ Focus on “doing” and not “getting done” * James Bach, http://www.satisfice.com/blog/archives/503 ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 17 Thread-Based Test Management How do you do it? Function area Test technique Thread ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 18 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 9
  • 12.
    4/16/2013 xBTM ©2012 Professional QualityAssurance Ltd. (PQA) -- http://www.pqatesting.com 19 xBTM – Demo ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 20 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 10
  • 13.
    4/16/2013 xBTM – Demo ©2012Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 21 xBTM – Demo ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 22 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 11
  • 14.
    4/16/2013 xBTM – Demo ©2012Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 23 xBTM – Demo ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 24 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 12
  • 15.
    4/16/2013 Traditional Workflow Planning Design Execution Reporting Waterfall ‣ Consecutivephases Agile ‣ All phases repeated in every iteration ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 25 xBTM Workflow Planning Design Execution Reporting Planning ‣ Threads in mind map ‣ Function areas and/or test techniques ‣ Group threads into sessions (optional) ‣ Estimate number of charters needed (optional) ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 26 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 13
  • 16.
    4/16/2013 xBTM Workflow Planning Design Execution Reporting Design, Execution,Reporting ‣ Update mind map ‣ Session reports (optional) ‣ Update test charters (optional) ‣ Add threads ‣ Create additional test charters (optional) ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 27 Summary and Conclusion ‣ Exploratory testing can be very structured ‣ Exploratory testing can be thoroughly documented ‣ Mind maps are great for visualization and collaboration ‣ Use SBTM when possible ‣ Use TBTM when environment is too hectic ‣ Adapt the method to your needs ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 28 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 14
  • 17.
    4/16/2013 Links – MindMapping XMind: Powerful tool with a lot of nice features, used in examples ‣ http://www.xmind.net mindmeister: Collaborative tool. ‣ http://www.mindmeister.com FreeMind: The simpler of the mind mapping tools, but still very useful. ‣ http://freemind.sourceforge.net/wiki/index.php/Main_Page ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 29 Links – SBTM Tools Rapid Reporter: A note taking tool for exploratory testing sessions. ‣ http://testing.gershon.info/reporter/ Session Tester: A tool for recording and managing exploratory testing session. ‣ http://sessiontester.openqa.org SBTExecute: A tool that produces summary reports and calculates metrics from an Excel session report template. ‣ http://www.addq.se/utforskande-testmetodik-xbtm/ (scroll down to bottom of page for English) ©2012 Professional Quality Assurance Ltd. (PQA) -- http://www.pqatesting.com 30 ©2011 Professional Quality Assurance (PQA) Ltd. -- http://www.pqa.ca 15