This document describes SAP Labs China's approach to exploratory testing within a Scrum framework, emphasizing the importance of manual testing for certain features. It introduces various testing techniques and tours that enhance the testing process, allowing testers to creatively explore software while focusing on specific aspects. The presentation also discusses challenges and the integration of exploratory testing into the Scrum process to improve team collaboration and test coverage.

1. Our Exploratory Testing Journey
Zhang Kebing
SAP Labs China
Public
June, 2012
© 2011 SAP AG. All rights reserved. Public 1

2. Disclaimer:
This presentation outlines our general product direction and should not be relied on
in making a purchase decision. This presentation is not subject to your license
agreement or any other agreement with SAP. SAP has no obligation to pursue any
course of business outlined in this presentation or to develop or release any
functionality mentioned in this presentation. This presentation and SAP's strategy
and possible future developments are subject to change and may be changed by
SAP at any time for any reason without notice. This document is provided without a
warranty of any kind, either express or implied, including but not limited to, the
implied warranties of merchantability, fitness for a particular purpose, or non-
infringement. SAP assumes no responsibility for errors or omissions in this
document, except if such damages were caused by SAP intentionally or grossly
negligent.
© 2011 SAP AG. All rights reserved. Public 2

3. The role of manual testing in Scrum
Shipping new features after each sprint
We use manual test when
1. Some features are too costly to automate.
2. Some features are not automatable.
3. Find the complicated bugs.
4. Explore new ways of testing a feature.
© 2011 SAP AG. All rights reserved. Public 3

4. Ad hoc vs. Scripted Test Case Design
Ad hoc Scripted
Testing Design effort Testing
Low High
• No test strategy • High
planning/maintenance
• No focus
effort
• No traceability
• No variations
• Kills creativity
© 2011 SAP AG. All rights reserved. Public 4

5. Exploratory Software Testing with Tours
A lightweight approach to test case design and execution
[1]
© 2011 SAP AG. All rights reserved. Public 5

6. Exploratory Testing
Interaction
Creativity
[2]
[3]
© 2011 SAP AG. All rights reserved. Public 6

7. Analogy: A software tester is like a tourist…
• You can not see everything
• There are lots of different ways of exploration
• You want to make sure to see the most
important things
[4]
© 2011 SAP AG. All rights reserved. Public 7

8. Feature: Campaign Management in Real Estate
© 2011 SAP AG. All rights reserved. Public 8

9. Garbage Collector Tour
Garbage men go street by street, house by house. They stay only
a few moments but they crisscross the neighborhood in a
methodical manner.
© 2011 SAP AG. All rights reserved. Public 9
[5]

10. Garbage Collector Tour 1
 Example for Campaign in Real Estate
1. Open the UI of add a new Campaign.
2. Execute all main sub-frames.
3. Make sure you go to each sub-frame and
create a new item.
4. Don’t test every detail.
5. Test document
© 2011 SAP AG. All rights reserved. Public 10

11. Garbage Collector Tour 2
 Example for Campaign in Real Estate
1. Open the UI of add a new Campaign.
2. Only test one campaign type, but check all the buttons on the menu.
3. Don’t touch other sub features.
4. Test Document
© 2011 SAP AG. All rights reserved. Public 11

12. Anti-Social Tour
You are not like everybody else.
[6]

13. Anti-Social Tour
 Example
If valid input is a number enter a
character
if valid input is a picture file use a word
file
if you are on the last page of a wizard
hit “Back” and change something, …
[7]
© 2011 SAP AG. All rights reserved. Public 13

14. Supermodel Tour
It’s all about the look… 
© 2011 SAP AG. All rights reserved. Public 14
[8]

15. Supermodel Tour
Design and execution of the tour in general:
1. Pick a feature
2. Make sure it follows the UI guidelines
3. All icons are correct
4. All texts contain no spelling mistakes
5. …
© 2011 SAP AG. All rights reserved. Public 15

16. Standard Tours
Business District Tourist District Seedy District
• Guidebook Tour (F1 • Collector's Tour • Saboteur
Tour) • Lonely Businessman • Antisocial Tour
• Money Tour Tour • Opposite
• Landmark Tour • Supermodel Tour • Crime Spree
• Intellectual Tour • TOGOF Tour • Wrong Turn
• FedEx Tour (TestOneGetOneFree) • Obsessive-Compulsive
• After-Hours-Tour • Scottish Pub Tour
• Garbage Collector's
Tour1
Historical District Entertainment District Hotel District
• Bad-Neighborhood • Supporting Actor Tour • Rained-Out Tour
• Museum Tour • Back Alley Tour • Couch Potato Tour
• Prior Version Tour • All-Nighter Tour
James Whittaker
Exploratory Software Testing: Tips, Tricks, Tours, and Techniques to Guide Test Design
Addison Wesley Pub Co Inc, 2009
© 2011 SAP AG. All rights reserved. Public 16

17. Exploratory Testing with Tours!
Failure!
Failure!
Failure!
Failure! [14]
© 2011 SAP AG. All rights reserved. Public 17

18. Exploratory Software Testing Ingredients
Time Box Protocol Test Tour
© 2011 SAP AG. All rights reserved. Public 18

19. Time Box
 Helps you to
stay focused
 It should
between 30 – 60
minutes per tour
 You must take
note, record
your test within
time box
 Prepare the test
before you start
to test
© 2011 SAP AG. All rights reserved. Public 19

20. Protocol – Test Document
 Written on the fly by the
tester
 Documents what has
been tested
 Documents which bugs
have been found
 Documents which test
opportunities are still
available
 Documents open
questions (bug or
feature?)
 Examples
© 2011 SAP AG. All rights reserved. Public 20

21. Test Tour
• Defines test approach
(How)
• Leaves the tester enough
freedom to explore
• Ensures tester stays
focused, not testing
everything
[9]
© 2011 SAP AG. All rights reserved. Public 21

22. Integration into the Scrum process
- Test Strategy
Team A:
Unit Test
Auto-System Test
ET
Team B: Scripted Test
ET
© 2011 SAP AG. All rights reserved. Public 22

23. Integration into the Scrum process
- Which tour?
Reports
• Intellectual Tours : ask the hardest
questions to the reports.
• Monthly close?
• Enter transactions with huge
amount?
• View reports with multiple
currencies?
• …
© 2011 SAP AG. All rights reserved. Public 23

24. Integration into the Scrum process
- Which tour？
Invest more than one tours for a feature.
• Garbage collector tour - Test
Coverage
• Anti-social tour - invalid inputs or
operations.
• Landmark tour - Features in different
sequences.
© 2011 SAP AG. All rights reserved. Public 24

25. Integration into the Scrum process
Story Tasks To-Do In Process Completed Done
Story Story
Bug
1. Define Tasks – Planning Meeting
2. Test Prepare - Tester
3. Who perform – any team member
4. Rules – Cross test, start test when all development tasks is done.
© 2011 SAP AG. All rights reserved. Public 25

26. What ET bring to our scrum team?
Long document Just Enough
VS
[10]
© 2011 SAP AG. All rights reserved. Public 26

27. What ET bring to our scrum team?
• Enlarge testing resource
• Involve everyone
[11]
© 2011 SAP AG. All rights reserved. Public 27

28. What ET bring to our scrum team?
• Better test coverage
• Each of tester has their own way
to explore the software
[12]
© 2011 SAP AG. All rights reserved. Public 28

29. Current Challenges
• Try more tours with scrum team
• Try more different approaches
• Follow up on bug reports and find out whether your test strategy
works or not.
© 2011 SAP AG. All rights reserved. Public 29

30. Try!
Contact information:
Zhang Kebing
Quality Engineer/ Scrum Master
KebingZhang@hotmail.com
© 2011 SAP AG. All rights reserved. Public 30

31. References
[1] http://www.flickr.com/photos/statelibraryofnsw/2876551803/
[2] http://www.quardev.com/articles/exploratory_testing
[3] http://www.flickr.com/photos/bobcatrock/2862617356/
[4] http://www.flickr.com/photos/gamene/4341746457/
[5] http://www.flickr.com/photos/nasacommons/4858566574/
[6] http://www.flickr.com/photos/foxypar4/1852428153/
[7] http://www.flickr.com/photos/judepics/1950452493/
[8] http://www.flickr.com/photos/istolethetv/4045140609/
[9] http://www.nipic.com/show/1/66/71cbc36ac3b50aa9.html
[10] http://www.superstock.com/stock-photos-images/1775R-6937
[11] http://www.allmoviephoto.com/photo/2003_poster_smith_international.html
[12] http://www.fs3000.com.cn/bbs/read.php?tid=610
[14] http://blog.sysomos.com/2010/05/19/twittera-as-a-resource-tool/question-
mark/
© 2011 SAP AG. All rights reserved. Public 31

32. © 2011 SAP AG. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or for any SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects
purpose without the express permission of SAP AG. The information contained Explorer, StreamWork, and other SAP products and services mentioned herein as
herein may be changed without prior notice. well as their respective logos are trademarks or registered trademarks of SAP AG
in Germany and other countries.
Some software products marketed by SAP AG and its distributors contain
proprietary software components of other software vendors. Business Objects and the Business Objects logo, BusinessObjects, Crystal
Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business
Microsoft Corporation. Objects products and services mentioned herein as well as their respective logos
are trademarks or registered trademarks of Business Objects Software Ltd.
IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, Business Objects is an
System x, System z, System z10, System z9, z10, z9, iSeries, pSeries, xSeries, SAP company.
zSeries, eServer, z/VM, z/OS, i5/OS, S/390, OS/390, OS/400, AS/400, S/390
Parallel Enterprise Server, PowerVM, Power Architecture, POWER6+, POWER6, Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other
POWER5+, POWER5, POWER, OpenPower, PowerPC, BatchPipes, Sybase products and services mentioned herein as well as their respective logos
BladeCenter, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF, are trademarks or registered trademarks of Sybase, Inc. Sybase is an SAP
Redbooks, OS/2, Parallel Sysplex, MVS/ESA, AIX, Intelligent Miner, WebSphere, company.
Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBM
All other product and service names mentioned are the trademarks of their
Corporation.
respective companies. Data contained in this document serves informational
Linux is the registered trademark of Linus Torvalds in the U.S. and other purposes only. National product specifications may vary.
countries.
The information in this document is proprietary to SAP. No part of this document
Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or may be reproduced, copied, or transmitted in any form or for any purpose without
registered trademarks of Adobe Systems Incorporated in the United States and/or the express prior written permission of SAP AG.
other countries.
Oracle and Java are registered trademarks of Oracle and/or its affiliates.
UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.
Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and
MultiWin are trademarks or registered trademarks of Citrix Systems, Inc.
HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C ®,
World Wide Web Consortium, Massachusetts Institute of Technology.
© 2011 SAP AG. All rights reserved. Public 32