Jon Bach, with over eighteen years of software testing experience, discusses the evolution of testing from 1994 to 2014, emphasizing the shift towards automation and the challenges faced in a DevOps environment. He shares insights on the nature of testing as a discovery process and presents practical solutions for improving testing practices. The presentation also highlights the importance of adapting to changing contexts and the need for creativity and innovation in testing.

1. MP
PM Tutorial
10/13/2014 1:00:00 PM
"Jon Bach: On Testing"
Presented by:
Jon Bach
eBay, Inc.
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Jon Bach
eBay, Inc.
With more than eighteen years of software testing experience, Jon Bach has held technical and
managerial positions in companies including Hewlett-Packard and Microsoft. As director of Live
Site Quality for eBay, Jon is dedicated to building “end-to-end” tests (activity flows) in eBay’s
core sites to discover important bugs that threaten its core business. He and his brother James
created Session-Based Test Management, a method to manage and report exploratory testing.
Jon frequently speaks at the STAR conferences and usually can be found wearing a ball cap,
hanging out in the conference hallways, encouraging others, and sharing best testing ideas and
patterns.
Speaker Presentations

3. 9/16/2014
1
On Testing
Jon Bach
STAR West 2014
10/13/14
2
Is testing in 2014 different from testing in 1994?
If you’ve been in the business that long, maybe you’ve seen it move
from a little bit of automation and tooling to almost total automation and
tooling.
Maybe you’ve seen lab setups go from hours of loading OS images on
“boat anchor” boxes to virtual, on-demand, scalable cloud provisioning
in seconds.
Maybe you think testing is dead because we live in a DevOps world
where it’s good enough to run a “happy-path” checklist.
Maybe you think testing isn’t dead because you’ve seen recent
computer science graduates know dangerously nothing about the craft
of testing.
Preamble
On Testing

4. 9/16/2014
2
What is testing?
How have you tested?
What problem have you solved?
What’s your best idea?
What still vexes you?
3
Five Topics
On Testing
4
Me (the tester) 1995 - 2010
•1995 – Microsoft Bookshelf (contractor)
•1996 – ContactBuilder
•1997 – Microsoft SMS 2.0
•2000 – Satisfice, Inc.
•2002 – Microsoft Flight Simulator
•2003 – Quardev, Inc.
•2009 – LexisNexis (outsourcing layoff)
•2010 – Quardev, Inc.
•2011 – eBay, San Jose
On Testing

5. 9/16/2014
3
5
The Road
QE Director
(Search: Front
End)
Director
(Live Site
Quality)
QE/PM Director
(End-to-End
Solutions)
PM Director
(CommerceOS)
Quality
Evangelist
(Global Data
Infrastructure)
2011 2012 2013 2014 NOW
On Testing
6
Me, the evangelist, 2014
On Testing

6. 9/16/2014
4
7
Me, the evangelist, 2014
On Testing
What is testing?
How have you tested?
What problem have you solved?
What’s your best idea?
What still vexes you?
8
Five Topics
On Testing

7. 9/16/2014
5
What is Testing?
9
I’ll go first
On Testing
Testing is about DISCOVERY…
On Testing

8. 9/16/2014
6
…finding emerging context
On Testing
enhanced by conference…
On Testing

9. 9/16/2014
7
...affected by inference
On Testing
…framed by reference
On Testing

10. 9/16/2014
8
Testing is…
an infinite process
of comparing the invisible
to the ambiguous
in order to avoid the unthinkable
happening to the anonymous.
-- James Bach
15On Testing
Testing is Journalism
16
Testing Premise #1
On Testing

11. 9/16/2014
9
What is testing?
How have you tested?
What problem have you solved?
What’s your best idea?
What still vexes you?
17
Five Topics
On Testing
18
Agile
On Testing

12. 9/16/2014
10
My desk
On Testing
What is testing?
How have you tested?
What problem have you solved?
What’s your best idea?
What still vexes you?
20
Five Topics
On Testing

13. 9/16/2014
11
Solutions that worked for me
1) Boring test plans? Use color to convey info
2) Test this when? Time-box your tests (RST / SBTM)
3) Customer needs relief? Scenario task force
4) No morale? Tester games
5) I’m stuck! Heuristics
6) When to ship / stop? Good Enough
On Testing
Is it good enough?
Good enough quality bar
Further time and effort
is crucial.
Further time and effort
is a waste of resources.
Unacceptable Quality
Too Much Quality
floating line
On Testing

14. 9/16/2014
12
What is testing?
How have you tested?
What problem have you solved?
What’s your best idea?
What still vexes you?
23
Five Topics
On Testing
What is testing?
How have you tested?
What problem have you solved?
What’s your best idea?
What still vexes you?
24
Five Topics
On Testing