The original IEEE 829 Test Documentation standard is thirty years old this year. Boris Beizer’s first book on software testing, Software Testing Techniques, also passed thirty. Testing Computer Software, the best-selling book on software testing, is more than twenty five. During the past three decades, hardware platforms have evolved from mainframes to minis to desktops to laptops to smartphones to tablets. Development paradigms have shifted from waterfall to agile. Consumers expect more functionality, demand higher quality, and are less loyal to brands. The world has changed dramatically—and testing must change to match it. Testing processes that helped us succeed in the past may prevent our success in the future. Lee Copeland shares his insights into the future of testing, including his views in the areas of technology, organization, test processes, test plans, and automation. Join Lee for a thought-provoking look at creating a better testing future.

1. 1
Lee Copeland
Software Quality Engineering
lee@sqe.com
@grandpacopeland
Creating a Better
Testing Future
The World Is Changing and
Testing Must Change With It

2. 2
Topics
• Our World is Changing
• The Curse of Past Successes
• Lee’s Don’ts and Do’s for Future Success

3. Technology Is Changing
Then
Now

4. Customers Are Changing
Less loyal to
brands
Expect more
capability
Less tolerant
of low quality

5. Business Is Changing

6. • In the past, organizations controlled external
communication, both content and channels
• Today, with the rise of Social Media …
Communication is Changing

7. Communication is Changing
• Organizational structure controlled tester
communication, both content and channels
• Today, social media can immediately connect
testers world-wide

8. Development Process Is Changing
Then
Now

9. Development Process Is Changing
• COBOL
• CICS
• VSAM
• C#
• Java
• HTML 4/5
• JavaScript
• XML
• SQL
• noSQL
Then
Now

10. Testing Must Match Today’s Technology
IEEE 829
Testing
Standard
1983
1983 1987

11. Testing Must Match Development

12. Topics
• Our World is Changing
• The Curse of Past Successes
• Lee’s Don’ts and Do’s for Future Success

13. The Curse of Past Successes

14. The Curse of Past Successes
• Processes that made us successful
in the past may prevent us from
being successful in the future

15. The Curse of Past Successes
Camera Film

16. The Curse of Past Successes
Video Rental Stores

17. The Curse of Past Successes
Newspapers

18. The Curse of Past Successes
Brick and Mortar Stores

19. The Curse of Past Successes
Getting Bills in the Mail

20. Topics
• Our World is Changing
• The Curse of Past Successes
• Lee’s Don’ts and Do’s for Future Success

21. The Don’ts and Do’s
Focus
Organization
TestProcess
TechnologyPlanning
Automation

22. The Don’ts and Do’s
Organization

23. Organization
• DON’T
– Maintain teams of “developers” and “testers”,
throwing products over the wall
– This was supposed to foster “independence.”
Instead, it created “us vs. them” and “huh?”
Development Test

24. Organization
• DON’T
– Establish Testing Centers of Excellence
– “It’s where the disengaged gather to not
produce”
– David Hussman

25. Organization
• DO
– Integrate developers and testers together
on the same team

26. The Don’ts and Do’s
Test Process

27. Test Process
• DON’T
– Cling to a Waterfall style when your
organization is moving toward Agile

28. Test Process
• DON’T
– Try to test EVERYTHING

29. Test Process
• DON’T
– Confuse Checking with Testing
Michael Bolton

30. Test Process
• When we already believe something to be true,
we verify our belief by checking. Checking is a
process of confirmation, verification, and
validation
• Checking is a highly automatable process

31. Test Process
• Testing is something that we do with the
motivation of finding new information
• It is a process of exploration, discovery,
investigation, and learning
• Testing is not automatable. It’s using our brain
in real time, and it’s really fun

32. Test Process
• DO
– Adopt a Customer-focused approach to
testing

33. Test Process
• DO
– Focus on “What Value Does Testing Add?”
rather than “What Processes Does Testing
Execute?”

34. The Don’ts and Do’s
Technology

35. Technology
• DON’T
– Use Stone Age techniques, tools, and
technologies

36. Technology
• DO
– Embrace new technologies

37. Technology
• DO
– Consider open source test tools

38. The Don’ts and Do’s
Planning

39. Planning
• DON’T
– Adopt rigid, overly detailed
plans
– Restrict testing to explicit
requirements
– Remember: “No battle plan
survives contact with the enemy”
– Helmuth von Moltke

40. Planning
• DO
– Create plans that
 Tell a story
 Present essential data
 Inspire and communicate
 Describe a reasoned plan of action

41. The Don’ts and Do’s
Automation

42. Automation
• DON’T
– Rely on Capture/Replay

43. Automation
• DON’T
– Try to automate every test

44. Automation
• DON’T
– Try to replace humans in testing

45. Automation
• DO
– Use Test-driven Development

46. Automation
• DO
– Use Acceptance Test-driven Development

47. Automation
• DO
– Use Exploratory Testing
“Simultaneous
learning, test design,
and test execution.”
– James Bach

48. Automation
• DO
– and Session-based Exploratory Test
Management
Perform Exploratory
Testing in Sessions:
 Charter
 Timebox
 Reviewable result
 Debriefing
Jon Bach

49. The Don’ts and Do’s
Focus

50. Focus on Prevention Over Finding
• DO
– Remember, while finding defects is
important
– We should be focused on defect prevention
activities

51. The Don’ts and Do’s
Focus
Organization
TestProcess
TechnologyPlanning
Automation

52. Thanks for Attending
Shameless
commercial
message
lee@sqe.com

53. My Thanks To
• Dorothy Graham
• David Hussman
• Jonathan Kohl
• Alon Linetzki
• Dan North
• Martin Pol
• Rob Sabourin
• Alan Shalloway
• Adrian Zwingli
For their ideas,
review, and
comments