In software development, we transform ideas to code, together. Problems - bugs - that emerge, come in many forms. Have you ever wondered how some people notice problems from blatantly obvious to very subtle? This talk shares the secret to seeing bugs: how to listen when the software talks to recognize a variety of problems, at the right time. First, you need to recognize the many forms bugs come in. Second, you need to look for problems in ways that increases serendipity - lucky accident to find bugs - by varying your ideas on how to use the system. Third, you need to avoid hindsight to be open to hear what software has to say, to harness the potential of everyone being smarter every day we spend at work
Empirical evidence vs. speculation Testing feeding development before development again feeding testing’
Story of my team: 14 % program errors.
THE BEER GAME
introducing variation; all testing is exploratory, including regression testing tacit knowledge: brain and fingers acting independently mapping the territory to know what you’ve covered holding the space: mindset to see bugs can be infectious, invite your testers to hold the space
Ru’s story: regression means exploring, never understood that it means it is worse than it was. Retest / regression.
Brain and fingers sometimes seem to be acting independently (story of double-clicks) and the role of tacit knowledge in finding bugs
Links in code modules, protocol
Accept we’re learning – software development is about continuous learning. Testing is about information, learn on information.
Choices, you would have missed something else if you focused on this. Example with 50 states. Everyone thinks they saw with 9/11 showed two planes, we put the memories together. They just showed one. We don’t even remember them correctly.
Looking back drains energy. Hindsight is wrong, you make that up!
Design with a change in mind. Change from what you’ve learned (validated learning) not just because you can.
Example: if we knew this when we implemented… But we did not. And while we focused on doing what we did, we got that out and it brings value already.
hindsight and forward thinking attitude, let yourself learn when the software speaks to you, don’t punish yourself for not being perfect yesterday, treat every day as a learning opportunity (games, save point and do-over)
Agile is about lowering the cost of change that is inevitable anyway.
Stupidest day on project: day 1. You know more every day since.
#1: default return date is date of search
#2: no arrival time search on front page
#3: detailed search != previous page
#4: date text misaligned in detailed search
#5: search on arrival time does not work
#6: technical errors on console
Ideas to Code
Pipe: One thing at a time,
Funnel: Going through
the options and
making a selection
Examples of Variety
Does not scale
Appendix A from Kaner et al.
1999. Testing Computer Software.
- Includes short descriptions of
over 400 bugs
THE SHIFT-LEFT REVOLUTION OF AGILE