Many resources describe how to accelerate performance of your development organization through adoption of agile methodologies, but very few cover testing in a practical manner. And those that do generally focus on technical details, leaving out how to build an agile testing culture while facing numerous adoption challenges. Leigh Ishikawa describes how an organization needs to rethink testing in the agile world. He begins by taking a holistic look at how different groups combine in an agile testing culture. Then Leigh dives into key components including messaging, concepts, metrics, and tools that can be implemented across different groups; how they are integral to one another; how various data from metrics across different teams should be interpreted; and what actions should be taken. Through real world examples from various companies, Leigh takes you through lessons he learned—from both success and failure.

1. AT4
Agile Development Concurrent Session
11/13/2014 10:00 AM
"Establishing an Agile Testing
Culture"
Presented by:
Leigh Ishikawa
TripAdvisor
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. With a career spanning more than two decades in various
roles, Leigh Ishikawa facilitates change in how organizations view
testing by establishing test frameworks and processes, and
educating and building teams across multiple continents. In the past
decade, Leigh has focused on building test cultures inside high
velocity engineering teams. After spending a year and half creating
test frameworks and coaching engineers on testing, Leigh is
currently a developer on TripAdvisor’s mobile team.

3. Establishing an testing culture
Leigh Ishikawa

4. Agile Culture
• It’s not something you mandate for small
segment of your population
• It’s something that every individual and
process has to respect and uphold
• Must support organization’s vision
– Some organizations are not meant for agile

5. Where do we begin?

6. Failures Observed
• Promises
– Improve quality and customer satisfaction while
reducing development time
• (Trying to) replicate someone else’s culture
– Bring a person from another company
• Using Agile as means to solely drive
engineering productivity
– Business continues it’s merry way

7. Successes observed
• Agile(like) organizations have accepted the
agile culture as a whole
• Agile(like) organizations apply agile to
iteratively improve their overall organization
as well as their engineering
• Agile(like) organizations value people and
fundamentally practice the manifesto

8. Agile Manifesto
• Individuals and interactions over processes and
tools
• Working software over comprehensive
documentation
• Customer collaboration over contract
negotiation
• Responding to change over following a plan
• That is, while there is value in the items on
the right, we value the items on the left more.

9. Good reasons to adopt Agile
• Desire to make a better product
– Better product != $$$
• Desire to have a better work environment
– Attract and retain talent

10. Bad reasons for using Agile
• Improve quality with little/no effort
• Improve productivity with little/no effort
• Put better process in place
– More monitoring
– More metrics
– Reduce cost - Move resources overseas
• Agile is just for engineers

11. Testing
Are we there yet?

12. Testing is hard
• Most companies struggle
• Most never get to where they are really happy
with their testing
• Most companies put tools in place that is
ignored and neglected
• Many people think there’s a ‘secret recipe’

13. Poor quality speaks of many things
• Indicator of bigger organizational problems
– Either disconnect between staff and management
or staff who are disengaged
• If careless, could be a downward spiral
– The more you focus on the wrong things, the
worse it gets
• Qualitative goals are hard to measure
– And difficult to enforce correctly

14. Automation
• Fails most of the time because people
underestimate the difficulty
– Hire/promote wrong (or bad) talent
• Treat testing as a second class citizen
– Desperately try to use existing resource
• Most QA people can’t code and they shouldn’t either
– Buy expensive software/contracts who promises just that
– Have a non SE design a test framework
• Your first/key hire often determines the fate

15. Developers should test
• Good or bad intentions?
• Good organizations are willing to attract and
allocate developers for testing projects
• Bad organizations will mandate that testing
should happen (more or less) on developer’s
personal time and shame them
• Bad organizations think testing just happens
• Testing is an art

16. Test Metrics
• Drive your teams with mandated metrics will
lead you to failure
– “The Logic of Failure”
• While trying to achieve immediate goals, they
destroyed something crucial
• Metrics should be used as a means to observe
and understand the state

17. Metrics – Code Coverage
• Code coverage goals are, generally, a bad idea
• That said, In Java (at the package level)
– 40% = Team is beginning to get grasp of testing
– 60% = a sweet spot of where your test is getting a
good coverage
• Percentage is meaningless without
understanding what is tested. Above % is
based on team prioritizing what they are
testing. (Ignore setters/getters, etc)

18. You’ve heard this before
1. Hire the right people
2. Provide infrastructure
– CI, dashboards, etc.
3. Provide goals and visual aids
– Dashboards
– Signs - No build break for ### days
4. Re-enforce the good behavior
– Build breaks stops everything
– Test failures = build breaks

19. Reality
• You will hire the wrong
person/people/consulting firm
• You will buy/build wrong infrastructure
• You won’t know what metrics to measure (or
avoid)
• ALL THE ABOVE DOESN’T MATTER

20. Great testing culture is not
• Bugs
• Metrics (Test coverage, customer surveys, etc)
• Dashboards
• Speed of your development

21. Great Testing culture is
• Desire of individual and organization to build
better product
• Open to new ideas
• Willing to invest and try different ways to
make your software better through iterative
approach
• Transparency within organization and your
customers

22. Your company is unique
Your company is a living entity filled with unique
individuals.
No two companies are alike and should try to be
Each company has their strengths and weakness
Understand what makes your company special

23. Before you attempt to change
• Find your company’s strengths
• Understand why your company works the way
they do
• Figure out where you company wants to get to
and where it needs to
• Iterate over various tools and process to make
improvements over long period of time

24. Conclusion?

25. To Establish a testing culture, you
must grok organization’s culture
and Agile Manifesto. You must
practice agile manifesto at the
cultural level while preserving the
core values of the organization