As organizations transition to agile processes, Quality Assurance (QA) activities and roles need to evolve. Traditionally, QA activities occur late in the process, after the software is fully functioning. As a consequence, QA departments have been “quality gatekeepers” rather than actively engaged in the ongoing development and delivery of quality software. Agile teams incrementally deliver working software. Incremental delivery provides an opportunity to engage in QA activities much earlier, ensuring that both functionality and important system qualities are addressed just in time, rather than too late. Agile teams embrace a “whole team” approach. Even though special skills may be required to perform certain development and Quality Assurance tasks, everyone on the team is focused on the delivery of quality software.

1. 10/13/2014
1
QA to AQ: Shifting towardsBeing Agile at QualityJoseph Yoderwww.refactory.comwww.teamsthatinnovate.com
Copyright 2014 Joseph Yoder & The Refactory, Inc.
Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more. …From the Agile ManifestoAgile and Quality

2. 10/13/2014
2
Agile Design Values Core values: Design Simplicity Communication Continuous Improvement Teamwork/Trust Satisfying stakeholder needs Building Quality Software Keep learning Lots of Testing!!! Many Agile Teams “only” Focus on Functional TestingFunctional: How do I …? Tests user stories work as advertised•“As a reviewer I want to add a note to a chart” •Compute the charge for an invoice Tests boundary conditions•Can I add more than one•note at the same place? •Are excess charges computed correctly?

3. 10/13/2014
3
Non-functional RequirementsAccessibility CompatibilityEfficiencyEffectivenessExtensibilityMaintainability Performance
Other terms for non-functional requirements: "constraints", "quality attributes“, and "quality of service requirements"
Qualities are usually described by “ilities” as seen innon-functional requirements…but quality can also focuson how well functional requirements are met
Reliability
Safety
Scalability
Security
Stability
Supportability
UsabilitySome Agile Myths System Qualities can be added with an evolving architecture We can easily adapt to changing requirements (new requirements) You can change the system fast!!! Don’t worry about the performance, scalability, security, usability …. until functionality is working…

4. 10/13/2014
4
Design is about Tradeoffs Usability and Security often have orthogonalqualities Designing Interfaces: Patterns for Effective Interaction Design Security Patterns: Integrating Security and Systems Engineering Performance vsSmall Memory•Quality of being good enough
Secure
Usable
Cheap
Scientist
Artist
Engineer
Designer
important/boring
true/false
cool/uncool
good/badWho are your system’sQuality Stakeholders? Users of the system Executives / System sponsors Internal quality stakeholders: Developers Database admins Business process experts Corporate compliance guys

5. 10/13/2014
5
Quality Assurance TeamsUnderstands testing well and knows how to specify and validate system qualitiesCertify the functionality of the application based upon the contract and requirementsMany QA groups work independently from the software teamUsually involved late in the process and not a lot of communication with teamUsually not part of the Agile team… BECOMING AGILE AT QUALITY

6. 10/13/2014
6
Tearing Down the Wallsaka “Breaking Down Barriers”
Physical Barriers, Cultural Differences
Language/Communication, Background
Expertise, Lack of Time, Us and Them
Mentality
How can agile teams remove the barriers and become more agile at quality?
Tear down the walls through various actions: include QA early on; make them part of the sprints, embed them in the teamsAGILE QUALITYBuild quality into your project rhythms

7. 10/13/2014
7
How Quality Scenarios and Quality Testing Fit Into An Agile Process
Product Envisioning
/
Roadmap
Deploy to Stakeholders
Functional
Acceptance Testing
Develop
and Manage
the Backlog
Plan a Sprint
Run a Sprint
Daily Review
Incorporate Feedback
Identify Key
Quality Scenarios
Can Include Quality
Scenarios
Include
relevant
quality
tasks
Quality Testing
Requirements Envisioning (days/weeks/...)
Architecture Envisioning (days/weeks/…)
Iteration 0: Envisioning
Iteration Modeling (hours)
Model Storming(minutes)
Fast TDD(hours)
Iteration n: Development
a little bit of modeling then a lot of coding, think about what qualities might fit into the “Sprint”
Conceptual Modeling
What Qualitiesare important? Test-Driven DevelopmentShould Include Quality

8. 10/13/2014
8
Focus on Quality QA can gather / organize quality scenarios in collaboration with the development team Additional quality scenarios can be gleaned from Service Level Agreements (SLAs) Include relevant quality tests as part of each sprint Test important qualities early Automate “easy” quality testsQuality Scenarios, Quality Stories and Measuring
“Users initiate 1,000 order transactions per minute under normal operations; transactions are processed with an average latency of 2 seconds.”

9. 10/13/2014
9
Specify Measurable ResultsScale: The values you expect Natural Scale: •Response timein milliseconds Constructed: •1-10 ranking Proxy: •Projecting throughput using sample dataMeter: An appropriate way to measureMake Qualities Visible Include quality scenarios fordev& testing in your backlog Maintain a separate quality scenario backlog Include quality and functional acceptance tests as acceptance criteria for releases Part of the Roadmap Quality Radiators

10. 10/13/2014
10
Monitor Qualities—
Build An Operational Dashboard
Making Quality Happen
 Budget time for quality
discussions and quality testing
 During envisioning and requirements
gathering, identify core qualities
 Write quality scenarios
 Determine appropriate times when
qualities can be delivered and tested
 Turn quality scenarios into
detailed quality acceptance tests

11. 10/13/2014
11
Agile TeamsCross FunctionalGood CommunicationFocus on Stakeholders NeedsIncrementally deliver working softwareAdapt to Change as neededCollaborative and Self OrganizingWhole Team working togetherAgile Quality TeamsQA works closely with the team integrated during the day to day developmentEngage in QA activities much earlierWorks with whole-team including product owner on understanding and defining qualitiesAssists teams with definition & validation of important quality requirementsProactive working closely & coordinates between business, management and developers

12. 10/13/2014
12
Who will lead? Who contributes? Big Teams vsSmall Teams???? How does QA interact with development? Embedding QA with Teamaka “Pair with Quality Advocate” Great experience report at Agile this year AgileAlliance.org
New Experience Report posted:
Tearing Down the Walls: Embedding QA in a
TDD/Pairing and Agile Environment by Stephanie Savoia

13. 10/13/2014
13
Testing System Qualities Some require production or near-production environments Load and performance tests Complex quality stories involving interactions with several systems Some require extensive setup Some are “easy” and canbe part of an automated quality test suiteTwo Types of Quality TestsSimple Measurements A measurement of a quality that (ideally) can be automated (performance, load, …) Complex Quality Stories A complex set of actions that need to be verified as working. Hard to automate and might take a lot of setup (security, usability...)
“Users initiate 1,000 order transactions per minute under normal operations; transactions are processed with an average latency of 2 seconds.”
“A known, authorized user transfers money between accounts. The user is later (next day) identified as an embezzler by the institution they belong to and the system then restores funds to the original account.”

14. 10/13/2014
14
When to test? Determine Important Qualities that should be considered early Involve Product owner, architect, tech and QA leads in deciding when / how they should be tested Some qualities take several Sprints to develop Test qualities at most responsible moment Do not wait until the end!!! Some Sprints may focus primarily on Qualities…BUT THERE’S A LOT MORE TO TEST THAT YOUR SOFTWARE WORKS AS ADVERTISEDMost testers spend the majority of their time writing functional tests

15. 10/13/2014
15
System Quality Tests How does the system handle…? system load …? number of add note transactions/minute under normal load system support for…? simultaneously updating charts usability…? ease of locating and selecting notes Tests that emphasize architecture capabilities and tangible system characteristicsTesting System Qualities Usability Security Performance Scalability Internationalization Availability Flexibility Accessibility Location Regulation
Qualities to consider…

16. 10/13/2014
16
Agreeing on scenarios and target valuesSome Options… Toss out a reasonable number, then discuss to come to a consensus Average informed individuals’ estimates Use an existing system as baseline Values for similar scenarios Benchmark working code … There is more than “pass” or “fail” Landing Zone: Lets you define a range of acceptable values Minimal: OK, we can live with that Target: Realistic goal, what we are aiming for Outstanding: This would be great, if everything goes well

17. 10/13/2014
17
Agile Quality Driven Development Is… Practical. Testing system qualities can fit into and enhance your current testing Thoughtful. What qualities need to be tested? Who should write quality tests? When should you test for qualities? Realistic. You only have so much time and energy so test “Essential Qualities” Agile MindsetBe Agile

18. 10/13/2014
18
Thinking Fast vs. Slow Fast thinking: decisions based on intuition, biases, ingrained patterns, and emotions Slow thinking: Reasoning, logical thinking, contemplatingTake Time For Both Slow thinking Pairing and discussing options or why you want to implement something a certain way Sketching, noodling, design spikes Fast thinking Following your intuition, deciding on the fly Fast turns of coding, testing and quick fixes… (Red/Green)

19. 10/13/2014
19
Sustainable Architecture Stewardship Follow through Ongoing attention Not ignoring the little things that can undermine our ability to grow, change and adapt our systemsSustaining an Architecture Minimize architectural debt:Support the ability to change/adapt what needs to change Make what is too difficult, time consuming, or tedious easier Decide at the most responsible moment, not the last possible moment Learn and evolve
38
Keep the system “livable” for its users and developers

20. 10/13/2014
20
Agile Values Drive Architectural Practices Do something. Don’t debate or discuss architecture too long Do something that buys you information Prove your architecture ideas Reduce risks Make it testable Prototype realistic scenarios that answer specific questions Incrementally refine your architecture Defer architectural decisions that don’t need to be immediately made
Do
something!
Prove &
Refine. Indicators You’ve Paid Enough Attention to Architecture Defects are localized Stable interfaces Consistency Developers can easily add new functionality New functionality doesn’t “break” existing architecture Few areas that developers avoid because they are too difficult to work in Able to incrementally integrate new functionality

21. 10/13/2014
21
Continuous Inspection
Asian PLoP2014 PaperContinuous Inspection
Asian PLoP2014 Paper
CODESMELLDETECTION
METRICS(TESTCOVERAGE, CYCLOMATICCOMPLEXITY, TECHNICALDEBT, SIZES, …)
APPLICATIONSECURITYCHECKS
ARCHITECTURALCONFORMANCE
AUTOMATEWHEREYOUCAN!!!

22. 10/13/2014
22
Other Techniques for Improving QualitySteve McConnellhttp://kev.inburke.com/kevin/the-best-ways-to-find-bugs-in-your-code/
Average is 40% for any one technique!
Combining techniques gives you quality (> 90%) Summary Quality doesn’t just happen. It needs to be thought about and carefully considered Quality Scenarios are easy to write and read Writing quality tests drives out important cross-cutting concerns Measuring system qualities can require specialized testing/measurement tools If you don’t pay attention to qualities they can be hard to achieve at the last moment

23. 10/13/2014
23
Dogmatic
Pragmatic
Synonyms: bullheaded,dictative, doctrinaire, fanatical, intolerant
Antonyms: amenable, flexible, manageable
Synonyms: common, commonsense, logical, practical, rational, realistic, sensible
Antonyms: idealistic, unrealisticResources Agile Myths: agilemyths.com Patterns: hillside.net The Refactory: www.refactory.com Joe’s website: joeyoder.com TeamsThatInnovate.com Pragmatic TDD: refactory.com/training/test-driven-development http://adaptiveobjectmodel.com/2012/01/what-is- pragmatic-tdd/

24. 10/13/2014
24
Thanks!!! Questions???
joe@refactory.com
Twitter: @metayoda