Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Keynote: Testing and Quality in the Scaled Agile Framework for Lean Enterprises (SAFe 4.6 version)


Published on

Scaled Agile Framework for Lean Enterprises (SAFe) is the most popular framework used by large programs and companies to achieve business agility. It challenges how testers, Q&A and test managers, and test specialists work together with the entire organisation. Derk-Jan de Grood and Mette Bruhn-Pedersen explain how test professionals can contribute with their knowledge and expertise in a SAFe organisation.

Scaled Agile Framework for Lean Enterprises (SAFe) is becoming the most popular framework used to help large programs and entire companies achieve business agility. It builds on well-known agile-lean principles and methodologies, and puts them together to address challenges not only on team level, but also on program, large solution and portfolio level. Although the framework is described in more details than other comparable frameworks it is not very elaborate on how testing and quality practices fit in. This raises new challenges for testers, Q&A and test managers, test architects, test specialists and people in similar roles together with the entire organisation. Derk-Jan de Grood and Mette Bruhn-Pedersen describe what guidance SAFe actually provides and suggest additional ways test and QA professionals can contribute with their special knowledge and expertise.

The presentation is based on the authors’ experience and an expert session at the 22nd Testing Retreat (2017, Hereford UK). The Testing Retreat is a peer-conference where leaders in the testing and quality profession share their experiences and insights. The participants discussed quality measures and actions that test professionals can take to ensure quality when using SAFe throughout the Software Development Lifecycle (SDLC). We thank all for their valuable contributions.

Published in: Technology
  • Be the first to comment

Keynote: Testing and Quality in the Scaled Agile Framework for Lean Enterprises (SAFe 4.6 version)

  1. 1. Mette Bruhn-Pedersen (Safe Journey)
 Derk-Jan de Grood (Valori) Testing and Quality in the Scaled Agile Framework for Lean Enterprises Keynote on the TestNet Fall Event 2018 10-10-2018 The Netherlands
  2. 2. Mette Bruhn-Pedersen 2 QA & Testing +10 years of software testing and QA experience • Testing • Test Management +10 years in DSTB and ISTQB • Chair of DSTB • ISTQB Release Manager • ISTQB Governance Officer • Vice Chair of ISTQB Examination Working Group +10 years of agile experience • Agile Team Member • Test Manager +4 years of Scaled Agile Framework • Agile Team Member • Community of Practice (CoP) Facilitator • Agile Coach • Agile Transformation Leader Agile
  3. 3. Derk-Jan de Grood 3 QA & Testing Helping organisations to get more 
 out of testing and embedding quality in the organisation 20+ years of Testing Test strategies & management Author, Trainer & Speaker Testing Excellence Award 2016 Helping organisations with embracing agile, increasing the value stram and embed the way of Working in the organisation Agile Transition Coach Team coach Workshop facilitator & trainer Author of “Starting with Scrum” Agile
  4. 4. Key takeaways • Test and QA professionals can add value at all organisational levels in ways that are (not) described in SAFe. • Defining a quality strategy is key to handle typical issues with test and QA activities spanning multiple teams and even multiple Release Trains. • Getting quality and testing anchored on Portfolio Level ensures that quality can be built-in from top to bottom. 4
  5. 5. Why Scale your Agile? 5
  6. 6. Business agility is key as disruption of sectors and industries becomes the new normal. 6 Organiza(ons more and more start to understand that business agility and responsiveness is a key factor to survive and stay ahead of their compe(tors. In order to yield value, the work of single agile teams should therefore be integrated and embedded in larger business processes. The adop(on of agile is shi?ing from a single team focus to a wider organiza6onal approach
  7. 7. TestNet Fall Event 2018 Transformation Challenges 7 • Body Level One • Body Level Two • Body Level Three • Body Level Four • Body Level Five Agile difficult to implement Integrating non-development functions Change resistance Requirements engineering challenges Hierarchical management and organizational boundaries Lack of investment Coordination challenges in multi-team environment Different approaches emerge in a multi-team environment Quality assurance challenges 0 5 10 15 20 25 30 35 40 45 50 Percentage of case organisations * Challenges and success factors for large-scale agile transformations: A systematic literature review, Digert, K. et al, The Journal of Systems and Software, 119 (2016) 87–108 Challenge categories
  8. 8. Agile Frameworks show the way ! 8
  9. 9. DAD SoS Agile Methodologies at a Glance 9 Scrum XP Kanban Spotify LeSS SAFe® Scalability Prescriptiveness Nexus DSDM Information combined from several sources: Agile Scaling Knowledgeable (ASK), Agile Scaling, 12th annual State of Agile Report, Collabnet VersionOne, World Quality Report 2017-18, Ninth Edition, Capgemini, Micro Focus, Sogeti, Kanban and Scrum making the most of both, Henrik Kniberg & Mattias Skarin, 2010, C4Media
  10. 10. SAFe® Portfolio Big Picture 10See latest description on
  11. 11. 11 TestNet Fall Event 2018
  12. 12. Quality in SAFe 12
  13. 13. SAFe 4.6 13 TestNet Fall Event 2018
  14. 14. What does SAFe say about testing 14 SAFE 4.6 SAFe still says little about: • Roles in QA & testing Increased guidance on quality and testing in the new version 4.6 to support technical agility: • Emphasis on using Behaviour-Driven Development (BDD) • Include more types of testing in the sprint • Modeling stories • Design with quality • Implement with quality
  15. 15. Agile Software Engineering 15 TestNet Fall Event 2018
  16. 16. Check for yourself what’s new ! TestNet Fall Event 2018 16 new-in-safe-46/
  17. 17. Team Layer 17 Strive for a continues flow of integrated quality solutions
  18. 18. What SAFe says: Build in Quality 18
  19. 19. SAFe Practises 19 Guidance in SAFe Practices Collective Ownership Advocate that anyone can adopt and refactor code, that code is understandable for everyone and quality is a collective responsibility Unit Testing & Test-First Approach Advocate, pilot and contribute to TDD, ATDD and BDD Pair Work Pair with e.g. Users, Developers, PO, BA Automate CI/CD Focus on Ci/CD in SAFe 4.6
  20. 20. Focus on CI/CD in SAFe 4.6 20 TestNet Fall Event 2018
  21. 21. What to test… 21 TestNet Fall Event 2018 Process of creating executable code - Syntax of source code is correct User Acceptance Test - System supports customer journey Exploratory tests - Validation check Integration test - System works within system of other systems Functional and NF tests - System delivers value as intended - System complies to capacity, security, availability, etc. requirements Smoke test and deployment test - Systems is correctly and installed and configured on target environment Configuration test - 1: ensure references to external service in configuration setting are OK - 2: Run smoke test Code Quality Criteria - e.g. test coverage and code metrics Unit testing - Code behaves as expected
  22. 22. Review the DOD • Define quality measures • Identify tests outside the sprint • Cross-team alignment Or more practical… 22
  23. 23. Or more practical… 23 Story impacts Programmer Tester User Web interface • • Calculation • • Business rule • • Letter/e-mail • • TEST MATRIX AS STRATEGY ON USER STORY LEVEL Feedback loop: found issues to improve testing Focus on NFTs Refinement: Discuss examples, exceptions and raise the “how to demo/test” question
  24. 24. Program Layer 24 Divide work between individual teams and merge completed features into a continuous delivery pipeline. The Agile Release Train (ART) bundles the work of the teams into controlled product increments.
  25. 25. Brian Marick’s Agile Testing Matrix 25
  26. 26. Test Strategy 26 Explain how test activities contribute to build a quality solution Ensure that the right tests are done (at the right time, by the right team) Assure that the testing is of sufficient quality (coaching and auditing) Explain how to gather intelligence on progress and quality
  27. 27. The last sprint of the PI 27
  28. 28. The last sprint of the PI 28 • Participate in product or technical experiments • Root Cause Analysis • Identify technical and business risks • Improve development practises • e2e integration starting point for planning Integration & Acceptance
  29. 29. Other suggestions 29 Operational andorganisationalreadiness Governance Emphasize the quality mind- set so that integration testing is taken into account at PI Planning Test environments
  30. 30. Large Solution Layer 30 Extending Built-In Quality Practices, Support system and solution integration, continuous compliance verification
  31. 31. 31 TestNet Fall Event 2018 See latest description on
  32. 32. Build Quality and Compliancy in 32 TestNet Fall Event 2018 See latest description on
  33. 33. 33 TestNet Fall Event 2018 See latest description on
  34. 34. There is value in the Solution Layer 34 TestNet Fall Event 2018 See latest description on • Not many Organisations use the Solution layer • SAFe offers solutions for embedding Quality • Benefit is gained by using the solution layer practises • Testers could emphasis and direct the organisations towards what is available in the framework • There are good things that are useful even when you don’t use the large solution layer
  35. 35. Portfolio Layer 35 Translate strategic themes into value streams and Epics
  36. 36. If we want to embed quality in to the organisa(on, it should be done at this level. 36
  37. 37. 37 In Order to embed quality in our development practices we are looking for an CHIEF QUALITY OFFICER (M/F) He/She wil be responsible for………
  38. 38. 38 In Order to embed quality in our development practices we are looking for an CHIEF QuALITY OFFICER (M/F) He/She wil be responsible for……… Prioritise Compliance Epics Defines KPIs to assess UX and Value streams Defining strategic themes with a quality focus
  39. 39. Our Quality Hero on C level • The Chief Quality Officer is a representa(on of quality management within the company. • Keeps quality in the forefront of the company • Is a strong advocate for the quality process. • Supports the QA team and stresses the importance of quality in the organisa(on. • Ensures that metrics such as Cost of Quality, Rework Rates, and equipment u(lisa(on and (me-to-market enhances the transparency of development and opera(ons. • Defines quality themes at a strategic level (all the way up to the SAFe PorPolio level) • Strives for sustainable quality of delivery and development (and even beRer constantly improving) 39
  40. 40. Next steps ! 40
  41. 41. More info…… 41 Download at https:// download-testing-quality-scaled- agile-framework-lean-enterprises/ Grab a copy of the Bits&Chips magazine Scaled Agile Insider Must Read
  42. 42. • Body Level One • Body Level Two • Body Level Three • Body Level Four • Body Level Five 42 Frameworks develop themselves (e.g. SAFe 5.x) Test Community defines Add- Ons
  43. 43. Derk-Jan de Grood Thoughtleader Testing & Agile Transition Coach E: W: Contact Information 43 Mette Bruhn-Pedersen Agile Transformation Leader and Partner E: W: