Practitest webinar - When a test manager makes a difference
•Download as PPTX, PDF•
1 like•437 views
Due to the adoption of Agile, the role of test manager seems redundant. Teams and individual professionals are self-managing and coordinating their own work. Nevertheless, in some organizations and projects, there is a need for a quality ambassador that understand the agile quality strategy and can help, monitor and ensure that working solutions are delivered. In this webinar we will explore situations in which you as a test manager can make a difference. For example: How do we e.g. deal with compliance, integration and final acceptance in a contractor’s context? How does a test manager coordinate the work over teams and ensures the quality of the testing itself? Thanks Joel Montvelisky, PractiTest Chief Solution Architect, for hosting my webinar.
![2
“The only constant I am sure of
Is this accelerating rate of
change”
Source:[PeterGrabriel–downsideup]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Practitest webinar - When a test manager makes a difference
Similar to Practitest webinar - When a test manager makes a difference (20)
More from Derk-Jan de Grood
More from Derk-Jan de Grood (20)
Recently uploaded
Recently uploaded (20)
Practitest webinar - When a test manager makes a difference
- 1. When a Test Manager Makes a Difference Practitest Webinar Derk-Jan de Grood / Valori 26 September 2017
- 2. 2 “The only constant I am sure of Is this accelerating rate of change” Source:[PeterGrabriel–downsideup]
- 3. 3
- 4. Goals Find out where Test managers can add value in todays context 4
- 5. The Traditional Test manager What does the test manager do? 5
- 6. Traditional test managers spend time on… 6 Test Policy Test strategy (for the team, the project, the organization) Budget & Resourcing PRA Environment Planning new releases to test Release Advise Test report Progress report Fight for quality (with e.g. the project manager) Connect (with e.g. Release management) Defect Triage Staff tasks like Team lead, Education, etc. TPI Tooling and Automation
- 7. Let’s go Agile 7 Part of Development Done = Tested Team is responsible for testing
- 8. … Continues Integration and Deployment 8 Part of Development Done = Tested Team is responsible for testing Part of Delivery Pipeline Automated
- 9. Agile testing differs Continues Involvement Tools Essential Multi Dimensional Skills Com- munication Quick Feedback 9 Nishi Grover Garg
- 11. Workshop TestNet 11 How did your work change over the last 5 years?
- 12. Workshop TestNet 12 How did your work change over the last 5 years?
- 13. More Less Started Stopped Keep • Teamwork • Exploratory testing • Automation • Feedback customers • Global TS • Releases • Security • PRA • E2E testing • Cloud • Working individually • Detailed scripts • Manual Testing • Formal acceptance • Reporting and Logging • Test planning • Uniform tests • Test automation • Using tools • Real time test results • Integrated tests • Using Production Data • Value driven decision making • Data driven decision making • Agile SCRUM • Auditing • Open source + own build tools • Testers that can only test • Detailed test plan • Manual written test reports • PRA (while doing devops) • Quality Focus • Specifications • Drafting test cases based on specifications • Hardcore Testing • Keep learning • Waterfall • UAT 13 Test management tasks gone sour How did your work change over the last 5 years?
- 14. More Less Started Stopped Keep • Teamwork • Exploratory testing • Automation • Feedback customers • Global TS • Releases • Security • PRA • E2E testing • Cloud • Working individually • Detailed scripts • Manual Testing • Formal acceptance • Reporting and Logging • Test planning • Uniform tests • Test automation • Using tools • Real time test results • Integrated tests • Using Production Data • Value driven decision making • Data driven decision making • Agile SCRUM • Auditing • Open source + own build tools • Testers that can only test • Detailed test plan • Manual written test reports • PRA (while doing devops) • Quality Focus • Specifications • Drafting test cases based on specifications • Hardcore Testing • Keep learning • Waterfall • UAT 14 Test management tasks with value How did your work change over the last 5 years?
- 15. 15 THE TESTMANAGER There is no formal role for Test management in Agile BEWARE
- 16. Exploring in addition to checking How Quality is achieved in agile 16 Integration testing using virtualization Feature teams complete integrated features Testing is done in the sprint
- 17. Traditional test managers spend time on… 17 Test Policy Test strategy (for the team, the project, the organization) Budget & Resourcing PRA Environment Planning new releases to test Release Advise Test report Progress report Fight for quality (with …) Connect (with e.g. Release management) Defect Triage Staff tasks like Team lead, Education, etc TPI Tooling and Automation
- 18. Honesty Call 18
- 19. Workshop with Jan Jaap Cannegieter 19
- 20. Results of the dot voting 20 Developing Craftmanship Multiple teams Acceptance manager Integration Guarding Quality Specific tests like Performance/Security Compliance officer Manager Test pool Domain expert 0 5 10 15 20 25 30 35 40
- 21. Settings where TM adds value Meeting compliance and regulations Coordinating teams working on the same project Responsible for acceptance Responsible for the integration test Manager of a test pool Development of test craftsmanship 21
- 22. Meet our crew 22 Acceptance Multiple teamsIntegration
- 23. As overall test manager I ensure that all the teams in the project are working on the right tests So that we reduce overlap, prevent gabs and the teams work efficiently on the overall quality 23 Multiple teams
- 24. Tasks and activities • Define the overall quality strategy • Watch over the boundaries between teams • Perform risk analysis and monitor risks • Make sure the DoD’s of teams are in line with each other • Monitor test coverage • Monitor that all teams do (non-functional) testing professionally • Help teams to reach an acceptable level of testing • Tune test approach of the teams • Make sure end-to-end testing is done
- 25. Agile Quality Strategy 26 What needs to be tested Steering committee Dashboard/ Roadmapping Auditing the test work Coaching the testing team members Organizing tests that do not fit the sprint We still need a plan to know were complete There is a need feedback and traceability (comfort & adaptivety) Testing needs be of quality to justify itself Testing needs to add value to hold up an release
- 26. Benefits • Enhance overall quality (E2E) • More risk control • Create transparency (quality and progress) • Prevent double work or gaps • Take away bottlenecks • Efficiency – teams become more efficient • (NOT: more uniformity in the way teams test)
- 27. Acceptance 28
- 28. As Acceptance manager I ensure that we all parties focus on the right things and know what is needed for the formal acceptance of the solution So that we succeed our mission without late surprises 29 Acceptance
- 29. Acceptance manager When applicable • One system that support different parts of the organization • Not one PO that has the authority to decide • When working with different suppliers • Political environment Benefits • Different parts of the organization and suppliers are involved • Working towards a clear goal • Clearer acceptance criteria upfront
- 30. Tasks and activities • Elicitate the acceptance criteria • Make sure acceptance criteria are taken into account • Make sure acceptance criteria are used in testing • Stakeholder management • Keeping track on progress during the project • Stimulate collaboration between parties • Become a trusted wing partner
- 31. Experience: building an mobile network 32
- 32. 33
- 33. It ain’t done until its accepted 34 Witness Development en Tes ng Tested solu on Witness report Customer
- 35. 36 Transparency
- 36. As integration manager I ensure that we maintain a focus on integration of component and systems So that customer journeys can be completed and we actually deliver value 38 Integration
- 37. Integration Manager When Applicable • Project or changes that involve multiple systems • Organization is not yet completely on DevOps Benefits • Better Quality and user experience • Early detection of Integration problems • Sharing knowledge on system landscape • Bridging between teams • Better efficiency and saving money 39
- 38. Tasks and activities • Managing environments • Advising tooling and virtualisation • Version and configuration management • Scrum of Scrums • Communication and advising teams • Drafting customer journeys and scenario’s • Quality ambassador • Tracking progress, auditing test quality
- 39. 41 Experience story: Energy Market
- 40. Architecture • What are the business processes? • What are the components? • What are the interfaces? Acceptance criteria • What is the Minimal Viable Product? • What integrations are needed to make it work? Requirements traceability • When are we complete? • How do test results add up to acceptance? 42 Missing
- 41. What should a car minimally do? 43
- 43. Ensuring Integration (rough sketch) 45 Organization Component System Service Continuously (in the sprint) Occasionally (e.g. prior to a release)
- 44. Synergy 46
- 45. 47
- 46. Do’s and Don’ts (workshop results) 48 Do • Coach the teams • Build bridges • Give teams responsibilities • Create transparency • Align with e.g. PO’s • Create clear quality requirements • Talk with testers (1:1) • Be approachable • Help and Advice • Audit • Helicopter view • Be a quality ambassador Don't • Micro management • Telling people what to do • Interfere with team internal affairs • Short term focus • Tunnel vision • Combined functions (tester + manager) • Difficult complex reports
- 48. 50 Develop Craftsmanship As Practice lead I ensure that we exchange experiences and develop our profession So that we keep adding value and excel as testers
- 49. Development craftmanship When applicable • Big organizations with different testers • Testing important for the business • Organizations that invest in their employees Benefits • Enhanced craftmanship • Better, more professional testing • Happy testers • More happy customers • Save money and time 51
- 50. Tasks and activities • Coach testers • Organize knowledge sessions • Organize courses • Stimulate that testers visit conferences • Keep your own knowledge up-to-date • Test when necessary • Manage knowledge • Organize that testers share knowledge • Manage test community • Promote the importance of testing
- 51. Other crew members Compliance officer and Manager of the test pool 53
- 52. Finale 54
- 54. Role is changing 56 Traditional test managers spend time on… 15 Test Policy Test strategy (for the team, the project, the organization) Budget & Resourcing PRA Environment Planning new releases to test Release Advise Test report Progress report Fight for quality (with e.g. the project manager) Connect (with e.g. Release management) Defect Triage Staff tasks like Team lead, Education, etc TPI Tooling and Automation
- 55. Value as a test manager 5757 Acceptance Multiple teamsIntegration Develop Craftsmanship
- 57. We need less of them… Test manager Test Engineer 59
- 58. 60 Related roles
- 59. 61 So be good…. or grow into a new role
- 60. 62
- 61. Benefits of this webinar We looked at the activities of the Test manager the settings where a Test manager has value how to make a difference alternative roles 63
- 62. Derk-Jan Valori Coltbaan 4a 3439 NG NIEUWEGEIN The Netherlands • derkjandegrood@valori.nl • +31(0)651807878 • www.valori.nl • @DerkJanDeGrood • http://djdegrood.wordpress.com 64
Editor's Notes
- DevOps DevOps DevOps DevOps DevOps Lean Data-masking Data-masking IoT IoT IoT IoT IoT IoT Testbots Augmented-reality Crowd-sourcing Crowd-sourcing A/B A/B A/B A/B A/B TDD TDD TDD TDD TDD ATTD BDD BDD Agile/Scrum Agile/Scrum Agile/Scrum Agile/Scrum Agile/Scrum Agile/Scrum Agile/Scrum Programming CI/CD CI/CD CI/CD CI/CD CI/CD CI/CD CI/CD Micro Services Micro Services Blockchain Tooling Tooling Tooling Tooling ----- DevOps Lean Data masking IoT Test Bots Augmented reality Crowd sourcing A/B TDD ATTD DBB Agile/Scrum Programming CI/CD Micro Services Tooling
- https://www.agileconnection.com/article/5-ways-agile-testing-different-traditional-testing: Continuous Involvement In traditional projects, the test team works mostly in a silo, and there is little or no interaction needed with developers or other teams on a daily basis. But in agile, the test team is integrated with the Scrum team instead of being a separate unit. They need to be continuously involved in all aspects of the project, starting with the requirements and design of each feature. This makes the testers’ days busier with discussions, meetings, and interactions where they need to put forward their opinions. Agile necessitates that everybody report to the Scrum team first and their separate testing or development team later. In my experience as a part of a Scrum team, we would discuss our vacations or skill training needs first within the Scrum team, then inform our test manager afterward. Essential Tools Agile needs tool support more than traditional projects do because of the pace of development and continuous iterations. Each iteration brings along some regression work from the previous iterations that needs to be automated quickly. The same is true for test data generation, white-box testing tools, and static analysis tools, which become a necessity in an agile system. Given the time and quality constraints, performing white-box tests using control flow or data flow analysis, static analysis of code, or reviews for code and documents is no longer optional. Instead, it’s a mandate to prevent defects and ingrain quality into each work product. While in traditional projects, tools may be a luxury we might not be able to afford, in an agile project they become a necessity. Testers need these tools’ abilities in order to achieve their quality targets. When my team began our agile transformation, we had one automation engineer who would work on automation scripts for the entire project. But when pacing through the sprints, we quickly realized that one automation engineer was not able to keep pace with automating all features in all sprints. So, everybody started pitching in by scripting the features they tested, and eventually it was mandated. The automation expert was only used for framework-level implementation, and later he was absorbed as a functional tester. Due to frequent changes in the application within the sprint, we would follow an (n–1) approach to automation of our product, automating the features of the nth sprint in the next sprint, then subsequently using them for regression. But due to overload of regression building up as we progressed, it was crucial that each tester be able to perform and use the tools effectively, building up the test suites every sprint. Multidimensional Skills A traditional project has set expectations from the testers and testing activities. Each phase of testing gives set outputs, such as test design and specifications in the design phase, functional issues and defect reports in the execution phase, regression tests and retest results in reruns, and acceptance test reports in the end phase. This pattern does not leave much room for anything else because the project is already hard-pressed at the deadline. But an agile tester’s viewpoint covers not only the functional aspects of what he is testing, but also many broader aspects of the application. He need not be a performance testing expert to suggest during the design discussions that the design might not be able to support too many users. He may not be a usability expert, but he can suggest better ways to design the web form of their user story. He may not be a technical writer, but he may be required to review the installation guide steps. An agile tester has to have a broader perspective of quality in his project’s context and possess skills in all those areas. Effective Communication Agile requires effective communication among the team members at all times, and testers play a key role in establishing and maintaining that. For example, as a part of a Scrum team, a tester will be required to wear many hats: that of a requirement critic for the product owner, of a design reviewer for the developers and architects, of a functionality expert as a tester, and of a release adviser to the manager. Testers have to give their opinions and ideas at all stages of the project, which may be not required (or even much welcomed) in a traditional project. Testers act as the binding force of the team when they work in pairs with developers, sharing their test cases and ideas, as well as when they work as a quality reporter for the manager, with daily statistics and defect metrics for each sprint. The art of verbal and written communication is essential in an agile project. Quick Feedback from Testing The most important difference for agile testers is the quick feedback being given from the testing perspective at every point. The agile timeframes are shorter than on a traditional project, and testing needs to provide feedback about project quality on a regular basis. Daily stand-up meetings, design discussion or review meetings, the user story verification status, and sprint retrospectives all require constant feedback from testers. There is some additional pressure because the direction of the project gets defined by this feedback, and there is no final “quality gateway” at the end if the project does not meet the deadlines or quality goals. This keeps the testers on their feet at all times, unlike in traditional projects where testers are required only at the end of the project for a sign-off. Working in an agile environmnet can be challenging for testers coming from traditional projects, but by being open, flexible, and adaptive, they will find that an agile team is a wonderful place to be a tester.
- Interessant zijn: Value driven decision making -> heb je hier een TM voor nodig? PRA wordt meer gedaan? Testers that can only test + keep learing -> compentece lead Less formal acceptance en meer releases: maar in sommige omgevingen werkt dit anders Stopped: detaile dtestpplan and manual writen reports Real time resultd – manual written reports Actie: highlight een paar combies en concludeer dat TM minder wordt
- Interessant zijn: Value driven decision making -> heb je hier een TM voor nodig? PRA wordt meer gedaan? Testers that can only test + keep learing -> compentece lead Less formal acceptance en meer releases: maar in sommige omgevingen werkt dit anders Stopped: detaile dtestpplan and manual writen reports Real time resultd – manual written reports Actie: highlight een paar combies en concludeer dat TM minder wordt
- Meerwaarde : Inzage in kwaliteit Voorkomen gaten/dubbel werk Verhoging kwaliteit (E2E) Centrale aanspreekpunt Efficiëntie - teams worden efficiënter Kennisdeling wordt makkelijker Klanttevredenheid verhogen Tevredener medewerkers Test-techniek wordt beter Betere risico beheersing Mensen samen brengen (bijv. klanten/stakeholders/teams) Teams uit hun bubbel brengen Weghalen bottlenecks
- https://djdegrood.wordpress.com/2014/07/17/cdma-lianders-private-wireless-network/https://djdegrood.wordpress.com/2014/07/17/cdma-lianders-private-wireless-network/
- The way to avoid the second scenario is by putting into place witness process Set of goals that need to be reached by 4 activities. Extensive reviewing Witnessing is an ongoing phase and is all about transparency and working together. Avoid surprises, mitigate risk, clear understanding on quality0 Audit: De leverancier toont aan dat: checklist van uit te voeren activiteiten, -> paper trail klopt! Bewijslast ligt bij leverancier. Demo: periodieke inzage in kwaliteit van product. Toont test werkzaamheden van afgelopen periode, testscenario’s worden afgespeeld en inzicht wordt gegeven van de belangrijkste bevindingen (wat werkt nog niet). -> Demo rapport! Meetings: Strategisch testoverleg, Finding Meeting, Demo -> Kwaliteitsoverzicht Review: Testplannen, testcases, Testresultaten, Non functional testen.
- How to ensure Integration We do: CI/CD MBT UT Automated System test Automated e2e test Interface testing Manual Regression testing Integration sprints other
- When we need to take of so much items, wide spread, do we believe it will work out itself, What do we need? Yes a test strategy
- + Opbouwende kritiek + Samenbrengen teams + Helicopterview over ketens heen Kortzichtige blik: dus naar de toekomst kijken Micro management Tunnelvisie ------ + Coached optreden + Verantwoordelijkheid binnen teams leggen + Zorg voor afstemming & coördinatie tussen product owners + Maak kwaliteitseisen eenduidig + Wees toegankelijk + Praat regelmatig 1:1 met testers + Audit op gezette tijden + Bewustzijn kwaliteit en testen verhogen onder niet-testers Geen dubbele functie (tester + manager) Geen lange teksten & ingewikkeld verhaal bij test rapportage Testen herhalen (ook bij andere teams) Mensen vertellen wat & hoe ze het moet doen ----- Niet bemoeien met interne werkzaamheden in het team + ondersteunen indien men het vraagt + Adviseren bij een acceptatieaanpak + Testplannen maken buiten het team