Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

QA Fest 2017. Влада Бенюх.Организация End-to-End тестирования в cross-integration команде

312 views

Published on

В моем докладе я планирую рассказать что такое End-to-End тестирование в рамках “cross integration” команды, покрыть основные вопросы и проблемы, риски и помехи, которые могут возникнуть во время организации процесса тестирования для такой команды. Дам практические рекомендации как исправить и/или улучшить процесс, основываясь на реальном примере проекта.

Published in: Education
  • Be the first to comment

  • Be the first to like this

QA Fest 2017. Влада Бенюх.Организация End-to-End тестирования в cross-integration команде

  1. 1. Киев 2017 Первый в Украине фестиваль тестирования E2E testing: Challenges and recommendation in test process organization for cross-integration teams Vlada Benyukh
  2. 2. Киев 2017 Первый в Украине фестиваль тестирования Speaker info Vlada Benyukh, Ukraine QA Manager/QA Consultant at Ciklum, TCoE •7 years in QA •Manual QA Engineer, QA Manager, mentor •Certified: SAFe, ICAgile, ISTQB in/vlada-benyukh
  3. 3. Киев 2017 Agenda 1. E2E testing in cross-integration teams, What is it? 2. Test process and approach 3. Infrastructure management 4. Collaboration 5. Test team structure 6. Automation setup 7. Questions E2E testing: Challenges and recommendation in test process organization for cross integration teams
  4. 4. Киев 2017 What is E2E testing? Main goal is to identify system dependencies and to ensure that the data integrity is maintained between various applications/components correctly E2E testing: Challenges and recommendation in test process organization for cross integration teams
  5. 5. Киев 2017 What is E2E testing in cross- integration teams? Expectation E2E testing: Challenges and recommendation in test process organization for cross integration teams
  6. 6. Киев 2017 What is E2E testing in cross- integration teams? Process and approach Infrastructure management Communication and collaboration Most of the challenges in a E2E team relates to cross-integration work: E2E testing: Challenges and recommendation in test process organization for cross integration teams
  7. 7. Киев 2017 Test Process challenges E2E testing: Challenges and recommendation in test process organization for cross integration teams
  8. 8. Киев 2017 Test Process challenges – Team structure E2E testing: Challenges and recommendation in test process organization for cross integration teams
  9. 9. Киев 2017 Why team structure was a challenge?  3 QA Automation Specialists  1 Automation QA Lead E2E testing: Challenges and recommendation in test process organization for cross integration teams
  10. 10. Киев 2017 1. Understanding of the systems flows 2. Accesses to the systems 3. Requirements for end-to-end flows 4. Test Design for end-to-end flows 5. Test data for each system 6. Test environment readiness Why team structure was a challenge? E2E testing: Challenges and recommendation in test process organization for cross integration teams
  11. 11. Киев 2017 Do not change team structure until challenges resolved… Why team structure was a challenge? E2E testing: Challenges and recommendation in test process organization for cross integration teams
  12. 12. Киев 2017 Test Process challenges – Solution 1. Systems:  7 separate systems which integrated into 1 CRM  1 common aggregation level for all systems  1 common integration level for all systems E2E testing: Challenges and recommendation in test process organization for cross integration teams 2. Many development and test teams (in different countries and time zones)
  13. 13. Киев 2017 Test Process challenges – Solution Gather knowledge about the systems!  Get access to the systems  Understand systems specific and functionality  Get information about architecture overview and technologies used  Figure out scope and readiness dates of the systems E2E testing: Challenges and recommendation in test process organization for cross integration teams
  14. 14. Киев 2017 Test Process challenges – Solution Gather knowledge about key representatives in each system!  System teams  Integration development system teams E2E testing: Challenges and recommendation in test process organization for cross integration teams
  15. 15. Киев 2017 Ask key representatives about end-to-end flows Gather, document and approve acceptance criteria Prioritize acceptance criteria Requirements challenges – What was the plan? E2E testing: Challenges and recommendation in test process organization for cross integration teams
  16. 16. Киев 2017 No understanding and goal of end-to-end team No requirements for end-to-end flows No responsible person for end- to-end team Requirements challenges – What was in reality? E2E testing: Challenges and recommendation in test process organization for cross integration teams
  17. 17. Киев 2017 Requirements challenges How to get requirements … E2E testing: Challenges and recommendation in test process organization for cross integration teams
  18. 18. Киев 2017 Get system requirements Explain the goal of end-to-end testing Review system requirements Document, prioritize and approve with business side Requirements challenges – Solution E2E testing: Challenges and recommendation in test process organization for cross integration teams
  19. 19. Киев 2017 Requirements challenges – Summary E2E testing: Challenges and recommendation in test process organization for cross integration teams 1. General knowledge about system were gathered 2. Scope and priorities for end-to-end were identified 3. Detailed Requirements for end-to-end flows were documented
  20. 20. Киев 2017 Test Design challenges E2E testing: Challenges and recommendation in test process organization for cross integration teams
  21. 21. Киев 2017 • No technical skills to create test documentation based on the best practices • No input test data for each system Test Design challenges - Problem E2E testing: Challenges and recommendation in test process organization for cross integration teams
  22. 22. Киев 2017 Test Design challenges – Solution • Explain the necessity of manual QA specialists to senior management • Involve manual QA Engineers from system teams in test cases creation • Motivate automation team to collaborate with manual QA Engineers E2E testing: Challenges and recommendation in test process organization for cross integration teams
  23. 23. Киев 2017 How motivate automation team to create test cases? Test Design challenges – Solution E2E testing: Challenges and recommendation in test process organization for cross integration teams
  24. 24. Киев 2017  Explain the temporarily situation and what can happen without end-to-end team involvement  Be involved and continue to work on automation direction together Test Design challenges – Solution E2E testing: Challenges and recommendation in test process organization for cross integration teams
  25. 25. Киев 2017  Ask involved Manual QA Engineers to be responsible for test cases creation  Define test case template which will be used by manual and automation QAs  Structure test cases by functionality and priorities Test Design challenges – Solution E2E testing: Challenges and recommendation in test process organization for cross integration teams
  26. 26. Киев 2017 Test Design challenges – Summary E2E testing: Challenges and recommendation in test process organization for cross integration teams 1. Responsible team members for test cases creation were defined 2. Structure and template were identified 3. Flows for end-to-end were covered by test cases
  27. 27. Киев 2017 Testing approach agreement across teams Agreements should be aligned before starting test execution activities E2E testing: Challenges and recommendation in test process organization for cross integration teams
  28. 28. Киев 2017 Testing approach agreement across teams Discuss process/approach with teams Involve all stakeholders in process/approaches discussions Align procedures across teams Document all agreements E2E testing: Challenges and recommendation in test process organization for cross integration teams
  29. 29. Киев 2017 Testing approach – Testing Order Smoke Acceptance critical flows Regression UAT E2E testing: Challenges and recommendation in test process organization for cross integration teams
  30. 30. Киев 2017 Testing approach – Testing Order Type of Test Scope Outcome Smoke One End-to-end flow for most critical functionality Blocker bugs Acceptance critical flows End-to-end flows P1/P2/P3/P4 bugs Regression P1/P2 P1/P2 test cases P1/P2 bugs Regression P3/P4 P3/P4 test cases P3/P4 bugs UAT User Acceptance Testing P1/P2 and P3 bugs E2E testing: Challenges and recommendation in test process organization for cross integration teams
  31. 31. Киев 2017 Testing approach – Defect management RAISE ON E2E BOARD DISCUSS ON BUG TRIAGE DUPLICATE AND ASSIGN TO A PROPER TEAM ANALYSE CASE WHEN DEFECT HAPPENED E2E testing: Challenges and recommendation in test process organization for cross integration teams
  32. 32. Киев 2017 Test Execution challenges - Solution • Ask Manual QA representatives focus their attention on execution activities from manual point of view • Involve Automation QA representatives in technical details analysis to find a workaround for infrastructure challenges !!! Infrastructure challenges will be covered on the next section!!! E2E testing: Challenges and recommendation in test process organization for cross integration teams
  33. 33. Киев 2017 Infrastructure challenges E2E testing: Challenges and recommendation in test process organization for cross integration teams
  34. 34. Киев 2017 Test environment requirements for verification of end-to-end flows:  All items should be processed from start system to end system automatically Infrastructure challenges E2E testing: Challenges and recommendation in test process organization for cross integration teams
  35. 35. Киев 2017 But what we had…. Infrastructure challenges E2E testing: Challenges and recommendation in test process organization for cross integration teams
  36. 36. Киев 2017 Based on the system configurations, limitations and application specific, items are integrated from start system to end application into two different ways:  Semi-automatically  Non-automatically Infrastructure challenges - Problem E2E testing: Challenges and recommendation in test process organization for cross integration teams
  37. 37. Киев 2017 Automation QA team was involved to resolve next items:  Data formats and mappings for items processing  Access to file transfer folders which can help to make items processed to end system  Get permissions to monitor messages to diagnose the problem Infrastructure challenges - Solution E2E testing: Challenges and recommendation in test process organization for cross integration teams
  38. 38. Киев 2017 Infrastructure challenges – semi- automatic procedure Creation Status Verification Verification in CRM Automation QAsManual QAs • Create items in start system • Verify that details about created items are stored properly • Track the processing status and monitor issues • Verify that item details are displayed in end system E2E testing: Challenges and recommendation in test process organization for cross integration teams
  39. 39. Киев 2017 Infrastructure challenges – non- automated procedure Creation Status Verification Verification in CRM • Create items in start system • Verify that details about created items are stored properly • Generate data formats which required for processing • Verify that item details are displayed in end system Manual Processing • Process and track the processing status and monitor issues Automation QAsManual QAs E2E testing: Challenges and recommendation in test process organization for cross integration teams
  40. 40. Киев 2017 Infrastructure challenges – Summary E2E testing: Challenges and recommendation in test process organization for cross integration teams 1. End-to-end flows were executed successfully 2. End-to-end scenarios found bugs and discrepancies in the systems functionality
  41. 41. Киев 2017 Collaboration challenges E2E testing: Challenges and recommendation in test process organization for cross integration teams
  42. 42. Киев 2017 Collaboration challenges – Cross teams What is it if NO: 1. Frequency and scope for updates 2. Release readiness dates 3. Changes in the systems 4. Possible impact and risks for end-to-end flows 1. Unexpected issues occur while testing 2. Possibly unexpected configuration issues 3. Dependencies are not tracked and mitigated in time E2E testing: Challenges and recommendation in test process organization for cross integration teams
  43. 43. Киев 2017 How to avoid issues? Collaboration challenges – Cross teams E2E testing: Challenges and recommendation in test process organization for cross integration teams
  44. 44. Киев 2017 Regular cross-teams Sync Ups Participants:  Project management teams representatives  Integration Program Manager  Team Tech Leads  E2E QA Lead Goals:  Define scope for updates  Get system updates dates  Analyze and mitigate risks that might affect E2E testing before testing starts  Escalate critical problems  Discuss other common problems and questions E2E testing: Challenges and recommendation in test process organization for cross integration teams
  45. 45. Киев 2017 Collaboration challenges – cross- teams progress understanding What is it if NO: 1. Scope and activities for end-to-end testing 2. Understanding of the progress end-to-end testing 3. Risks and blockers for end-to-end testing 4. Release readiness dates E2E testing: Challenges and recommendation in test process organization for cross integration teams
  46. 46. Киев 2017 How to avoid issues? Collaboration challenges – cross- teams progress understanding E2E testing: Challenges and recommendation in test process organization for cross integration teams
  47. 47. Киев 2017 Regular E2E team Sync Up Participants:  Integration Program Manager  E2E QA Lead  E2E key QA specialists Goals:  Show the progress  Escalate critical problems  Discuss other common problems and questions E2E testing: Challenges and recommendation in test process organization for cross integration teams
  48. 48. Киев 2017 Example of Daily Progress Visibility E2E testing: Challenges and recommendation in test process organization for cross integration teams
  49. 49. Киев 2017E2E testing: Challenges and recommendation in test process organization for cross integration teams Example of Daily Progress Visibility
  50. 50. Киев 2017 Example of Daily Progress Visibility E2E testing: Challenges and recommendation in test process organization for cross integration teams
  51. 51. Киев 2017 Collaboration challenges – Summary E2E testing: Challenges and recommendation in test process organization for cross integration teams 1. Regular updates on system updates and readiness were completely clear 2. Risks were identified and mitigated regularly 3. Critical problems were escalated in time 4. Test progress and project health were synchronized between teams
  52. 52. Киев 2017E2E testing: Challenges and recommendation in test process organization for cross integration teams Test Team structure
  53. 53. Киев 2017 QA Lead - QA Process and Approaches implementation - Cross team ongoing QA work planning and monitoring - Regular QA reporting - People management between teams - Support in building and aligning QA automation approach Manual QAs -Main knowledge keepers in the systems - Responsible for test cases preparation and execution semi- automated procedure - Escalate issues - Define dependencies and quality risks. Inform about them in time Automation QAs - Support in test cases creation - Responsible for incomplete test procedure processing testing - Responsible for test automation implementation - Escalate issues - Define dependencies and quality risks.Inform about them in time Test Team structure - Solution E2E testing: Challenges and recommendation in test process organization for cross integration teams
  54. 54. Киев 2017 QA Lead QA Manual QA AutomationQA Manager QA Manual Lead QA Manual Lead Middle QA Middle QA Senior QA Middle QA Middle QA Senior QA Senior QA Automation team Test Team structure - Solution Team 1 Team 2 E2E testing: Challenges and recommendation in test process organization for cross integration teams
  55. 55. Киев 2017 Automation challenges E2E testing: Challenges and recommendation in test process organization for cross integration teams
  56. 56. Киев 2017 Automation challenges – What was it? 2. Complex and obsolete architecture 1. End-to-end flow is not fully automated 3. Many external and internal dependencies E2E testing: Challenges and recommendation in test process organization for cross integration teams
  57. 57. Киев 2017 Automation challenges – What was solution? 1. Get access to systems via API to create items 2. Generate required data formats and make to be processed it automatically 3. Automate validation of items in common CRM E2E testing: Challenges and recommendation in test process organization for cross integration teams
  58. 58. Киев 2017 Automation challenges – Order to automate Smoke Regression Acceptance flows E2E testing: Challenges and recommendation in test process organization for cross integration teams
  59. 59. Киев 2017 Recommendations 1. Do not change team structure until all important challenges were not clarified 2. Be proactive and look for the solutions 3. Do not try to resolve all issues by yourself 4. Collaborate with stakeholders and systems team members E2E testing: Challenges and recommendation in test process organization for cross integration teams
  60. 60. Киев 2017 Questions E2E testing: Challenges and recommendation in test process organization for cross integration teams

×